Information-Based Calibration Cuts Overestimation in Product-of-Expert Gaussian Processes

On August 15, 2026, researchers Yean Hoon Ong, Paolo Barucca, Wei Pan and Jun Wang published a new approach to improve the reliability of product‑of‑experts Gaussian process (GP‑pro) models by calibrating their posterior variance estimates based on information gain properties. The study appeared in the Journal of Artificial Intelligence Research (JAIR). ¹

The motivation behind the work lies in the computational advantages of GP‑pro architectures compared to single global Gaussian processes, which suffer from cubic time complexity as data size grows. Because each local expert is trained independently on disjoint subsets of the training set, GP‑pro often produces overly conservative uncertainty estimates—posterior variances are systematically higher than expected. This phenomenon can lead to suboptimal decision boundaries when such models are deployed in real‑world applications. The authors formalised this issue and proposed an explicit correction mechanism called GP‑pro‑c. ¹

GP‑pro‑c incorporates an information‑based calibration ratio derived from monotonicity and submodularity properties of mutual information in Gaussian processes. By adjusting the variance contribution of each local expert according to how much unique information it brings, the method reduces overestimation while preserving the overall predictive mean. The paper details the mathematical derivation of the calibration factor and demonstrates that it can be computed with negligible additional cost relative to standard GP‑pro inference. ¹

To evaluate the new model, the team ran experiments on several benchmark datasets commonly used for regression tasks. They measured negative log‑likelihood (NLL) and root mean squared error (RMSE). Across all tests, GP‑pro‑c consistently outperformed vanilla GP‑pro by reducing NLL values and tightening RMSE scores, indicating both better calibrated uncertainties and more accurate predictions. ¹

From an expert perspective, the work addresses a well‑known limitation of product‑of‑experts models: independent training leads to inflated variances. Similar attempts at variance correction were explored by Neal in 2006, but those approaches did not specifically target the product‑of‑experts architecture that has become increasingly popular since its introduction around 2015. By leveraging intrinsic properties of Gaussian processes—monotonicity and submodularity—the authors provide a principled way to recalibrate local contributions without sacrificing computational efficiency. This could enable broader adoption of GP‑pro models in safety‑critical domains where reliable uncertainty quantification is essential.

The paper also discusses potential extensions, such as applying the calibration scheme to other ensemble GPs or integrating it with sparse approximations for even larger datasets. While the current evaluation focuses on regression tasks, future work may explore classification scenarios where posterior variance directly influences decision thresholds. Overall, the study offers a practical tool for practitioners who rely on product‑of‑expert Gaussian processes while demanding trustworthy confidence estimates.