Machine Learning Learning Beyond Point Predictions: Uncertainty Quantification

About the bookWritten for practitioners and researchers with a strong mathematical background, this book offers a solid guide to uncertainty quantification in supervised machine learning. It explores both aleatoric and epistemic uncertainties, providing practical tools to improve the reliability of predictions in applications ranging from cosmology to infectious disease forecasting. The topics covered include conditional density estimation with mixtures and neural networks, Gaussian processes, Bayesian Additive Regression Trees, conformal prediction, calibration techniques for regression and classification, likelihood-free inference, and more. By combining theory with real-world examples, the book helps readers move beyond point predictions to create models that effectively communicate uncertainty. Hands-on examples and code are available in an accompanying GitHub repository.About the authorRafael Izbicki is a faculty member in the Statistics Department at the Federal University of São Carlos (UFSCar) in Brazil. He holds a bachelor's and a master’s degree in Statistics from the University of São Paulo and a Ph.D. in Statistics from Carnegie Mellon University. Rafael’s research focuses on uncertainty quantification, statistical theory, foundations of statistics, and machine learning, with applications across fields like cosmology, epidemiology, and biology. He has published numerous papers in leading journals and has held a CNPq Research Fellowship since 2017. For more information, visit rafaelizbicki.com. Table of ContentsIntroduction1.1 Motivating Example: Life Expectancy and GDP1.2 Aleatoric and Epistemic Uncertainty1.3 Applications of UQ in ML1.4 Outline of the BookReview of Supervised Learning2.1 Notation and Assumptions2.2 Loss Functions and Risk2.3 Model Selection: Overfitting and Underfitting2.4 Bias and Variance Tradeoff2.5 Tuning Parameters2.6 Methods to Create Prediction FunctionsQuantifying Aleatoric Uncertainty with Conditional Densities3.1 Loss Functions3.2 Probabilistic Classifiers3.3 Parametric Approaches3.4 FlexCode3.5 Mixture Models and Networks3.6 Normalizing Flows3.7 The Ratio Trick3.8 Other Conditional Density Estimators3.9 Quantile Regression3.10 Simulated Example: Gaussian Distribution3.11 Example: Twitter Location PredictionDiagnostics and Recalibration4.1 PIT Values: Evaluating Calibration in Regression4.2 Conditional PIT Values4.3 Calibration of Classification ModelsFrom Conditional Densities to Prediction Regions5.1 Optimal Prediction Regions5.2 Plug-in Prediction Regions5.3 Conformal Regions5.4 Comparing Methods5.5 SummaryCapturing Epistemic Uncertainty through Bayesian and Ensemble Techniques6.1 Bayesian Models6.2 Gaussian Process Regression6.3 Bayesian Additive Regression Trees (BART)6.4 Monte Carlo Dropout6.5 Deep Ensembles6.6 The BootstrapPhotometric Redshift Prediction7.1 Vera C. Rubin Observatory7.2 Southern Photometric Local Universe Survey (S-PLUS)Disease Surveillance: Dengue and COVID-198.1 Dengue Nowcasting8.2 COVID-19 Hospitalizations and Vaccination ImpactLikelihood-Free Inference (LFI)9.1 Approximate Bayesian Computation via Conditional Density Estimation (ABC-CDE)9.2 Examples9.3 Likelihood-Free Frequentist Inference (LF2I)9.4 SummaryOptimizing Construction Schedules: Mitigating Weather-Related Delays10.1 Estimating the Distribution10.2 Model SelectionClosing Thoughts11.1 Summary of Key Concepts11.2 Limitations of UQ Approaches11.3 Practical Recommendations Read more