Bayesian Deep Learning

The advent of deep learning over recent years has revolutionized machine learning, breaking new ground and solving complex problems once considered beyond reach. Despite its significant achievements across various fields, traditional deep learning models face critical challenges, particularly in quantifying prediction uncertainties. This gap often results in overconfident predictions when encountering novel scenarios, leaving models vulnerable to adversarial attacks and poorly equipped to handle data perturbations or out-of-distribution (OOD) inputs. Bayesian Deep Learning (BDL) emerges as a robust alternative, offering a principled approach to manage uncertainties. Unlike standard models that rely on point estimates, BDL focuses on developing posterior distributions for model parameters, integrating both prior knowledge and observed data likelihood. This shift not only mitigates the limitations associated with traditional deep learning techniques but also enhances model reliability and decision-making under uncertainty.

We aim to create explainable and actionable methods within the Bayesian deep learning framework that excel in not just precisely measuring predictive uncertainty but also in identifying, attributing, and reducing the effects of uncertainty to boost model accuracy. Our focus will be on three key areas of research: uncertainty quantification (UQ), uncertainty attribution (UA), and uncertainty mitigation (UM). The dynamic relationship among UQ, UA, and UM is depicted in the following figure, where UQ facilitates the process of UA in pinpointing uncertainty origins, which in turn supports UM in diminishing the influence of uncertainties. Specifically, UA plays a crucial role in UM by improving model performance via a more nuanced comprehension of uncertainty sources. In scenarios where uncertainty drives decision-making, the synergistic effect of UQ, UA, and UM can significantly enhance model functionality by leveraging insights derived from uncertainty analysis.

uncertainty.jpg

Uncertainty quantification

Uncertainty attribution

Uncertainty based interventions