✨ TL;DR
This paper develops a general mathematical theory explaining how symmetries in target distributions and variational families guarantee that variational inference can accurately recover certain statistics, even when the approximation is imperfect. The framework unifies existing results and enables derivation of new recovery guarantees across diverse settings including directional statistics.
Variational inference approximates intractable probability distributions by optimizing over simpler families, but these families often cannot represent the target exactly. This model misspecification raises critical questions about which properties of the target distribution can be reliably captured. While recent work has shown that symmetries can enable recovery of certain statistics despite misspecification, these results are problem-specific and lack a unified theoretical foundation. There is no general understanding of the fundamental mechanism by which symmetry forces statistic recovery, limiting our ability to predict when VI will succeed or fail at capturing specific properties of interest.
The authors develop a general mathematical framework for understanding symmetry-induced statistic recovery in variational inference. Their approach has three main components: First, they characterize when variational minimizers inherit symmetries from the target distribution and establish conditions under which these inherited symmetries pin down identifiable statistics. Second, they demonstrate that their theory subsumes existing results by showing that known recovery guarantees in location-scale families emerge as special cases of their general framework. Third, they apply their theory to new settings, specifically distributions on the sphere, to derive novel guarantees for recovering directional statistics in von Mises-Fisher families. The framework is designed to be modular, providing a systematic blueprint for deriving recovery guarantees across different symmetry settings.