Data Science and Statistics Seminar by Dr. Nicholas Marco (Duke)

About

Title: Adaptive Generalized Elliptical Slice Sampling

Abstract:

A central goal in Bayesian computation is designing MCMC algorithms that are broadly applicable, bypass the need for manual tuning, and scale efficiently with dimension. Elliptical slice sampling meets many of these desiderata: it is tuning-free, does not require differentiability of the target distribution, automatically adapts its step size to local geometry, and, through generalized elliptical slice sampling, can be applied to a broad collection of target distributions. Under favorable conditions, elliptical slice sampling can achieve mixing times that scale logarithmically with dimension, making it more scalable than HMC and an optimally tuned adaptive random walk. However, this advantage is delicate: its efficiency rapidly degrades if there is a mismatch between the target distribution and the distribution used to generate the ellipse-defining auxiliary variables, explaining why we often observe poor efficiency when using elliptical slice sampling in high-dimensional scenarios.

To overcome this substantial limitation, we introduce adaptive generalized elliptical slice sampling (AGESS), which gradually adapts the distribution of the ellipse-defining auxiliary variables, allowing the sampler to self-correct from a slow-mixing to a fast-mixing regime through online learning. Under mild regularity conditions, we prove that AGESS is ergodic, without requiring differentiability of the target and allowing for heavy-tailed targets. The resulting algorithm is robust and efficient, particularly in cases where the target distribution is multimodal, exhibits strong inter-parameter dependencies, has regions of high curvature, or is high-dimensional. We demonstrate this across a range of challenging applications, including deep Gaussian process surrogate modeling and high-dimensional sparse regression.