Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation
University of Amsterdam, National Taiwan University · University of Oxford · Normal Computing · University of Maryland, Baltimore County · University of California, Berkeley · University of Amsterdam & Qualcomm · Academia Sinica Taiwan; University of Amsterdam
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Stochastic-gradient MCMC methods enable scalable Bayesian posterior sampling but often suffer from sensitivity to minibatch size and gradient noise. To address this, we propose Stochastic Gradient Lattice Random Walk (SGLRW), an extension of the Lattice Random Walk discretization. Unlike conventional Stochastic Gradient Langevin Dynamics (SGLD), SGLRW introduces stochastic noise only through the off-diagonal elements of the update covariance; this yields greater robustness to minibatch size while retaining asymptotic correctness. Furthermore, as comparison we analyze a natural analogue of SGLD utilizing gradient clipping. Experimental validation on Bayesian regression and classification demonstrates that SGLRW remains stable in regimes where SGLD fails, including in the presence of heavy-tailed gradient noise, and matches or improves predictive performance.