CS Talk
Alex Lew, MIT
Host: Zhong Shao
Title: Scaling reliable probabilistic reasoning by generalizing automatic differentiation
Abstract:
Probability plays a central role in AI theory, as the mathematical framework that enables reliable reasoning with uncertain knowledge and noisy data. But in practice, probabilistic reasoning has not yet been scaled or automated as effectively as black-box deep learning. In this talk, I’ll discuss two new generalizations of a PL technique that has supercharged deep learning in the past decade — automatic differentiation — and show how they help to automate and scale trustworthy probabilistic inference.
In particular, I will present techniques for automating two types of “derivative” of a probabilistic model, when that model is represented as the source code of a simulator. The first measures how the input parameters of the model control its expected behavior (AD of expected values), enabling novel unbiased and low-variance gradient estimators. The second quantifies how much more likely an outcome is under one model than another (automated Radon-Nikodym derivatives), leading to a unifying perspective on automated algorithms for Monte Carlo and variational inference. I’ll showcase the application of these techniques in scaling common-sense probabilistic data cleaning, delivering superior performance and accuracy to machine learning approaches on datasets with millions of records. I will also discuss new directions enabled by this research for engineering reliably with large language models, and integrating automated probabilistic reasoning with conversational AI.
Bio:
Alex Lew is a PhD candidate at MIT, co-advised by Vikash Mansinghka and Joshua Tenenbaum. His research focuses on the intersection of programming languages and probabilistic machine learning, aiming to develop systems and theory that facilitate the invention, application, and understanding of scalable algorithms for probabilistic modeling and inference. Prior to his PhD, Alex taught computer science to high school students at the Commonwealth School in Boston, MA (and before that, studied computer science here at Yale). Alex’s work has been recognized with a 2019 Facebook Probability and Programming Award, a 2020 NSF Graduate Research Fellowship, and 2023 ACM SIGPLAN and ACM SIGLOG Distinguished Paper awards.