Date: 2022-04-01

Time: 15:35-16:35 (Montreal time)

https://mcgill.zoom.us/j/83436686293?pwd=b0RmWmlXRXE3OWR6NlNIcWF5d0dJQT09

Meeting ID: 834 3668 6293

Passcode: 12345

Abstract:

We introduce Learn then Test, a framework for calibrating machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees regardless of the underlying model and (unknown) data-generating distribution. The framework addresses, among other examples, false discovery rate control in multi-label classification, intersection-over-union control in instance segmentation, and the simultaneous control of the type-1 error of outlier detection and confidence set coverage in classification or regression. To accomplish this, we solve a key technical challenge: the control of arbitrary risks that are not necessarily monotonic. Our main insight is to reframe the risk-control problem as multiple hypothesis testing, enabling techniques and mathematical arguments different from those in the previous literature. We use our framework to provide new calibration methods for several core machine learning tasks with detailed worked examples in computer vision.

This is joint work with Anastasios Angelopoulos, Emmanuel Candès, Michael I. Jordan, and Lihua Lei.

Speaker

Dr. Bates is a postdoctoral researcher with Michael I. Jordan in the Statistics and EECS departments at UC Berkeley. He works on developing methods to analyze modern scientific data sets, leveraging sophisticated black box models while providing rigorous statistical guarantees. Specifically, he works on problems in high-dimensional statistics (especially false discovery rate control), statistical machine learning, conformal prediction and causal inference.

Previously, he completed his Ph.D. in the Stanford Department of Statistics advised by Emmanuel Candes. His thesis introduced methods for conditional independence testing and false discovery rate control in genomics, and he was honored to receive the Ric Weiland Graduate Fellowship and the Theodore W. Anderson Theory of Statistics Dissertation Award for this work. Before his Ph.D., he studied statistics and mathematics at Harvard University, and spent a year teaching mathematics at NYU Shanghai. Outside research, I enjoy triathlons, sailing, hiking, and reading speculative fiction novels.

McGill Statistics Seminar schedule: https://mcgillstat.github.io/