Special Topics: Statistical Evaluation for AI

9 units · Undergraduate

Intermittent: 9 units Modern AI systems arrive with benchmark scores, probability claims, and confident recommendations. This course asks what those numbers actually mean. Students will study AI systems through a statistician's perspective. They will define populations and estimands, investigate how data and labels were produced, assess the quality and assumptions of benchmarks, quantify uncertainty, compare systems fairly, test consequential assumptions, and interpret results in context. This class is not a survey of machine-learning algorithms but rather a study of how a statistician or data scientist should use these systems. In this course, students will both use statistical thinking to evaluate AI systems and use AI to enhance their statistical output, thus creating a feedback loop where one strategy can propel the other forward. Prerequisite: 36-401 Min. grade C