Sequential Decision Making: Theoretical Foundations and Modern Applications

Graduate / Professional

Sequential decision making in the face of uncertainty has garnered growing interest in recent years, with successes in a multitude of applications such as recommendation systems, robotics, and AI systems. This course aims to cover important theoretical and algorithmic foundations under the paradigms of bandits and reinforcement learning (RL) and discuss research issues arising from their modern applications in generative AI. We cover multiple important topics including stochastic bandits, adversarial bandits, planning in Markov decision processes, online and offline RL, policy optimization, and multi-agent RL, gravitating our discussions around issues such as sample complexity, computational efficiency, and function approximation. We also illustrate how RL is used to enable reasoning and alignment of foundation models, along with related (open) research questions. Prerequisite: basic fluency in linear algebra, probability, optimization, machine learning, and programming. T 9:25am-11:20am