An Introduction to Stochastic Control

This is a course on the optimal control of random processes. The first part of the course will focus on discrete time and the optimal control of Markov chains (also called Markov Decision Theory in the context of Reinforcement Learning). Various optimality criteria are introduced and questions of existence of solutions to the corresponding Bellman equation and characterization of optimal controls are addressed. Applications from finance, engineering and optimal stopping will be developed, as well as methods for numerical solution of the Bellman equation. We then consider problems in continuous time, and the difficulties that occur when there is no classical sense solution to the corresponding Bellman equation. If time permits, other applications areas and models with partial observations may be considered. Prerequisites: APMA 2630 / 2640 .