Decision and Estimation Theory for Discrete Stochastic Processes

3 units

This course is designed for senior students and first-year graduate students. The primary goal is to teach early-stage decision theory and estimation theory, two pivotal topics in system theory. The curriculum covers the fundamentals of discrete-time finite-state Markov processes (including asymptotic behavior, convergence analysis, and ergodicity), the Hidden Markov model along with associated filtering and smoothing problems, and the Markov decision process (including the principle of optimality and dynamic programming). Additionally, a few advanced topics, such as partially observable Markov decision processes (MDPs), Markov games, andreinforcement learning, will be discussed.