everything.study
ECE 686

Filtering and Control of Stochastic Linear Systems

Graduate

Winter 2025 · Filtering and Control of Stochastic Linear Systems Winter 2025

Readings

  • Stochastic Systems: Estimation, Identification and Adaptive Control
    P. R. Kumar and P. Varaiya · unspecified
    1986; source for portions of the course; no required textbook.
  • Stochastic Processes, Estimation and Control
    J. L. Speyer and W. H. Chung · recommended
    2008; supplementary reference.
  • Dynamic Programming and Optimal Control
    D. P. Bertsekas · recommended
    2005; supplementary reference.
  • /Materials /T_here is no required textbook, notes will be provided each week with blanks that should be /f_illed in during lecture. Parts of the course are based on notes by Prof. Andrew Heunis, and the following book:
    unspecified
Winter 2024 · Filtering and Control of Stochastic Linear Systems Winter 2024

Readings

  • Stochastic Systems: Estimation, Identification and Adaptive Control
    P. R. Kumar and P. Varaiya · unspecified
    1986; source for portions of the course; no required textbook.
  • Stochastic Processes, Estimation and Control
    J. L. Speyer and W. H. Chung · recommended
    2008; supplementary reference.
  • Dynamic Programming and Optimal Control
    D. P. Bertsekas · recommended
    2005; supplementary reference.
  • /Materials /T_here is no required textbook, notes will be provided each week with blanks that should be /f_illed in during lecture. Parts of the course are based on notes by Prof. Andrew Heunis, and the following book:
    unspecified
Winter 2023 · Filtering and Stochastic Control

Topics

  • Elements of metric space theory and Hilbert spaces.
  • The geometric theory of linear filtering.
  • Discrete-time linear filtering and the discrete-time Kalman filter.
  • Discrete-time stochastic optimal control with complete and partial observations. Part II
  • Markov Decision Processes; Optimality of Markov controls. Dynamic programming and the markov operator. 6 Discounted costs and infinite horizon dynamic programming
  • Average-cost dynamic programming
  • Multi-armed Bandit problems: Gittins Index, Indexable systems. Regret.

Readings

  • M.H. A. Davis and R.B. Vinter: Stochastic Modelling and Control, Chapman and Hall, London, 1984 For Part II
    recommended
  • S. M. Ross : Introduction to Stochastic Dynamic Programming, Academic Press, N.Y., 1982 2
    recommended
  • • Problem sets will be handed out. The onus is on you all to attempt them. Solutions will be posted.
    unspecified
Winter 2021

Topics

  • Introduction and Mathematical Background
  • Motivation and overview
  • Review of probability theory and stochastic processes
  • Derivation of the Projection Theorem
  • Metric and linear spaces
  • Hilbert spaces and the projection theorem for linear estimation
  • Estimation of Discrete-Time Linear Systems
  • Least-squares and recursive estimation
  • The discrete-time Kalman filter
  • Stochastic Optimal Control
  • Dynamic programming and the principle of optimality
  • Optimal control with complete and partial observations
  • Linear quadratic regulators
  • Control of Markov Chains
  • Overview of Markov chains
  • Markov policies and the cost of optimal policies
  • Optimal Stopping Time Problems

Readings

  • Stochastic Systems: Estimation, Identification and Adaptive Control
    P. R. Kumar and P. Varaiya · unspecified
    1986; source for portions of the course; no required textbook.
  • Stochastic Processes, Estimation and Control
    J. L. Speyer and W. H. Chung · recommended
    2008; supplementary reference.
  • Dynamic Programming and Optimal Control
    D. P. Bertsekas · recommended
    2005; supplementary reference.
  • Parts of the course are based on notes by Professor Andrew Heunis.
    unspecified
Winter 2020

Topics

  • Introduction and Mathematical Background
  • Motivation and overview
  • Review of probability theory and stochastic processes
  • Derivation of the Projection Theorem
  • Metric and linear spaces
  • Hilbert spaces and the projection theorem for linear estimation
  • Estimation of Discrete-Time Linear Systems
  • Least-squares and recursive estimation
  • The discrete-time Kalman filter
  • Stochastic Optimal Control
  • Dynamic programming and the principle of optimality
  • Optimal control with complete and partial observations
  • Linear quadratic regulators
  • Control of Markov Chains
  • Overview of Markov chains
  • Markov policies and the cost of optimal policies
  • Optimal Stopping Time Problems

Readings

  • Stochastic Systems: Estimation, Identification and Adaptive Control
    P. R. Kumar and P. Varaiya · unspecified
    1986; source for portions of the course; no required textbook.
  • Stochastic Processes, Estimation and Control
    J. L. Speyer and W. H. Chung · recommended
    2008; supplementary reference.
  • Dynamic Programming and Optimal Control
    D. P. Bertsekas · recommended
    2005; supplementary reference.
  • Parts of the course are based on notes by Professor Andrew Heunis.
    unspecified
Sources