Data-Driven Control Methods and Reinforcement Learning

3 units

Modeling and control approaches of the past decades are usually concerned with analytically described control systems with relatively mild complexity, which allows for a highly successful treatment by rigorous systems theoretic methods. Recent years, however, have witnessed a significant shift towards the consideration of far more complicated control systems in which purely analytical approaches are infeasible. This is a research-focused course that will introduce and explore systematic approaches towards augmenting the core foundations of systems and control theoretic frameworks with data-integrating and learning-based capabilities to efficiently harness the vast amounts of valuable operational data and computing resources in order to solve challenging control tasks that escape the traditional setting. The starting point for these new developments are specific macroscopic considerations of dynamical systems associated with transfer operators and Koopman operators. After reviewing these operator-theoretic frameworks, we will explore a family of sample-based approaches that emerge out of the macroscopic viewpoint. These sample-based approaches not only mitigate drawbacks of the original operator-theoretic approaches but also facilitate more direct and efficient data-integrated paths for elucidating important features of dynamical systems with applications to control and estimation. Moreover, connections with established methods from Reinforcement Learning will be integrated into the course material.