everything.study
6.832

Underactuated Robotics

Graduate · Historical course

Spring 2022

Robots today move far too conservatively, using control systems that attempt to maintain full control authority at all times. Humans and animals move much more aggressively by routinely executing motions which involve a loss of instantaneous control authority. Controlling nonlinear systems without complete control authority requires methods that can reason about and exploit the natural dynamics of our machines. This course introduces nonlinear dynamics and control of underactuated mechanical systems, with an emphasis on computational methods. Topics include the nonlinear dynamics of robotic manipulators, applied optimal and robust control and motion planning. Discussions include examples from biology and applications to legged locomotion, compliant manipulation, underwater robots, and flying machines.

Spring 2009

Topics

  • Nonlinear dynamics of the simple pendulum
  • Dynamic programming and value interation: grid world, double integrator, and pendulum examples
  • Acrobot and cart-pole: controllability, partial feedback linearization (PFL), and energy shaping
  • Acrobot and cart-pole (cont.)
  • Policy search: open-loop optimal control, direct methods, and indirect methods
  • Policy search (cont.): trajectory stabilization, iterative linear quadratic regulator (iLQR), differential dynamic programming (DDP)
  • Simple walking models: rimless wheel, compass gait, kneed compass gait
  • Feedback control for simple walking models
  • Simple running models: spring-loaded inverted pendulum (SLIP), Raibert hoppers
  • Midterm
  • Motion planning: Dijkstra’s, A-star
  • Randomized motion planning: rapidly-exploring randomized trees and probabilistic road maps
  • Feedback motion planning: planning with funnels, linear quadratic regulator (LQR) trees
  • Function approximation and system identification
  • Model systems with uncertainty: state distribution dynamics and state estimation
  • Stochastic optimal control
  • Aircraft
  • Swimming and flapping flight
  • Randomized policy gradient
  • Randomized policy gradient (cont.)
  • Model-free value methods: temporal difference learning and Q-learning
  • Final project presentations

Readings

  • Strogatz, Steven H. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering . Boulder, CO: Westview Press, 2001. ISBN: 9780738204536.
    Recommended
  • Slotine, Jean-Jacques E., and Weiping Li. Applied Nonlinear Control . Upper Saddle River, NJ: Prentice Hall, 1991. ISBN: 9780130408907.
    Recommended
  • Fantoni, Isabelle, and Rogelio Lozano. Non-linear Control for Underactuated Mechanical Systems . New York, NY: Springer-Verlag, 2002. ISBN: 9781852334239.
    Recommended
  • Bertsekas, Dimitri P. Dynamic Programming and Optimal Control . 3rd ed. Vols. I and II. Nashua, NH: Athena Scientific, 2007. ISBN: 9781886529083 (set).
    Recommended
  • Recommended
  • Bertsekas, Dimitri P., and John N. Tsitsiklis. Neuro-Dynamic Programming . Nashua, NH: Athena Scientific, 1996. ISBN: 9781886529106.
    Recommended
  • LaValle, Steven M. Planning Algorithms . New York, NY: Cambridge University Press, 2006. ISBN: 9780521862059.
    Recommended
  • Chapter 1 and Appendix A
    Listed reading
  • SES #: 2 · TOPICS: Nonlinear dynamics of the simple pendulum · READINGS: Chapter 2
    Reading assignment
  • Chapter 9 (cont.)
    Listed reading
  • SES #: 6 · TOPICS: Acrobot and cart-pole: controllability, partial feedback linearization (PFL), and energy shaping · READINGS: Chapter 3
    Reading assignment
  • SES #: 7 · TOPICS: Acrobot and cart-pole (cont.) · READINGS: Chapter 3 (cont.)
    Reading assignment
  • SES #: 8 · TOPICS: Policy search: open-loop optimal control, direct methods, and indirect methods · READINGS: Chapter 12
    Reading assignment
  • SES #: 9 · TOPICS: Policy search (cont.): trajectory stabilization, iterative linear quadratic regulator (iLQR), differential dynamic programming (DDP) · READINGS: Chapter 12 (cont.)
    Reading assignment
  • SES #: 10 · TOPICS: Simple walking models: rimless wheel, compass gait, kneed compass gait · READINGS: Chapter 5
    Reading assignment
  • SES #: 11 · TOPICS: Feedback control for simple walking models · READINGS: Chapter 5 (cont.)
    Reading assignment
  • SES #: 12 · TOPICS: Simple running models: spring-loaded inverted pendulum (SLIP), Raibert hoppers · READINGS: Chapter 6
    Reading assignment
  • SES #: 13 · TOPICS: Motion planning: Dijkstra’s, A-star · READINGS: Chapter 13
    Reading assignment
  • SES #: 14 · TOPICS: Randomized motion planning: rapidly-exploring randomized trees and probabilistic road maps · READINGS: Chapter 13 (cont.)
    Reading assignment
  • SES #: 15 · TOPICS: Feedback motion planning: planning with funnels, linear quadratic regulator (LQR) trees · READINGS: Chapter 14
    Reading assignment
  • SES #: 16 · TOPICS: Function approximation and system identification · READINGS: Chapter 8 and Appendix B
    Reading assignment
  • SES #: 17 · TOPICS: Model systems with uncertainty: state distribution dynamics and state estimation · READINGS: Chapter 8 (cont.)
    Reading assignment
  • SES #: 18 · TOPICS: Stochastic optimal control · READINGS: Chapter 15
    Reading assignment
  • SES #: 19 · TOPICS: Aircraft · READINGS: Chapter 7
    Reading assignment
  • SES #: 20 · TOPICS: Swimming and flapping flight · READINGS: Chapter 7 (cont.)
    Reading assignment
  • SES #: 21 · TOPICS: Randomized policy gradient · READINGS: Chapter 17
    Reading assignment
  • SES #: 22 · TOPICS: Randomized policy gradient (cont.) · READINGS: Chapter 17 (cont.)
    Reading assignment
  • SES #: 23 · TOPICS: Model-free value methods: temporal difference learning and Q-learning · READINGS: Chapter 16
    Reading assignment
  • Listed reading
  • Reading assignment
  • Reading assignment
  • CHAPTERS: I. Nonlinear dynamics and control
    Reading assignment
  • Reading assignment
  • Reading assignment
  • CHAPTERS: 4 · TOPICS: Manipulation
    Reading assignment
  • Reading assignment
  • CHAPTERS: 6 · TOPICS: Running
    Reading assignment
  • CHAPTERS: 7 · TOPICS: Flight
    Reading assignment
  • CHAPTERS: 8 · TOPICS: Model systems with stochasticity
    Reading assignment
  • CHAPTERS: II. Optimal control and motion planning
    Reading assignment
  • Reading assignment
  • Reading assignment
  • CHAPTERS: 11 · TOPICS: Analytical optimal control with Pontryagin’s minimum principle
    Reading assignment
  • Reading assignment
  • CHAPTERS: 13 · TOPICS: Feasible motion planning
    Reading assignment
  • CHAPTERS: 14 · TOPICS: Global policies from local policies
    Reading assignment
  • CHAPTERS: 15 · TOPICS: Stochastic optimal control
    Reading assignment
  • CHAPTERS: 16 · TOPICS: Model-free value methods
    Reading assignment
  • Reading assignment
  • CHAPTERS: 18 · TOPICS: Actor-critic methods
    Reading assignment
  • CHAPTERS: IV. Applications and extensions
    Reading assignment
  • CHAPTERS: 19 · TOPICS: Learning case studies and course wrap-up
    Reading assignment
  • CHAPTERS: Appendix
    Reading assignment
  • Reading assignment
Sources