everything.study
CS 886

Advanced Topics in Artificial Intelligence

Graduate

Fall 2026 · Learning Theory for Modern AI: Transformers and Large Language Models

Topics

  • Exact Algorithmic Learning, Certification, and Hardness
  • Computational Expressivity, Formal Languages, and Circuit Classes
  • Parallel and Fine-Grained Complexity of Transformers
  • Transformer Trainability: Normalization, Depth, and Rank Collapse
  • Sparse Structure and Token Selection in Self-Attention
  • Bayesian and Frequentist Foundations of In-Context Prediction
  • Optimization and Training Dynamics of In-Context Learning
  • Finite-Sample Generalization and Minimax Optimality of In-Context Learning
  • Sample Complexity of Autoregressive Chain-of-Thought
  • Curricula, Scratchpads, and Length Generalization for Reasoning

Readings

Sources