AI Training and Inference Systems: From First Principles to Self-Design

Harvard FAS

The systems that train, fine-tune, and serve large models are the engines of the AI revolution, and their performance comes down to two goals: minimizing data movement across the memory hierarchy and maximizing GPU and CPU utilization. This course teaches the first principles through which system designs and implementations achieve these goals: modularity and abstractions, breaking problems into smaller pieces, parallelism, overlapping computation with data movement, and navigating vast…