Applied High-Performance and Parallel Computing
This course introduces performance tuning and parallelization, with an emphasis on scientific and data-intensive codes. Topics include single-processor architecture, caches, and serial performance tuning; performance modeling and analysis, including arithmetic intensity, the Roofline model, and strong and weak scaling; the organization of modern parallel machines; shared-memory programming with OpenMP; distributed-memory programming with MPI; single-GPU programming with CUDA; multi-GPU programming with NVSHMEM; and programming emerging accelerators such as the Cerebras CS-3. The course also covers recurring parallel patterns such as data partitioning, synchronization, and load balancing, and applies them to parallel numerical algorithms, parallel graph algorithms, and parallel algorithms arising in science and engineering. Students complete a sequence of programming assignments in which they parallelize and tune real codes on a leadership-class supercomputer, measure and analyze performance, take in-class quizzes, and carry out a substantial final project of their own design.
Topics
- Analyze the performance of serial and parallel programs by measuring runtime, identifying bottlenecks, and applying performance models such as the Roofline model and strong and weak scaling analysis to relate observed behavior to the underlying architecture and memory hierarchy.
- Implement parallel solutions to scientific and data-intensive problems using distributed-memory, single-GPU, and multi-GPU programming models.
- Evaluate design trade-offs in data partitioning, synchronization, and load balancing, and justify implementation choices using empirical scaling results.
Official roster learning outcomes
Prerequisites & course information
CS 3410
Sources
- Course catalogue ↗2026-2027