Quantization and Signal Processing for AI

3 credits · Graduate

This course introduces fundamental concepts in signal processing and quantization and explores their role in modern machine learning and AI. Topics include signal representations, Fourier and time-frequency analysis, wavelets, statistical learning, matrix decompositions, quantization, neural networks, and deep learning. Building on classical autoregression and adaptive filtering, the course develops neural sequence models, attention, transformers, and state-space models. Applications include signal classification, prediction, and generation, as well as model compression, mixed-precision computation, and efficient inference for large language models and other generative AI systems.