Large Language Models: From Foundations to Modern Practice

Undergraduate

This course provides a deep dive into modern Large Language Models (LLMs), spanning both foundational concepts and cutting-edge developments and practice. It begins with core neural network concepts for language modeling, covering word embeddings, sequence modeling, and attention mechanisms. Building on these foundations, we examine transformer architectures and their evolution from task-specific models to general language models, including early models such as BERT, GPT, and T5. We explore how these models enable sophisticated language understanding and generation through pre-training and transfer learning. The latter portion focuses on contemporary advances: scaling of language models, reasoning, retrieval augmentation, post-training and reinforcement learning, alignment, parameter-efficient fine-tuning, evaluation, and LLM-based agents. Prerequisites: Familiarity with the basics of machine learning is required, and students should be comfortable with programming. Introduction to Machine Learning ( CPSC 3810 /5810, S&DS 2650 or S&DS 3650 /6650) OR Artificial Intelligence ( CPSC 3700 /5700). If you have passed equivalent courses, contact the instructor for permission before taking the course. QR HTBA