LLMs for Comp Linguistics

3 Credits · Level: Graduate, Undergraduate

Large language models (LLMs) are the foundational technology behind today’s most advanced artificial intelligence systems. They have revolutionized the field of natural language processing and challenged traditional ideas about how language is represented in the human mind. This course offers an introduction to LLMs from the perspective of computational linguistics. Through lectures and hands-on demonstrations, we will explore three interrelated questions: How do LLMs work at a technical level? How can they be used to process natural language data? And how can they be used to model human linguistic cognition? The first half of the course will cover technical foundations of LLMs, including the transformer architecture, tokenization, interpretability techniques, and scaling. The second half of the course will focus on LLMs’ applications in natural language processing, linguistics, and cognitive science. Topics will include security and privacy, ethical issues, multilingualism, and LMs as models of human language processing and acquisition. This course is appropriate for advanced undergraduates and graduate students. Students will gain experience training (small) language models, implementing basic interpretability techniques, and reading recent research papers in the area. Knowledge of programming in Python, and basic math and statistics, are required as a prerequisite.