Natural Language Processing: Self-Supervised Models

3 Credits.

The rise of massive self-supervised (pre-trained) models have transformed various data-driven fields such as natural language processing (NLP). In this course, students will gain a thorough introduction to self-supervised learning techniques for NLP applications. Through lectures, assignments, and a final project, students will learn the necessary skills to design, implement, and understand their own self-supervised neural network models, using the Pytorch framework. Required course background: data structures, linear algebra, probability, familiarity with Python/PyTorch.