Vector Search+LLM Engineering

3 Credits · Level: Graduate

This course focuses on how modern language model-based applications combine retrieval systems and large language models (LLMs) to build groundbreaking technologies. As LLMs become increasingly central to decision-making pipelines, their effectiveness depends critically on the retrieval and integration of external knowledge. This course teaches the tools and techniques behind these systems, from inverted indices and vector databases to end-to-end retrieval-augmented generation (RAG) pipelines. Students will build scalable retrieval systems using industry-standard tools including Elasticsearch and Qdrant, while learning how dense and sparse neural retrievers, late-interaction architectures, and approximate nearest neighbor search enable fast and effective search over large datasets. Classical information retrieval concepts are introduced first, with the main focus on modern neural and vector-based approaches. The course then turns to RAG and LLM system design. Students will construct end-to-end pipelines that are evaluated and monitored for observability and failure analysis. Students will serve models and explore efficient inference techniques, while considering tradeoffs between model quality, latency, and cost. Applications will involve speech, text, image, and video modalities. By the end of the course, students will be able to design and deploy production-grade RAG systems that can be used over complex, real-world data. The course culminates in a final project in which students build a complete RAG system tailored to their chosen applications. Students are expected to be proficient in Python and have prior exposure to machine learning or deep learning.