Special Topics: Introduction to Data Engineering

This course covers fundamental Data Engineering concepts and techniques for building reliable data pipelines. Students will learn to design, implement, and maintain systems that collect, transform, and deliver data. Topics include ETL processes, data modeling, distributed processing frameworks, pipeline orchestration, and incorporating generative AI and other technologies into intelligent data pipelines, We will examine tooling ranging from bash scripts to DuckDB to managed Spark clusters to transform small and large datasets and orchestrate intricate data flows while communicating the usefulness and limitations of data products to stakeholders. Students will acquire practical skills applicable across a variety of industries and technical roles, including analytics engineer, data engineer, and data analyst. Prerequisites : Proficient in Python; familiarity with SQL and basic cloud concepts (Fall)