STSCI 5065
Big Data Management and Analysis
Concepts, challenges, and industry trends of big data, with a focus on the Hadoop system. Topics include: basics of the Apache Hadoop platform and Hadoop ecosystem; the Hadoop distributed file system (HDFS); MapReduce or its alternative, a parallel programming model for distributed processing of large data sets; common big data tools, such as Pig (a procedural data processing language for Hadoop parallel computation), Hive (a declarative SQL-like language to handle Hadoop jobs), HBase (the most popular NoSQL database), and YARN; case studies; and integration of Hadoop with statistical software packages, e.g., SAS and R.
Topics
- Comprehend the basic concepts of big data, the Apache Hadoop platform and its ecosystem, and the Hadoop distributed file system (HDFS).
- Be able to use MapReduce, a parallel programming model for distributed processing of large data sets.
- Master Hive, a declarative SQL-like language that presents data in the form of tables to handle Hadoop jobs.
- Master Pig, a procedural data processing language for Hadoop parallel computation.
- Understand components of the Hadoop ecosystem, especially PySpark.
- Know how to integrate the Hadoop system with statistical analysis software/packages, e.g., SAS and/or R.
- Master the basic skills of the CentOS Linux OS.
Official roster learning outcomes
Prerequisites & course information
background equivalent to: STSCI 4060; STSCI 5060; STSCI 5010; STSCI 4520 or STSCI 4030.
Sources
- Course catalogue ↗2026-2027
- Catalogue entry ↗
Catalogue variants
- Big Data Management and Analysis · 3.0 · Graduate