The Language Code

4 credits

Natural languages are codes used for communication. This course examines the design features of language that allow them to satisfy two desiderata: communicative effectiveness and efficiency. Topics covered include elementary probability theory (conditional probability; Bayes’ Rule), binary representations, elementary information theory (entropy), data compression, error correction, n-gram language models, noisy channel models, and large language models (e.g., ChatGPT).