INFORMATION THEORY

Prerequisites & course information

Requisites: Graduate/professional standing Course Designation: Grad 50% - Counts toward 50% graduate coursework requirement Repeatable for Credit: No Last Taught: Spring 2022 Learning Outcomes: 1. Calculate the entropy of a random variable from its distribution, the mutual information between two random variables from their joint distribution, the Kullback-Leibler divergence between two probability distributions, and the entropy rate of a Markov process Audience: Graduate 2. Construct source codes such as the Huffman code, the Shannon code, and the Elias-Fano code for a given probability distribution of the source Audience: Graduate 3. Define channel capacity and apply Shannon’s channel coding theorem to Calculate the channel capacities of discrete channels such as the binary symmetric channel and the binary erasure channel, and continuous channels such as the additive Gaussian noise channel Audience: Graduate Learning Outcomes: 1. Calculate the entropy of a random variable from its distribution, the mutual information between two random variables from their joint distribution, the Kullback-Leibler divergence between two probability distributions, and the entropy rate of a Markov process Audience: Graduate 2. Construct source codes such as the Huffman code, the Shannon code, and the Elias-Fano code for a given probability distribution of the source Audience: Graduate 3. Define channel capacity and apply Shannon’s channel coding theorem to Calculate the channel capacities of discrete channels such as the binary symmetric channel and the binary erasure channel, and continuous channels such as the additive Gaussian noise channel Audience: Graduate