Information Theory

1 Course Unit

Deterministic and probabilistic information. The pigeon-hole principle. Entropy, relative entropy, and mutual information. Random processes and entropy rate. The asymptotic equipartition property. Optimal codes and data compression. Channel capacity. Source channel coding. The ubiquitous nature of the theory will be illustrated with a selection of applications drawn from among: universal source coding, vector quantization, network communication, the stock market, hypothesis testing, algorithmic computation and kolmogorov complexity, and thermodynamics. Not Offered Every Year Prerequisites: Students must take ESE 5300 before taking ESE 6740. 1 Course Unit