Information Theory

3 credits

Entropy, relative entropy, mutual information; entropy rates of a stochastic process; optimal codes and Huffman codes; channel capacity; channel coding theorem; differential entropy, Gaussian channel; rate distortion theory.