Bayes Theory, Information Theory, and Neural Networks

Graduate / Professional

This course provides the tools with which to demonstrate statistical accuracy and computational feasibility of the training of artificial neural networks. The construction and properties of Bayes procedures are developed; their frequentist accuracy is revealed by information theory and arbitrary sequence regret analysis; Markov chain Monte Carlo techniques and their analysis are reviewed; and a log-concave coupling is demonstrated for the intrinsically multimodal artificial neural net models which establishes rapid computation of Bayes estimates. The approximation capabilities are presented for flexibly trained neural networks that are not available with classical linear models and not available with more rigidly trained gradient methods or neural tangent kernel methods. The course equips students for research on the statistical and computational foundations of deep learning, to recognize the important open problems in the field, and to appraise neural network training techniques with an appropriate critical perspective. Prerequisite: Prior exposure to mathematical probability and statistics at levels comparable to S&DS 2410 and 2420 . This course is suitable for graduate students in Statistics & Data Science and related programs. Enrollment by advanced undergraduates is possible though subject to instructor evaluation for approval. W 10:30am-11:45am, M 11:35am-12:50pm