everything.study
6.432

Stochastic Processes, Detection, and Estimation

Graduate · Historical course

Spring 2004

Topics

  • Overview; Problem Review; Random Vectors
  • Course Information; Review of Linear Algebra
  • Covariance Matrices; Gaussian Variables
  • Gaussian Vectors; Bayesian Hypothesis Testing
  • Diagonalization of Symmetric Matrices; Symmetric Positive Definite and Semidefinite Matrices
  • Binary Hypothesis Testing; ROCs
  • More on Symmetric Positive Definite Matrices; Hypothesis Testing for Gaussian Random Vectors
  • ROCs; M-ary Hypothesis Testing
  • Bayesian Estimation; LS; MAP
  • Binary Hypothesis Tests: Receiver Operating Characteristic (ROC); Geometry of M-ary Hypothesis Tests
  • Bayes and Linear LS
  • Vector Spaces
  • Bayes’ Least Squares Estimation; Vector Spaces and Linear Least Squares
  • Nonrandom Parameter Estimation CRB
  • ML Estimation
  • Nonrandom Parameter Estimation
  • Stochastic Processes
  • Linear Systems Review
  • Second-Order Descriptions
  • PSD’s
  • Examples of Stochastic Processes; Second Order Statistics and Stochastic Processes
  • Whitening, Shaping; K-L
  • K-L; Freq, Domain Representation
  • Discrete Time Processes and Linear Systems; Discrete Time Karhunen–Loeve Expansion
  • Detection and Estimation in White Noise
  • Nonlinear Estimation
  • Binary Detection in White Gaussian Noise; Detection and Estimation in Colored Gaussian Noise
  • Det/estimation in Colored Noise; LLSE of Processes
  • Linear Detection from Continuous Time Processes; Karhunen–Loeve Expansions and Whitening Filters
  • Wiener Filtering
  • Discrete–Time Wiener Filtering; Prediction and Smoothing
  • Innovations, State Models
  • Kalman Filtering
  • State Space Models and Kalman Filtering
  • KF; Estimation of Statistics
  • Estimation of Statistics; Modeling
  • Estimation and Detection Using Periodograms
  • Modeling

Readings

  • This course has no required or recommended text. However, the optional reference text (A. Papoulis, Probability, Random Variables, and Stochastic Processes, McGraw-Hill, third ed., 1991, ISBN: 0070484686) can be used as a supplement to the course notes. This book covers many of the same topics we will, but overwhelming student opinion in the past has been that the book is one they tend to refer to much more after the course than during it for a variety of reasons. While you may well find this book a useful addition to your personal library, we will not assume you have access to the book during the term. Several other texts which may be a useful resource to you are listed below.
    Listed reading
  • Anderson, B. D. O., and J. B. Moore. Optimal Filtering. Prentice-Hall, 1979. ISBN: 0-13-638122-7 (Out of Print). Very readable treatment of Kalman and Wiener filtering.
    Listed reading
  • Drake, A. Fundamentals of Applied Probability Theory. McGraw-Hill, 1967. ISBN: 0-07-017815-1 (Out of Print). Basic engineering text on probability theory.
    Listed reading
  • Feller, W. An Introduction to Probability Theory and Its Applications. Vols. 1 and 2. Wiley, 1968. ISBN: 0-471-25708-7. Valuable formal reference set on probability theory.
    Listed reading
  • Gardner, W. A. Introduction to Random Processes: with Applications to Signals and Systems. 2nd ed. McGraw-Hill, 1990. ISBN: 0-07-022855-8. Useful auxilliary reference for a number of topics covered in the course, particularly stochastic processes.
    Listed reading
  • Gray, R. M., and L. D. Davisson. Random Processes: A Mathematical Approach for Engineers. Prentice-Hall, 1986. ISBN: 0-13-752882-5 (Out of Print). Bridges the gap between formal mathematical texts and engineering texts on probability theory.
    Listed reading
  • Helstrom, C. W. Probability and Stochastic Processes for Engineers. Macmillan, 1991. ISBN: 0-02-353571-7. Nice undergraduate level introduction to several themes in the course.
    Listed reading
  • Kay, S. M. Fundamentals of Statistical Signal Processing and Estimation Theory. Prentice-Hall, 1993. ISBN: 0-13-345711-7. Accessible and thorough treatment of estimation theory.
    Listed reading
  • Lee, E., and D. G. Messerschmitt. Digital Communication. 2nd ed. Kluwer Academic, 1994. ISBN: 0-89838-274-2. Advanced reading on applications in communication theory.
    Listed reading
  • Loève, M. Probability Theory. Vol. 1. 4th ed. Springer-Verlag, 1977. ISBN: 3-540-90210. Formal but reasonably readable treatment of probability theory. A classic.
    Listed reading
  • Naylor, A. W., and G. R. Sell. Linear Operator Theory in Engineering and Science. Springer-Verlag, 1982. ISBN: 0-387-90748-3 (Out of Print). Accessible treatment of the mathematical foundations for vector space concepts.
    Listed reading
  • Oppenheim, A. V., and R. W. Schafer. Discrete-Time Signal Processing. Prentice Hall, 1989. ISBN: 0-13-216292-X (Out of Print). Standard text on discrete-time linear systems and signals.
    Listed reading
  • Oppenheim, A. V., and A. S. Willsky. Signals and Systems. Prentice-Hall, 1982. ISBN: 0-13-814757-4. Standard undergraduate text on signals and systems.
    Listed reading
  • Parzen, E. Stochastic Processes. Holden-Day, 1962. ISBN: 0-8162-6664-6. Classic, formal text on stochastic processes.
    Listed reading
  • Scharf, L. L. Statistical Signal Processing: Detection, Estimation, and Time Series Analysis . Addison-Wesley, 1991. ISBN: 0-201-19038-9. Slightly more advanced treatment of several detection and estimation topics.
    Listed reading
  • Stark, H., and J. W. Woods. Probability, Random Processes, and Estimation Theory for Engineers . 2nd ed. Prentice-Hall, 1994. ISBN: 0-13-728791-7 (Out of Print). A useful reference for many topics in the course, as well as the background material.
    Listed reading
  • Strang, G. Linear Algebra and its Applications . 3rd ed. Harcourt Brace Jovanovich, 1988. ISBN: 0-15-551006-1. Standard reference text on linear algebra.
    Listed reading
  • Therrien, C. W. Discrete Random Signals and Statistical Signal Processing. Prentice-Hall, 1992. ISBN: 0-13-852112-3 (Out of Print). Very accessible reference for many topics in the course.
    Listed reading
  • Van, H. L. Trees, Detection, Estimation and Modulation Theory, Part I. Wiley, 1968. ISBN: 0-471-89955-0 (Out of Print). Classic and valuable reference text on detection and estimation theory.
    Listed reading
  • Papoulis, A. Probability, Random Variables, and Stochastic Processes. 3rd ed. McGraw-Hill, 1991. ISBN: 0070484686.
    Recommended
Sources