18.0651
Matrix Methods in Data Analysis, Signal Processing, and Machine Learning
Reviews linear algebra with applications to life sciences, finance, engineering, and big data. Covers singular value decomposition, weighted least squares, signal and image processing, principal component analysis, covariance and correlation matrices, directed and undirected graphs, matrix factorizations, neural nets, machine learning, and computations with large matrices. Students in Course 18 must register for the undergraduate version, 18.065 .
Spring 2018
Topics
- The Column Space of \(A\) Contains All Vectors \(A\boldsymbol{x}\)
- Multiplying and Factoring Matrices
- Orthonormal Columns in \(Q\) Give \(Q’Q= I\)
- Eigenvalues and Eigenvectors
- Positive Definite and Semidefinite Matrices
- Singular Value Decomposition (SVD)
- Eckart-Young: The Closest Rank \(k\) Matrix to \(A\)
- Norms of Vectors and Matrices
- Four Ways to Solve Least Squares Problems
- Survey of Difficulties with \(A\boldsymbol{x} = \boldsymbol{b}\)
- Minimizing \(‖\boldsymbol{x}‖\) Subject to \(A\boldsymbol{x} = \boldsymbol{b}\)
- Computing Eigenvalues and Singular Values
- Randomized Matrix Multiplication
- Low Rank Changes in \(A\) and Its Inverse
- Matrices \(A(t)\) Depending on \(t\), Derivative = \(dA/dt\)
- Derivatives of Inverse and Singular Values
- Rapidly Decreasing Singular Values
- Counting Parameters in SVD, LU, QR, Saddle Points
- Saddle Points Continued, Maxmin Principle
- Definitions and Inequalities
- Minimizing a Function Step by Step
- Gradient Descent: Downhill to a Minimum
- Accelerating Gradient Descent (Use Momentum)
- Linear Programming and Two-Person Games
- Stochastic Gradient Descent
- Structure of Neural Nets for Deep Learning
- Backpropagation: Find Partial Derivatives
- Computing in Class [No video available]
- Computing in Class (cont.) [No video available]
- Completing a Rank-One Matrix, Circulants!
- Eigenvectors of Circulant Matrices: Fourier Matrix
- ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule
- Neural Nets and the Learning Function
- Distance Matrices, Procrustes Problem
- Finding Clusters in Graphs
- Alan Edelman and Julia Language
Readings
- LEC #: 1 · TOPICS: The Column Space of \(A\) Contains All Vectors \(A\boldsymbol{x}\) · READINGS: Section I.1: Multiplication \(A\boldsymbol{x}\) Using Columns of \(A\)
- LEC #: 2 · TOPICS: Multiplying and Factoring Matrices · READINGS: Section I.2: Matrix-Matrix Multiplication \(AB\)
- LEC #: 3 · TOPICS: Orthonormal Columns in \(Q\) Give \(Q’Q= I\) · READINGS: Section I.5: Orthogonal Matrices and Subspaces
- LEC #: 4 · TOPICS: Eigenvalues and Eigenvectors · READINGS: Section I.6: Eigenvalues and Eigenvectors
- LEC #: 5 · TOPICS: Positive Definite and Semidefinite Matrices · READINGS: Section I.7: Symmetric Positive Definite Matrices
- LEC #: 6 · TOPICS: Singular Value Decomposition (SVD) · READINGS: Section I.8: Singular Values and Singular Vectors in the SVD
- LEC #: 7 · TOPICS: Eckart-Young: The Closest Rank \(k\) Matrix to \(A\) · READINGS: Section I.9: Principal Components and the Best Low Rank Matrix
- LEC #: 8 · TOPICS: Norms of Vectors and Matrices · READINGS: Section I.11: Norms of Vectors and Functions and Matrices
- LEC #: 9 · TOPICS: Four Ways to Solve Least Squares Problems · READINGS: Section II.2: Least Squares: Four Ways
- LEC #: 10 · TOPICS: Survey of Difficulties with \(A\boldsymbol{x} = \boldsymbol{b}\) · READINGS: Intro Chapter 2: Introduction to Computations with Large Matrices
- LEC #: 11 · TOPICS: Minimizing \(‖\boldsymbol{x}‖\) Subject to \(A\boldsymbol{x} = \boldsymbol{b}\) · READINGS: Section I.11: Norms of Vectors and Functions and Matrices
- LEC #: 12 · TOPICS: Computing Eigenvalues and Singular Values · READINGS: Section II.1: Numerical Linear Algebra
- LEC #: 13 · TOPICS: Randomized Matrix Multiplication · READINGS: Section II.4: Randomized Linear Algebra
- LEC #: 14 · TOPICS: Low Rank Changes in \(A\) and Its Inverse · READINGS: Section III.1: Changes in \(A^{-1}\) from Changes in \(A\)
- LEC #: 15 · TOPICS: Matrices \(A(t)\) Depending on \(t\), Derivative = \(dA/dt\) · READINGS: Section III.1: Changes in \(A^{-1}\) from Changes in \(A\) Section III.2: Interlacing Eigenvalues and Low Rank Signals
- LEC #: 16 · TOPICS: Derivatives of Inverse and Singular Values · READINGS: Section III.1: Changes in \(A^{-1}\) from Changes in \(A\) Section III.2: Interlacing Eigenvalues and Low Rank Signals
- LEC #: 17 · TOPICS: Rapidly Decreasing Singular Values · READINGS: Section III.3: Rapidly Decaying Singular Values
- LEC #: 18 · TOPICS: Counting Parameters in SVD, LU, QR, Saddle Points · READINGS: Section III.2: Interlacing Eigenvalues and Low Rank Signals
- LEC #: 19 · TOPICS: Saddle Points Continued, Maxmin Principle · READINGS: Section III.2: Interlacing Eigenvalues and Low Rank Signals Section V.1: Mean, Variance, and Probability
- LEC #: 21 · TOPICS: Minimizing a Function Step by Step · READINGS: Section VI.1: Minimum Problems: Convexity and Newton’s Method Section VI.4: Gradient Descent Toward the Minimum
- LEC #: 22 · TOPICS: Gradient Descent: Downhill to a Minimum · READINGS: Section VI.4: Gradient Descent Toward the Minimum
- LEC #: 23 · TOPICS: Accelerating Gradient Descent (Use Momentum) · READINGS: Section VI.4: Gradient Descent Toward the Minimum
- LEC #: 24 · TOPICS: Linear Programming and Two-Person Games · READINGS: Section VI.2: Lagrange Multipliers = Derivatives of the Cost Section VI.3: Linear Programming, Game Theory, and Duality
- LEC #: 25 · TOPICS: Stochastic Gradient Descent · READINGS: Section VI.5: Stochastic Gradient Descent and ADAM
- LEC #: 26 · TOPICS: Structure of Neural Nets for Deep Learning · READINGS: Section VII.1: The Construction of Deep Neural Networks
- LEC #: 27 · TOPICS: Backpropagation: Find Partial Derivatives · READINGS: Section VII.3: Backpropagation and the Chain Rule
- LEC #: 28 · TOPICS: Computing in Class [No video available] · READINGS: Section VII.2: Convolutional Neural Nets
- LEC #: 30 · TOPICS: Completing a Rank One Matrix, Circulants! · READINGS: Section IV.8: Completing Rank One Matrices Section IV.2: Shift Matrices and Circulant Matrices
- LEC #: 31 · TOPICS: Eigenvectors of Circulant Matrices: Fourier Matrix · READINGS: Section IV.2: Shift Matrices and Circulant Matrices
- LEC #: 32 · TOPICS: ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule · READINGS: Section IV.2: Shift Matrices and Circulant Matrices
- LEC #: 33 · TOPICS: Neural Nets and the Learning Function · READINGS: Section VII.1: The Construction of Deep Neural Networks Section IV.10: Distance Matrices
- LEC #: 34 · TOPICS: Distance Matrices, Procrustes Problem · READINGS: Section IV.9: The Orthogonal Procrustes Problem Section IV.10: Distance Matrices
- LEC #: 35 · TOPICS: Finding Clusters in Graphs · READINGS: Section IV.6: Graphs and Laplacians and Kirchhoff’s Laws Section IV.7: Clustering by Spectral Methods and \(k\)-means
- LEC #: 36 · TOPICS: Alan Edelman and Julia Language · READINGS: Section III.3: Rapidly Decaying Singular Values Section VII.2: Convolutional Neural Nets
Prerequisites & course information
Subject meets with 18.065 Prereq: 18.06 G (Spring) 3-0-9 units
Sources
- Course catalogue ↗2026–2027
- MIT OpenCourseWare ↗Spring 2018
- MIT OCW readings ↗
- MIT OCW calendar ↗
- MIT OCW syllabus ↗
- Historical MIT OpenCourseWare offering. Named reading entries and schedule topics are automatically extracted from the linked source pages.