Introduction to Machine Learning

Learning problem, data, and leakage

Specify the input, target, prediction time, and decision supported by the model. Supervised learning uses labeled examples; unsupervised learning looks for structure without the same target signal. Split data to reflect deployment: time-based or group-based separation may be necessary when observations are related. Leakage occurs when training gains information unavailable at prediction time. Fit preprocessing on training data and apply the learned transformation to validation and test data, rather than estimating it from the whole dataset.

References: Goodfellow, Bengio, Courville — Deep Learning, free author edition

Regression and classification objectives

Linear regression predicts a weighted sum of features and often minimizes squared error. Logistic regression maps a linear score through a sigmoid to model a binary probability; it is a classification model despite its name. An optimization objective combines a data-fit term with any penalty. The scale of features affects optimization and regularization, so preprocessing is part of the method. A low training loss alone says little about generalization, calibration, or suitability for the downstream decision.

References: Goodfellow, Bengio, Courville — Deep Learning, free author edition

Evaluation, selection, and regularization

A validation set supports model and hyperparameter choices; a test set estimates performance after those choices are fixed. Cross-validation repeats this separation across folds but must preserve temporal or group constraints. Regularization restricts effective complexity, trading some training fit for more stable predictions. Compare against a baseline and choose a metric matching error costs. Accuracy can hide a classifier that misses a rare class, while a ranking metric does not guarantee useful probabilities at a chosen threshold.

References: Goodfellow, Bengio, Courville — Deep Learning, free author edition

Worked confusion-matrix example

Consider 100 cases with 10 positives. A classifier identifies 8 positives correctly, misses 2, and incorrectly flags 12 negatives. It has precision 8/(8+12) = 0.40 and recall 8/(8+2) = 0.80; accuracy is (8+78)/100 = 0.86. Predicting every case negative reaches 0.90 accuracy but zero recall. The example shows why the metric should follow the operational goal. Changing the threshold moves the precision-recall tradeoff, so report the threshold or evaluation protocol.

References: Goodfellow, Bengio, Courville — Deep Learning, free author edition

Different models make different assumptions

A decision tree partitions feature space using rules; an ensemble combines several learners to improve stability or expressive power. A support vector machine seeks a separating margin and can use kernels to represent nonlinear boundaries. Clustering groups observations under a similarity assumption; its output is not automatically a set of true categories. PCA finds orthogonal directions of high variance after centering, and feature scale can dominate its result. Understand the geometry and failure conditions before comparing algorithms by name.

References: Goodfellow, Bengio, Courville — Deep Learning, free author edition

Neural networks, representation, and data ethics

A neural network composes parameterized transformations and nonlinearities; backpropagation computes gradients used by an optimizer. CNNs encode useful spatial structure, while attention-based models combine information according to learned compatibility. Model capacity, optimization, and regularization interact. Document data provenance, consent constraints, missingness, and subgroup performance. An aggregate improvement can conceal worse outcomes for an affected group. Distinguish a measured association from a causal claim, and assess whether the intended deployment resembles the data on which performance was measured.

References: Goodfellow, Bengio, Courville — Deep Learning, free author edition

Practice & worked answers

Original exercises for this guide.

Public notes & references

Course sources establish the scope above. Reference materials support the explanations; linked lecture notes and books remain with their authors.

• Carleton COMP 3105 — Fall 2026 public outline ↗Course scope · Fall 2026
• Goodfellow, Bengio, Courville — Deep Learning, free author edition ↗Open textbook
• Carleton COMP 3105 — Fall 2026 model and evaluation topic map ↗Topic map