Scientific Computing II (Honours)

Conditioning, stability, and residuals

For Ax = b, the residual r = b - A x_hat measures how closely the computed vector satisfies the supplied equations. Forward error measures distance from the true solution. These can differ greatly when A is ill-conditioned. For A = diag(1,10^-8), changing b from (1,10^-8) to (1,2 times 10^-8) changes the solution from (1,1) to (1,2), although the absolute data change is tiny. Analyze input uncertainty and scale before demanding more output digits. Backward stability describes a solution to a nearby problem, not immunity to sensitivity.

References: Carleton Undergraduate Calendar — COMP 4806 · Driscoll and Braun — Fundamentals of Numerical Computation

Factorizations and structured linear systems

Direct methods factor a matrix and solve simpler triangular systems. LU with suitable pivoting handles many general systems; Cholesky exploits a symmetric positive-definite matrix; QR supports stable least-squares formulations. Avoid explicit inversion when a solve is sufficient. Reuse a factorization when several right-hand sides share the same matrix. Sparse matrices require storage and ordering choices that limit fill-in, because factorization can create nonzeros where the original matrix had zeros. A theoretically smaller operation count can still perform poorly if memory movement dominates the computation.

References: Carleton Undergraduate Calendar — COMP 4806 · Driscoll and Braun — Fundamentals of Numerical Computation · Netlib — LAPACK Users’ Guide, Third Edition

Iterative solvers, Krylov spaces, and preconditioning

Large sparse systems may be addressed through matrix-vector products without storing a dense factorization. Krylov methods build approximations from spaces generated by r, Ar, A squared r, and further products. Conjugate gradients has specific symmetry and positive-definiteness requirements; GMRES addresses a broader nonsymmetric setting with different storage costs. Preconditioning transforms the problem to improve convergence while remaining cheaper to apply than solving the original problem. Judge an iterative method by residual history, cost per iteration, memory, and failure modes rather than comparing iteration counts alone.

References: Carleton Undergraduate Calendar — COMP 4806 · Driscoll and Braun — Fundamentals of Numerical Computation

Eigenvalue problems and the power method

An eigenpair satisfies Av = lambda v with v nonzero. The power method repeatedly multiplies by A and normalizes, tending toward a dominant-magnitude eigendirection under appropriate assumptions. If two eigenvalues have similar magnitudes, convergence can be slow; a missing initial component in the desired direction can also matter. Estimate an eigenvalue with the Rayleigh quotient and inspect the eigen-residual Av - lambda v. Distinguish eigenvalues from singular values: the latter describe stretching and remain useful for rectangular matrices and conditioning analysis.

References: Carleton Undergraduate Calendar — COMP 4806 · Driscoll and Braun — Fundamentals of Numerical Computation

Numerical optimization and curvature

For a smooth objective, the gradient identifies first-order local change and the Hessian describes curvature. Gradient descent steps opposite the gradient with a chosen step size; Newton-type methods use curvature information and require safeguards when assumptions are weak. A small gradient is necessary for an unconstrained differentiable local optimum but does not distinguish a minimum from a saddle by itself. For f(x) = one-half x transpose A x - b transpose x with symmetric positive-definite A, the gradient is Ax - b and the unique minimizer solves Ax = b.

References: Carleton Undergraduate Calendar — COMP 4806 · Driscoll and Braun — Fundamentals of Numerical Computation

Compiled implementations, libraries, and parallel cost

Efficient scientific code combines sound algorithms with appropriate data layout, precision, and numerical libraries. Dense kernels benefit from contiguous access and optimized BLAS/LAPACK implementations; sparse kernels often depend strongly on memory bandwidth and sparsity structure. Parallel speedup is limited by serial work, communication, and synchronization. Floating-point reductions can change rounding order across thread counts, so validate using appropriate tolerances rather than assuming identical output bits. Record compiler options, library versions, matrix properties, and stopping criteria. A performance report should include an accuracy diagnostic so that speed is compared at a meaningful numerical quality.

References: Carleton Undergraduate Calendar — COMP 4806 · Driscoll and Braun — Fundamentals of Numerical Computation · Netlib — LAPACK Users’ Guide, Third Edition

Practice & worked answers

Original exercises for this guide.

Public notes & references

Course sources establish the scope above. Reference materials support the explanations; linked lecture notes and books remain with their authors.

• Carleton Undergraduate Calendar — COMP 4806 ↗Calendar course scope · 2026–27
• Driscoll and Braun — Fundamentals of Numerical Computation ↗Open author textbook · Public reference; checked 2026-11-05
• Netlib — LAPACK Users’ Guide, Third Edition ↗Primary numerical-library guide · Third edition, 1999