MGU Mathematics Series — Linear Algebra
Part V: Applications, Computation, and Calculus Readiness
Modeling · Transformations · Least Squares · Markov · PCA · Calculus Bridge
Chapter 14 of 14
MGU Mathematics Series  ·  Linear Algebra  ·  Library Textbook Edition

Chapter 14:
Linear Algebra Applications
and Capstone Modeling

Bringing together systems, transformations, projections, eigenvalues, least squares, Markov chains, and principal components into a unified framework for real-world modeling and calculus readiness

■ Math Level: Linear Algebra ■

■ 16 Sections  •  Part V of V  •  Capstone Chapter ■

Follows Chapters 1–13 of Linear Algebra  ·  Bridges to Calculus, Multivariable Calculus, Differential Equations, Data Science, Machine Learning

Chapter 14 — Purpose and Overview

Capstone: The Linear Algebra Toolkit in Action

This chapter is the capstone of the entire book. In the thirteen chapters that came before, you built a complete toolkit: vectors, systems of equations, matrices, inverses, determinants, linear transformations, vector spaces, span, basis, dimension, rank, null space, orthogonality, projections, eigenvalues, eigenvectors, and diagonalization. You also encountered dozens of applied settings along the way.

Chapter 14 brings all of that together. Rather than introducing fundamentally new machinery, this chapter shows you how to choose the right tool, combine ideas across chapters, and build complete linear models for real problems. It also shows you explicitly how the ideas you have learned in this book connect to topics you will encounter in Calculus, Multivariable Calculus, Differential Equations, Data Science, Machine Learning, Computer Graphics, Robotics, Economics, and beyond.

By the time you finish this chapter, linear algebra should not feel like a collection of separate procedures. It should feel like a coherent language for describing structure, change, data, geometry, and computation.

Opening Question

A company tracks three products sold across four regional markets. Each week, sales data from every region arrive as a vector. Management wants to know: Which combinations of products drive the most variation across markets? Can the data be compressed into fewer dimensions without losing essential information? Can next week's sales be estimated from this week's trends?

Each of these questions involves linear algebra. Organizing the data involves matrices. Understanding variation involves projections and orthogonality. Compressing information involves eigenvalues, eigenvectors, and the idea of principal components. Estimating future values involves linear models and least squares. And all of it rests on the foundations of vectors, systems, and matrix operations you have already mastered.

16Sections
10Learning Objectives
14Key Terms
5+Worked Examples
13Chapters Unified

Learning Objectives

  1. 1Describe the five-step linear algebra modeling process from problem setup to interpretation.
  2. 2Identify which linear algebra structure fits a given applied problem.
  3. 3Build and solve linear models involving systems, transformations, projections, and eigenvalues.
  4. 4Construct and interpret least squares fits for data.
  5. 5Apply matrix transformations in computer graphics contexts using homogeneous coordinates.
  6. 6Analyze networks and flow models using systems of equations and incidence matrices.
  7. 7Set up and analyze Markov chain models for discrete processes.
  8. 8Describe what principal components are and why they matter in data analysis.
  9. 9Explain how linear algebra ideas connect to calculus, multivariable calculus, and differential equations.
  10. 10Design a capstone modeling project using linear algebra tools.
Chapter 14 — Contents

Sections at a Glance

Sections 14.1 – 14.2
The Modeling Process and Choosing the Right Tool
Five-Step Process · What Makes a Problem Linear · Tool Selection Guide
Section 14.1

The Linear Algebra Modeling Process

Every application of linear algebra follows a common arc: you observe something in the real world, you describe it mathematically, you analyze it using linear algebra tools, and then you interpret what the mathematics tells you. This process is sometimes called mathematical modeling, but for linear algebra it has a specific character because linear algebra tools are particularly powerful when relationships are additive and proportional.

What Makes a Problem Linear?

A relationship is linear when it satisfies two conditions. First, it is additive: combining two inputs gives a combined output. Second, it is homogeneous: scaling an input scales the output proportionally. You have seen this definition applied to linear transformations in Chapter 7 and to vector spaces in Chapter 8. It applies equally to modeling — a linear model is one where doubling an input doubles the output, and where combined inputs give combined outputs.

Many real-world relationships are approximately linear, especially over limited ranges. A budget that allocates fixed costs per unit, a traffic network where flow through intersections obeys conservation, a data set where variation is driven by a small number of underlying factors — all of these situations can be captured, at least approximately, by linear algebra.

The Five-Step Modeling Process

Note on Iteration

This five-step process is not a rigid algorithm. Real modeling often cycles back through earlier steps as new information emerges. But keeping these five questions in mind — describe, represent, analyze, check, interpret — will serve you throughout your study of mathematics, science, engineering, and data analysis.

Section 14.2

Choosing a Linear Structure

One of the most valuable skills you can develop is recognizing which linear algebra structure fits a given situation. The tools you have learned are not interchangeable: a system of equations is the right tool when you have multiple conditions that must all be satisfied simultaneously, while a projection is the right tool when you are looking for the closest approximation in a given direction. This section provides guidance on how to choose.

SituationToolWhat You Use
Multiple simultaneous linear conditionsSystem of equationsRow reduction, Ax = b, pivot analysis
A rule that acts consistently on vectorsLinear transformationMatrix multiplication, composition, T(x) = Ax
Structure of a solution or data setVector spaces, rank, nullityColumn space, null space, Rank-Nullity Theorem
Best approximation or data fittingProjection, least squaresNormal equations, ATAx̂ = ATb
Repeated action, long-run behaviorEigenvalues, diagonalizationCharacteristic equation, eigenvectors, Ak

In practice, many problems combine more than one of these tools. A Markov chain requires eigenvalues to find the steady state, matrix multiplication to evolve the state vector, and vector space ideas to understand the structure of solutions. A PCA problem requires eigenvalues, symmetric matrices, projections, and orthogonality. The goal of this chapter is to give you practice moving fluidly across all five categories.

Sections 14.3 – 14.6
Modeling with Systems, Transformations, Spaces, and Projections
Resource Allocation · Traffic Flow · Graphics Pipelines · Least Squares Setup
Section 14.3

Modeling with Systems

Systems of linear equations appear in almost every quantitative field. The key insight is that a real-world situation with multiple constraints — machine time, budget, ingredient quantities, flow conservation rules — often becomes a system Ax = b. Row reduction then reveals whether the constraints can all be satisfied, how many degrees of freedom remain, and what all valid solutions look like.

Resource Allocation

A manufacturing plant produces three products: desks, chairs, and bookcases. Each product requires time on three machines: a cutter, a sander, and a finisher. The available machine hours per week are fixed. Let x1, x2, x3 represent weekly production quantities. Each machine constraint gives one linear equation. The system Ax = b records all three simultaneously. The rank of A tells you whether a unique production plan exists; the nullity tells you whether there is a family of valid plans.

Traffic Flow and the Rank-Nullity Theorem

In a city grid, flow conservation at each intersection gives one equation per node. Row reducing the resulting system may reveal free variables — the degrees of freedom in routing traffic. The null space of the coefficient matrix describes how traffic can be redistributed without violating any intersection balance. The Rank-Nullity Theorem tells you exactly how many such free parameters exist.

Example 14.3.1
Solving a resource allocation system
  • ProblemA craftsperson makes tables and chairs. Each table requires 4 hours of carpentry and 2 hours of finishing. Each chair requires 2 hours of carpentry and 1 hour of finishing. Available: 24 hours carpentry, 12 hours finishing. How many of each can be made to use all time?
  • ThinkTwo unknowns, two equations. Write and row-reduce the augmented matrix.
  • RepresentLet x1 = tables, x2 = chairs. Carpentry: 4x1 + 2x2 = 24. Finishing: 2x1 + x2 = 12.
  • Calculate Augmented matrix: [ 4 2 | 24 ] [ 2 1 | 12 ] Multiply row 2 by 2: [ 4 2 | 24 ] [ 4 2 | 24 ] Subtract row 1 from row 2: [ 4 2 | 24 ] [ 0 0 | 0 ]
  • CheckRow 2 becomes all zeros, so the system is dependent. The finishing constraint was automatically satisfied whenever carpentry was satisfied — both equations were multiples of each other.
  • InterpretFrom row 1: 4x1 + 2x2 = 24, so x1 = 6 − x2/2. With x2 = t (free), the solution is x1 = 6 − t/2, x2 = t, for any non-negative integer t with t even. The craftsperson can make 6 tables and 0 chairs, 5 tables and 2 chairs, 4 tables and 4 chairs, and so on. The free variable represents genuine production flexibility — the model allows many valid schedules.
Answer: Infinitely many valid production plans. The family is parameterized by the number of chairs t: x1 = 6 − t/2, x2 = t.
Section 14.4

Modeling with Matrix Transformations

Linear transformations described by matrices are the backbone of computer graphics, robotics, signal processing, and physical simulations. When you compose transformations by multiplying their matrices, you are building a pipeline — a sequence of operations that moves, rotates, scales, or reflects objects in a consistent and reversible way. The order of multiplication matters because matrix multiplication is not commutative.

Homogeneous Coordinates

Scaling and rotation are standard linear transformations. Translation, however, is not linear in ordinary coordinates — shifting a vector by a fixed amount does not pass through the origin. Computer graphics solves this by using homogeneous coordinates: a 2D point (x, y) is represented as the 3D column vector (x, y, 1)T. A translation by (tx, ty) then becomes:

T = | 1 0 t_x | | 0 1 t_y | | 0 0 1 | T · (x, y, 1)^T = (x + t_x, y + t_y, 1)^T

With homogeneous coordinates, the full graphics pipeline — scale, rotate, translate — can be expressed as a single matrix product, which is highly efficient for computation.

Example 14.4.1
Building a 2D transformation pipeline in homogeneous coordinates
  • ProblemA triangle has a vertex at (1, 0). Apply in order: (1) scale by factor 2, (2) rotate 90° counterclockwise, (3) translate by (3, 1). Find the image of this vertex.
  • ThinkWrite each step as a 3×3 matrix in homogeneous coordinates. Compose by multiplying in the correct order: T · R · S (rightmost matrix is applied first).
  • RepresentVertex in homogeneous form: v = (1, 0, 1)T. Scale by 2: S = | 2 0 0 | | 0 2 0 | | 0 0 1 | Rotate 90° CCW: R = | 0 -1 0 | | 1 0 0 | | 0 0 1 | Translate (3,1): T = | 1 0 3 | | 0 1 1 | | 0 0 1 |
  • Calculate Step 1 — R · S: | 0 -1 0 | | 2 0 0 | | 0 -2 0 | | 1 0 0 | · | 0 2 0 | = | 2 0 0 | | 0 0 1 | | 0 0 1 | | 0 0 1 | Step 2 — M = T · (R · S): | 1 0 3 | | 0 -2 0 | | 0 -2 3 | | 0 1 1 | · | 2 0 0 | = | 2 0 1 | | 0 0 1 | | 0 0 1 | | 0 0 1 | Apply M to v = (1, 0, 1)^T: | 0 -2 3 | | 1 | | 0·1 + (-2)·0 + 3·1 | | 3 | | 2 0 1 | · | 0 | = | 2·1 + 0·0 + 1·1 | = | 3 | | 0 0 1 | | 1 | | 0·1 + 0·0 + 1·1 | | 1 |
  • CheckThe last component is 1, confirming the result is a valid homogeneous point.
  • InterpretThe vertex (1, 0) maps to (3, 3) after scaling by 2, rotating 90°, and translating by (3, 1). The composed matrix M allows all three operations to be applied to any vertex in a single multiplication — this is why matrix composition is so efficient in graphics pipelines.
Answer: Vertex (1, 0) maps to (3, 3) under the composed pipeline.
Section 14.5

Modeling with Vector Spaces

Vector spaces appear in modeling whenever you need to understand the structure of a collection of objects — solutions, signals, states, or data — that can be combined and scaled.

Recall from Chapter 10 that the complete solution to Ax = b has the form x = xp + xh, where xp is any particular solution and xh ranges over the null space of A. In applications, the null space often describes the degrees of freedom: how much freedom you have to adjust a solution while still satisfying all the constraints. The dimension of the null space (the nullity) tells you how many free parameters describe the family of solutions.

In signal processing and data analysis, signals or data vectors often live in a high-dimensional space, but the meaningful content occupies a much lower-dimensional subspace. If you can find a basis for this subspace, you can represent the data efficiently using just the coordinates relative to that basis. This is the idea behind data compression and dimensionality reduction.

Section 14.6

Modeling with Projections

Projection is the mathematical idea behind approximation. When you cannot satisfy a condition exactly, you look for the closest possible approximation in a given direction or subspace. This idea appears in data fitting, signal decomposition, error correction, and many other settings.

Recall from Chapter 11 that the projection of b onto the column space of a matrix A (with linearly independent columns) is:

Projection Formula and Least Squares Setup

Projection of b onto col(A): b̂ = A(A^T A)^{-1} A^T b

Least squares solution x̂ satisfies: A^T A x̂ = A^T b (normal equations)

Residual vector: r = b - A x̂

The residual vector r is orthogonal to the column space of A. When r = 0, the fit is exact. When r is nonzero, the fit is the closest possible approximation.

Sections 14.7 – 14.9
Least Squares, Computer Graphics, and Network Models
Normal Equations · Data Fitting · Graphics Pipeline · Incidence Matrix · Flow Conservation
Section 14.7

Least Squares and Data Fitting

In most real data problems, the data does not lie exactly on a line, plane, or other simple model. Least squares provides a principled way to find the best-fit model when an exact solution does not exist. The setup is always the same: form the matrix equation Ac = y by substituting each data point into the model, recognize that the system is overdetermined, and solve the normal equations ATAĉ = ATy.

Example 14.7.1
Fitting a line to temperature data
  • ProblemA scientist measures temperature y (°C) at times x (hours): (0, 4), (1, 6), (2, 7), (3, 9). Find the least squares best-fit line y = c0 + c1x.
  • ThinkFour data points, two unknowns. The system is overdetermined — no line passes through all four points exactly. Use the normal equations.
  • Represent A = | 1 0 | y = | 4 | | 1 1 | | 6 | | 1 2 | | 7 | | 1 3 | | 9 |
  • Calculate A^T A = | 1 1 1 1 | | 1 0 | | 4 6 | | 0 1 2 3 | · | 1 1 | = | 6 14 | | 1 2 | | 1 3 | A^T y = | 1 1 1 1 | | 4 | | 26 | | 0 1 2 3 | · | 6 | = | 55 | | 7 | | 9 | Normal equations: | 4 6 | | c_0 | | 26 | | 6 14 | | c_1 | = | 55 | det(A^T A) = 4(14) - 6(6) = 56 - 36 = 20 (A^T A)^{-1} = (1/20) | 14 -6 | | -6 4 | ĉ = (1/20) | 14 -6 | | 26 | (1/20) | 364 - 330 | (1/20) | 34 | | 1.7 | | -6 4 | | 55 | = | -156 + 220 | = | 64 | = | 3.2 |
  • CheckAt x = 0: predicted 1.7, actual 4, residual = 2.3. At x = 3: predicted 11.3, actual 9, residual = −2.3. Nonzero residuals confirm this is an approximation, not an exact fit. The residuals have opposite signs, which is typical of a good least squares fit.
  • InterpretBest-fit line: y = 1.7 + 3.2x. Temperature starts at about 1.7°C and increases by approximately 3.2°C per hour. The slope is the rate of increase; the intercept is the estimated initial temperature. Neither is exact because no line passes through all four data points. This is the best possible linear approximation in the least squares sense.
Answer: Best-fit line is y = 1.7 + 3.2x. Residual vector r = (2.3, 1.1, −1.1, −2.3)T.
Least Squares vs. Exact Solution

If a student sees an inconsistent overdetermined system and reports "no solution exists," they are technically correct about the exact problem — but wrong about the modeling task. The correct response is to use the normal equations to find the least squares solution. Reporting "no solution" without proceeding to least squares leaves the modeling problem unfinished.

Section 14.8

Computer Graphics Transformations

Every image rendered on a screen involves linear transformations applied to vectors representing positions, vertices, and normals. The standard 2D transformation matrices, already studied in Chapter 7, form the building blocks of a graphics pipeline.

TransformationMatrix (2D)Effect
Scale by (sx, sy)[[s_x, 0], [0, s_y]]Stretch or shrink each axis independently
Rotate by θ (CCW)[[cos θ, −sin θ], [sin θ, cos θ]]Rotate every vector by angle θ around origin
Reflect (x-axis)[[1, 0], [0, −1]]Flip y-coordinates
Horizontal shear by k[[1, k], [0, 1]]Slant horizontally; y-coordinates unchanged

In 3D, a rotation about the z-axis by angle θ acts on (x, y, z)T by leaving z unchanged and rotating the xy-plane. In 3D homogeneous coordinates, a point (x, y, z) is embedded as (x, y, z, 1)T, and all translations, rotations, and scales combine into a single 4×4 matrix product. Modern graphics processors are designed to multiply 4×4 matrices and 4-component vectors with extreme efficiency.

MGU Library Connection
  • MGU Computer Science — Graphics Programming: Extensions into OpenGL, 3D rendering pipelines, and shader programming
  • MGU Robotics — Transformation matrices for arm kinematics and coordinate frame changes
  • Chapter 7 of this textbook: Linear Transformations (standard matrices, composition, inverse transformations)
Section 14.9

Network and Flow Models

Networks appear throughout science, engineering, economics, and social analysis. Linear algebra provides tools for analyzing flow, connectivity, and behavior through the incidence matrix — the matrix encoding the directed structure of a network.

In a flow network, conservation requires that the flow entering each node equals the flow leaving it (adjusted for supply or demand at that node). Writing one equation per node and one unknown per edge gives a system Ax = b. The rank of A gives the number of independent conservation constraints; the nullity counts the degrees of freedom in routing.

Example 14.9.1
Analyzing a four-node flow network
  • ProblemA network has four nodes and five directed edges: (1→2), (1→3), (2→4), (3→4), (2→3). External supply enters node 1 (10 units) and exits node 4 (10 units). Find all valid flow assignments.
  • ThinkWrite one conservation equation per node. This gives four equations in five unknowns — expect free variables.
  • RepresentLet x1,...,x5 be flows on edges 1–5. Node 1 (source): x_1 + x_2 = 10 Node 2 (interior): x_1 = x_3 + x_5 → -x_1 + x_3 + x_5 = 0 Node 3 (interior): x_2 + x_5 = x_4 → -x_2 - x_5 + x_4 = 0 Node 4 (sink): x_3 + x_4 = 10
  • Calculate Augmented matrix: [ 1 1 0 0 0 | 10 ] [ -1 0 1 0 1 | 0 ] [ 0 -1 0 1 -1 | 0 ] [ 0 0 1 1 0 | 10 ] Row reducing yields one free variable (e.g., x_5 = t). General solution: x_1 = 10 - t - x_2, determined by t and any split at node 1.
  • CheckSet t = 0 and verify: if x5 = 0 (no flow on diagonal edge), the 10 units split between two paths with no loop routing. Conservation holds at all four nodes.
  • InterpretThe free variable x5 represents the routing choice on the diagonal edge. The network can carry all 10 units along many different paths. The nullity equals 1, reflecting one independent loop in the network. Operators can choose t based on additional criteria such as capacity limits or travel times.
Answer: One-parameter family of solutions. The free parameter t represents flow along the diagonal edge.
Sections 14.10 – 14.12
Markov Models, Principal Components, and Data Science
Transition Matrices · Steady States · Covariance · Eigenvectors · ML Connections
Section 14.10

Markov Models

A Markov chain is a model for a system that moves among a finite set of states. At each step, the system transitions from its current state to a new state according to fixed probabilities that depend only on the current state, not on the history. The behavior is captured by a transition matrix P, where column j gives the probabilities of transitioning from state j to each other state. Columns must sum to 1.

If xk is the state vector at step k, then xk+1 = Pxk, and after k steps from an initial state x0, we have xk = Pkx0. This is why diagonalization matters: it simplifies computing Pk for large k.

Under mild conditions, the state vector converges to a unique steady-state vector q satisfying Pq = q — that is, an eigenvector of P with eigenvalue λ = 1. Computing the steady state reduces to finding the eigenspace of P corresponding to eigenvalue 1, then normalizing so the entries sum to 1.

Example 14.10.1
Market share Markov model
  • ProblemTwo companies compete for customers. Each year, company A retains 80% of its customers and loses 20% to B. Company B retains 70% and loses 30% to A. Currently A holds 60%, B holds 40%. What is the long-run steady state?
  • Represent P = | 0.80 0.30 | x_0 = | 0.60 | | 0.20 0.70 | | 0.40 |
  • Calculate — After 1 Year x_1 = P · x_0 = | 0.80·0.60 + 0.30·0.40 | | 0.60 | | 0.20·0.60 + 0.70·0.40 | = | 0.40 |
  • Calculate — Steady State Solve (P - I)q = 0: P - I = | -0.20 0.30 | | 0.20 -0.30 | Row reduce → -0.20 q_1 + 0.30 q_2 = 0 → q_1 = 1.5 q_2 Normalize (q_1 + q_2 = 1): q_2 = 2/5 = 0.40, q_1 = 3/5 = 0.60 q = (0.60, 0.40)^T
  • CheckThe current state vector (0.60, 0.40)T is already the steady state, which is why multiplying by P left it unchanged.
  • InterpretNo matter where market shares start, they converge to company A holding 60% and B holding 40%. The eigenvalue 1 corresponds to this stable long-run distribution. The other eigenvalue of P is 0.5, corresponding to the transient behavior that decays as (0.5)k → 0 with each additional year.
Answer: Steady state is q = (0.60, 0.40)T. Company A holds 60%, B holds 40% in the long run.
Transition Matrix Orientation

Column j of the transition matrix P gives the probabilities of transitioning from state j. Reading rows as "from" gives the wrong interpretation and wrong calculations. If P is oriented correctly, multiplying P on the left of a column state vector evolves the system one step forward. Transposing the setup reverses every calculation and every conclusion.

Section 14.11

Principal Components as an Introduction

Principal Component Analysis (PCA) is one of the most widely used techniques in data science, statistics, and machine learning. At its core, it is a problem in linear algebra: given a data matrix, find the directions in which the data varies the most.

These directions are called principal components. They are the eigenvectors of the data's covariance matrix, and the corresponding eigenvalues measure how much variation occurs in each direction. The data can then be projected onto the first few principal components to obtain a lower-dimensional representation that retains as much variation as possible.

The Mathematical Setup

Given a data matrix X with rows representing observations and columns representing variables, center the data by subtracting the mean of each column. Form the covariance matrix:

Covariance Matrix and Principal Components

C = (1/(m−1)) · X^T X    where X is the centered data matrix, m = number of observations.

C is symmetric. Its eigenvectors with the largest eigenvalues are the principal components — the directions of maximum variation. Projecting data onto the first k principal components gives the best rank-k approximation to the data.

PCA Use CaseWhat Linear Algebra Does
Data visualizationReduces 100-dimensional data to 2–3 dimensions for plotting
Data compressionStores fewer numbers while preserving essential patterns
Noise reductionSeparates signal (high-variance eigenvectors) from noise (low-variance)
ML preprocessingRemoves redundant features before training a classifier
All the Tools Come Together in PCA

PCA connects eigenvalues, eigenvectors, symmetric matrices, projections, and least squares into a single framework. Every major idea from this book plays a role: eigenvalues measure variation, eigenvectors identify direction, projections perform the dimensionality reduction, and orthogonality ensures the components are independent.

MGU Library Connection
  • MGU Data Science Foundations — Full treatment of PCA, covariance, and dimensionality reduction
  • MGU Machine Learning Mathematics — Singular value decomposition (SVD), image compression, recommender systems, and neural network preprocessing
Section 14.12

Linear Algebra in Data Science and Machine Learning

The language of modern data science and machine learning is linear algebra. Every major idea in these fields rests on the tools you have learned in this book.

Data as Matrices

A data set is a matrix. Each row is an observation; each column is a feature. The rank of the data matrix tells you how many independent sources of variation are present. The null space describes combinations of features that contribute nothing additional. The column space is the space of all patterns the model can produce.

Linear Regression

The most basic machine learning model is linear regression: predict a target variable as a linear combination of features. This is exactly the least squares problem from Section 14.7, generalized to multiple features. The solution minimizes the sum of squared errors, found by solving the normal equations ATAŵ = ATy.

Neural Networks and Linear Layers

Each layer of a neural network computes a matrix-vector product followed by a nonlinear function. The matrix contains the learned weights; the input vector contains the activations from the previous layer. Training a neural network involves adjusting these matrices to minimize prediction error — a massive optimization problem built on linear algebra.

Singular Value Decomposition (Preview)

Beyond eigenvalues, data science uses the singular value decomposition (SVD), which generalizes diagonalization to non-square matrices. Every matrix A can be written as A = UΣVT, where U and V have orthonormal columns and Σ is diagonal with non-negative entries called singular values. The SVD underlies PCA, low-rank approximation, image compression, recommender systems, and text analysis. The eigenvalue and orthogonality ideas from Chapters 11–13 are the direct prerequisites.

Section 14.13
How Linear Algebra Prepares Students for Calculus
Vector-Valued Functions · Jacobian Matrices · Gradients · Tangent Planes · Differential Equations
Section 14.13

How Linear Algebra Prepares Students for Calculus

This book was designed as a bridge. You have built a rich understanding of vectors, matrices, transformations, spaces, dimension, and structure — all without calculus. Now it is time to look forward and see what calculus will add to this foundation. This section previews calculus connections conceptually. No calculus formulas are derived here; each idea is described in terms of what you already know.

Calculus ConceptLinear Algebra FoundationWhat Calculus Adds
Vector-valued functionsVectors in ℝⁿ, component-wise operationsRates of change along the path (derivatives)
Gradient ∇fDot products, vectors in ℝⁿPartial derivatives; direction of steepest increase
Jacobian matrixLinear transformation T: ℝⁿ → ℝᵐ and its m×n matrixBest local linear approximation to a nonlinear function
Tangent planeSubspaces, flat structure in spaceLocal linear approximation to a surface at a point
Systems of DEsEigenvalues, eigenvectors, diagonalizationContinuous-time evolution; eigenvalues determine growth, decay, oscillation
OptimizationSystems of equations, least squaresSetting derivatives to zero; Lagrange multipliers

The Jacobian: A Linear Transformation at a Point

In Chapter 7, you studied linear transformations T: ℝⁿ → ℝᵐ and learned that every such transformation is represented by an m×n matrix. The Jacobian matrix is the natural extension of this idea to nonlinear functions: if F: ℝⁿ → ℝᵐ is a differentiable function, its Jacobian at any point is the m×n matrix that best describes F locally at that point. The linear transformation you already understand is what the Jacobian looks like at each instant.

Eigenvalues and Systems of Differential Equations

In Differential Equations, systems of the form x′ = Ax describe how a vector of quantities changes over time. The solution involves eigenvectors and eigenvalues of A: each eigenvector defines a mode of behavior, and the corresponding eigenvalue determines whether that mode grows (λ > 0), decays (λ < 0), or oscillates (λ complex). You already understand eigenvalues, eigenvectors, and diagonalization from Chapters 12 and 13. When you study differential equations, you will see that these linear algebra tools are the key to understanding the structure of solutions.

Bridge to Calculus

Every major idea in this chapter has a calculus counterpart. Vectors become vector-valued functions; matrix transformations become Jacobian matrices; eigenvalues appear in the solution of systems of differential equations; least squares becomes the foundation of statistical regression in continuous settings. The structural thinking you have developed — decomposing problems, identifying the right mathematical object, checking answers, and interpreting results — is exactly the thinking that succeeds in Calculus, Multivariable Calculus, and every advanced field of applied mathematics.

Sections 14.14 – 14.15
Capstone Project Design and Common Mistakes
Project Structure · Suggested Topics · Ten Recurring Errors
Section 14.14

Capstone Project Design

The capstone project is an opportunity to apply linear algebra to a problem you find genuinely interesting. A strong capstone project goes beyond computation: it connects a real question to a linear model, uses appropriate linear algebra tools, interprets the results, and reflects on the limitations of the model.

Seven-Step Project Structure

Suggested Topics

Topic AreaLinear Algebra ToolExample Question
Sports or climate dataLeast squaresDoes a team's win rate correlate linearly with scoring differential?
Population ecologyMarkov chains, diagonalizationWhat is the long-run age distribution of a species?
Transportation networksFlow systems, incidence matrixHow can a city route all commuters while minimizing congestion?
2D/3D graphics pipelineTransformation matrices, compositionDesign an animated logo using rotation, scale, and translation.
Economic input-outputSystems Ax = b, (I−A)x = dWhat total production satisfies all consumer demands in a two-sector economy?
Real data setPCA, covariance matrixWhat are the dominant patterns in a set of 15 health variables?
MGU Library Connection
  • Appendix S: Capstone Project Rubric — full assessment rubric for evaluating projects
  • Appendix T: Advanced Mathematics and Data Science Readiness Check — self-assessment before advancing
  • Appendix R: Calculus Readiness Bridge Guide — preview of how each chapter connects to Calculus and Multivariable Calculus
Section 14.15

Common Linear Modeling Mistakes

Mistake 1 — Using a linear model for a nonlinear relationship
"The linear model gives negative outputs, but I'll use it anyway."

If doubling an input does not double the output, the relationship is nonlinear. Always check whether a linear model is appropriate before committing to it. Negative population counts or probability values are a signal to reconsider.

Mistake 2 — Setting up the system in the wrong orientation
"I multiplied a row vector on the left of A to get the next state."

Column vectors represent states, data, or solutions; rows represent equations or transformations applied from the left. Confusing row and column orientation leads to wrong dimensions and wrong answers. In Markov chains: always use column state vectors and left-multiply by P.

Mistake 3 — Forgetting free variables
"The system has infinitely many solutions so there is no answer."

Free variables describe genuine degrees of freedom in a solution. Forgetting them gives an incomplete answer. Parameterize the solution family using a free variable and interpret what the parameter represents in context.

Mistake 4 — Trying to solve an overdetermined system exactly instead of using least squares
"The system is inconsistent — no solution exists."

When data does not fit a model exactly, use the normal equations ATAx̂ = ATb to find the least squares solution. Reporting "no solution" without proceeding to least squares leaves the modeling problem unfinished.

Mistake 5 — Reading the transition matrix in the wrong direction
"Row 2, column 3 gives the probability of going from state 3 to state 2." (Stated as if rows are current states.) Correct: Column j of P gives the probabilities of transitioning from state j. Entry P_{ij} = probability of going from state j to state i.

Transposing the interpretation swaps "from" and "to" for every entry and gives completely different (wrong) predictions.

Mistake 6 — Forgetting to normalize the steady-state vector
Reporting the steady state as q = (3, 2)T without normalizing. Correct: q = (3/5, 2/5)T = (0.60, 0.40)T. The entries must sum to 1 to be a valid probability distribution.

Finding the eigenvector direction is only half the work. Always normalize so the entries represent valid proportions.

Mistake 7 — Composing transformations in the wrong order
"Scale first, then rotate, so I compute R · S — wait, or is it S · R?"

If you apply S first and then R, the combined matrix is R · S (rightmost applied first). Test with a specific vector whenever unsure. Drawing the geometric effect of each operation in order is a reliable check.

Mistake 8 — Confusing principal components with columns of the data matrix
"The principal components are just the columns of my data table."

Principal components are eigenvectors of the covariance matrix C = (1/(m−1))XTX. They point in the directions of greatest variation in feature space. The columns of the data matrix represent features; principal components represent directions across features.

Mistake 9 — Stopping at the computation without interpreting the result
"The eigenvalue is λ = 0.7. Done."

A matrix answer without interpretation is only half-finished. In a Markov model, λ = 0.7 means the corresponding transient mode decays as (0.7)k. After 10 steps it contributes only (0.7)10 ≈ 0.028 of its initial magnitude. Always connect the number back to the original problem.

Mistake 10 — Assuming a model is exact when it is only approximate
"My linear model gives a precise prediction for every case."

Every model is an approximation. Real data has noise; real networks have complexity; real markets have nonlinear dynamics. A good modeler identifies the assumptions, tests the model where possible, and reports results with appropriate caveats about limitations and scope.

Section 14.16

Readiness for Calculus and Advanced Applied Fields

You have reached the end of this book. Before you go forward, it is worth pausing to take stock of what you know and where it leads.

What You Now Know

What Comes Next
Calculus and Beyond

Calculus adds limits, derivatives, and integrals. The geometry of curves, instantaneous rates of change, and the accumulation of continuous quantities are the new ingredients. The setting — ℝⁿ, vectors, coordinate geometry — is already familiar to you.

Multivariable Calculus extends derivatives and integrals to functions of several variables. Gradients, Jacobians, tangent planes, and directional derivatives all combine linear algebra with limits. The language of linear algebra appears at every step.

Differential Equations studies how functions change over time. Systems of differential equations involve matrices and eigenvalues directly. The solution theory for linear systems is built almost entirely on the tools in this book.

Statistics, Data Science, and Machine Learning use matrices, least squares, covariance, principal components, and regression constantly. Every data analysis tool you will encounter was designed using linear algebra.

You are ready. The next step is yours.

MGU Library Connection
  • Appendix R: Calculus Readiness Bridge Guide — detailed preview of how each Linear Algebra chapter connects to Calculus and Multivariable Calculus
  • Appendix T: Advanced Mathematics and Data Science Readiness Check — self-assessment before continuing
  • MGU Calculus, Chapter 1 — Limits and the Idea of Instantaneous Change
  • MGU Multivariable Calculus — Gradients, Jacobians, and Vector-Valued Functions
  • MGU Differential Equations — Systems of the Form x' = Ax
  • MGU Data Science Foundations — Matrices in Regression, PCA, and Model Evaluation
Chapter 14 — Back Matter

Chapter Summary

Key Terms Review

linear model
A mathematical model in which relationships are expressed as linear equations, matrix products, or vector operations.
modeling process
The five-step cycle: describe, represent, analyze, check, and interpret. Connects a real situation to a mathematical answer.
least squares solution
The vector x̂ minimizing ‖Ax − b‖². Found by solving the normal equations ATAx̂ = ATb.
normal equations
The system ATAx̂ = ATb whose solution gives the least squares approximation to an overdetermined system.
residual vector
r = b − Ax̂. Measures the error of the least squares fit. Orthogonal to the column space of A.
homogeneous coordinates
A system embedding ℝⁿ inside ℝⁿ⁺¹ by appending a 1, enabling translations to be expressed as matrix multiplication.
transition matrix
An n×n matrix P where column j gives the probabilities of transitioning from state j. Columns sum to 1.
steady-state vector
A probability vector q satisfying Pq = q; an eigenvector of the transition matrix with eigenvalue 1.
incidence matrix
The matrix encoding the directed structure of a network; entries are +1, −1, or 0 according to edge direction.
covariance matrix
The symmetric matrix C = (1/(m−1))XTX measuring how variables vary together in a centered data set X.
principal components
The eigenvectors of the covariance matrix of a data set, pointing in the directions of greatest variation.
Jacobian matrix
The matrix of partial derivatives of a vector-valued function; the linear approximation of the function near a point.
gradient
A vector of partial derivatives of a scalar-valued multivariable function; points in the direction of steepest increase.
singular value decomposition
The factorization A = UΣVT of any matrix into orthonormal factor matrices and diagonal singular values. Generalizes diagonalization.

Concept Review Questions

  1. Name the five steps of the linear algebra modeling process and give a brief description of each.
  2. For each situation below, identify which linear algebra tool is most directly applicable: (a) twelve equations constraining eight unknowns, (b) finding the best-fit plane through a set of data points, (c) computing the long-run market share in a three-company industry, (d) composing a 3D rotation then a scale then a translation, (e) finding directions of greatest variation in a 50-variable data set.
  3. Explain in your own words what the residual vector in a least squares problem represents. What does it mean when the residual is zero?
  4. Why does a Markov chain's steady-state vector satisfy Pq = q? What does this equation mean in terms of the eigenvalue equation?
  5. A student sets up a least squares problem but tries to solve it by row-reducing Ax = b directly, finds the system inconsistent, and concludes no solution exists. What should the student do instead?
  6. Explain why principal components are the eigenvectors of the covariance matrix and not the columns of the data matrix itself.
  7. How does the Jacobian matrix generalize the idea of a linear transformation from Chapter 7? What does the Jacobian do that the transformation matrix does not need to do?

Model Selection Practice

Part AIdentifying the Right Tool
  1. A city engineer is designing a water distribution network with six pumping stations and nine pipes. Describe how linear algebra applies. What matrix is formed? What does the null space represent?
  2. A researcher has 200 observations on 15 health variables and wants to reduce to the three most important directions. Which tool applies? What matrix does she form? What does she look for in that matrix?
  3. A company sells three products in five regions. Using last year's sales data (a 5×3 matrix), explain how to check whether any two products have perfectly redundant sales patterns across regions.
  4. A game designer wants to animate a character that first rotates 45° counterclockwise, then scales to 150%, then moves right by 10 units. Describe the three matrices (in 2D homogeneous coordinates) and how to compose them.

Application Problems

For each problem: define all variables, set up the matrix model, carry out the analysis, check, and interpret fully in context.

Part BLeast Squares Data Fitting
  1. Fit a line to the data: (1, 2), (2, 3), (3, 5), (4, 4). Set up the matrix A, form the normal equations ATAc = ATy, solve for the best-fit line y = c0 + c1x, compute the residual vector, and interpret the slope and intercept.
  2. Fit a quadratic model y = c0 + c1x + c2x2 to the data (−1, 3), (0, 1), (1, 2), (2, 7). Set up the 4×3 matrix A and the normal equations, then solve for c0, c1, c2.
  3. A student measures study time (min) and quiz scores for five friends: (20, 55), (40, 72), (30, 63), (60, 85), (50, 78). Find the least squares line, interpret the slope, and use the line to estimate the score for 45 minutes of studying.
Part CMarkov Models
  1. A three-state Markov chain has the transition matrix P shown below. Starting from x0 = (1, 0, 0)T, compute x1 and x2. Interpret each state vector. P = | 0.5 0.2 0.3 | | 0.3 0.6 0.1 | | 0.2 0.2 0.6 |
  2. A social media platform models user engagement in three states: Active (A), Casual (C), Inactive (I). Given the transition matrix P = [[0.6, 0.3, 0.1], [0.3, 0.5, 0.2], [0.1, 0.2, 0.7]], find x1 starting from x0 = (0.4, 0.4, 0.2)T. Write the eigenvalue equation whose solution gives the steady state.
  3. For the market share model in Example 14.10.1, change company A's retention rate to 70% and B's to 80%. Recompute the transition matrix and find the new steady-state distribution.
Part DTransformation Pipelines
  1. A square has vertices at (0,0), (1,0), (1,1), (0,1). Apply in 2D homogeneous coordinates: (1) horizontal shear with k = 1, (2) reflection across the y-axis. Write both matrices, compose in the correct order, find the images of all four vertices, and describe the resulting quadrilateral.
  2. Design a 2D animation pipeline in homogeneous coordinates for a logo that should be: first scaled to half size, then rotated 60° clockwise, then translated to position (5, 3). Write the three matrices and compose them into a single matrix M. Find the image of the point (2, 0).
  3. Explain in words why translation is not a linear transformation in ordinary 2D coordinates, and show how embedding in homogeneous coordinates solves this problem.
Part EApplied Systems
  1. Leontief Input-Output Model. A small economy has two industries: agriculture and manufacturing. Agriculture uses 20% of its own output and 40% of manufacturing's output as inputs. Manufacturing uses 30% of agriculture's output and 10% of its own output as inputs. Consumer demand is 100 units of agriculture and 80 units of manufacturing. Set up the system (I − A)x = d and solve for total production levels. Interpret the solution.
  2. Network Flow. A directed network has five nodes and six edges with the following directed structure: (1→2), (1→3), (2→4), (3→4), (2→3), (4→5). External supply enters node 1 (20 units); node 5 is the sink. Write the conservation equations and set up the augmented matrix. Identify all free variables and interpret them as routing choices.

Calculus Readiness Reflection

  1. A vector-valued function r(t) = (2t, t²)T describes a path in the plane. Describe in your own words what the vector r(1) = (2, 1)T represents. No calculus needed.
  2. Explain how the Jacobian matrix of a function F: ℝ² → ℝ² is analogous to the matrix of a linear transformation from Chapter 7. What does the Jacobian do that the linear transformation matrix does not need to do?
  3. A system of two differential equations is written as x′ = Ax where A = [[3, 1], [1, 3]]. Without solving any differential equations, find the eigenvalues and eigenvectors of A and explain intuitively what these values would tell a Differential Equations student about the long-run behavior of the system.
  4. In optimization, finding a constrained extremum requires solving a system of linear equations (the critical point conditions). Why does linear algebra appear naturally here? What do the solution vectors represent geometrically?

Error Analysis

Problem 1
A student reads a Markov transition matrix and says: "Row 2, column 3 gives the probability of going from state 3 to state 2. So I multiply state vectors on the right of P."

Identify both errors. State the correct orientation of the transition matrix and the correct direction of multiplication.

Problem 2
A student builds a pipeline: "first scale by 3, then rotate by 60°," and computes the composed matrix as S · R (scale times rotation).

Is this correct? If not, which order should be used? Show with the vector (1, 0)T that the two matrices give different results.

Problem 3
After fitting a least squares model, a student finds a residual vector r ≠ 0 and concludes: "My model is wrong — the answer should give r = 0."

Explain why this reasoning is incorrect. When should a student expect a nonzero residual in least squares?

Problem 4
A student finds the steady-state eigenvector for a 3-state Markov chain and reports q = (6, 4, 2)T.

What is wrong with this answer? Correct it and explain the normalization requirement.

Problem 5
A student says: "The principal components of my data are just the column vectors of the original data matrix X."

Explain the error precisely. What matrix must be computed, and what are principal components in terms of that matrix?

Chapter 14 Checkpoint

Chapter 14 Checkpoint
Complete without assistance — assesses the five-step modeling process, least squares, Markov chains, transformations, and calculus readiness
Section 1 — Modeling Process and Tool Selection
  1. Name the five steps of the linear algebra modeling process. For each step, give one sentence describing what happens at that step.
  2. A factory has four machines and three product lines. Each product requires a given number of hours on each machine. The available machine hours are fixed. State which linear algebra tool applies and what the null space represents in this context.
  3. A climatologist wants to fit a quadratic trend y = c0 + c1t + c2t2 to 50 years of temperature data. Which linear algebra tool is appropriate? Why is an exact solution unlikely to exist?
Section 2 — Least Squares
  1. Given data points (1, 2), (2, 3), (3, 5), (4, 4), set up the matrix A and vector y for a least squares line y = c0 + c1x. Write out the normal equations ATAĉ = ATy with numerical values.
  2. Solve the normal equations from problem 4 for ĉ. State the best-fit line and compute the residual vector r = y − Aĉ. Interpret the slope and intercept in context.
Section 3 — Markov Chains
  1. A three-state Markov chain has transition matrix P = [[0.5, 0.2, 0.3], [0.3, 0.6, 0.1], [0.2, 0.2, 0.6]]. Compute x1 from x0 = (1, 0, 0)T. Describe what x1 represents.
  2. For the same matrix P, write the eigenvalue equation whose solution gives the steady-state distribution. (You do not need to solve it here.) What normalization condition must the steady-state vector satisfy?
Section 4 — Transformation Pipeline
  1. In 2D homogeneous coordinates, write the 3×3 matrix for each operation: (a) scale by factor 3 in both directions, (b) rotate 90° counterclockwise, (c) translate by (−2, 4). Compose them in the correct order for the pipeline: scale, then rotate, then translate.
  2. Apply your composed matrix from problem 8 to the point (1, 0) in homogeneous coordinates. State the final 2D position and verify the last component is 1.
Section 5 — Calculus Readiness and Reflection
  1. Explain in one paragraph how the Jacobian matrix is related to the linear transformation matrix from Chapter 7. What additional idea does the Jacobian require beyond what you studied in Chapter 7?
  2. A system of differential equations is governed by the matrix A = [[2, 1], [1, 2]]. Find the eigenvalues and eigenvectors of A. Describe in words (no differential equations required) what the eigenvalues suggest about whether each mode of behavior grows, decays, or stays constant over time.
  3. Describe a real-world situation — not mentioned in this chapter — that could be modeled using: (a) a system of linear equations, (b) a Markov chain, (c) least squares. For each, identify the vectors and matrices involved and the question the model would answer.

Capstone Project Guide

See also Appendix S: Capstone Project Rubric

The full assessment rubric, scoring guide, and peer-review checklist are in Appendix S. Use the guide below for the structural steps; use the rubric to evaluate your work before submitting.

  1. Choose a Topic. Select a real-world situation involving multiple quantities and relationships. Identify a specific question linear algebra can help answer. Use the suggested topics in Section 14.14 for ideas, or propose your own.
  2. Collect or Construct Data. Find real data or construct a realistic numerical example that gives your model meaningful numbers to work with.
  3. Build the Linear Model. Identify the appropriate tools (system, transformation, projection, eigenvalues, or a combination). Define all vectors and matrices explicitly. Write the complete mathematical formulation.
  4. Analyze. Carry out the full linear algebra. Solve, reduce, project, diagonalize, or compute eigenvalues as needed. Show all work.
  5. Interpret. Translate every result back into the context of the original problem. Every number, vector, and matrix should have a plain-language interpretation.
  6. Identify Limitations. Describe at least two ways your model simplifies reality. Under what conditions would the linear model be unreliable or give misleading predictions?
  7. Reflect. What did you learn? What surprised you? What would you study next if you had more time and more mathematical tools?
Closing Bridge
You Have Built the Foundation

Every major idea in this chapter has a calculus counterpart. Vectors become vector-valued functions. Matrix transformations become Jacobian matrices. Eigenvalues appear in the solution of systems of differential equations. Least squares becomes the foundation of statistical regression in continuous settings.

The structural thinking you have developed here — decomposing problems into manageable pieces, identifying the right mathematical object, checking answers against the original conditions, and interpreting results meaningfully — is exactly the thinking that succeeds in Calculus, Multivariable Calculus, Differential Equations, and every advanced field of applied mathematics.

You are ready for the next step.

MGU Library Connection — Full Textbook
  • Appendix A: Algebra and Function Readiness Review
  • Appendix B: Coordinate Geometry and Trigonometry Review
  • Appendix C–P: Linear Algebra Reference Sheets (Vectors through Least Squares)
  • Appendix Q: Linear Algebra Technology Guide
  • Appendix R: Calculus Readiness Bridge Guide
  • Appendix S: Capstone Project Rubric
  • Appendix T: Advanced Mathematics and Data Science Readiness Check
  • Glossary · Answer Key · Index · MGU Library Connections

Answers to selected problems may be placed in the answer key.