The Central Idea

A system such as

\[ \begin{aligned} x+2y&=5,\\ 3x-y&=4 \end{aligned} \]

contains several equations in the same unknowns.

The goal is to find values of the variables that satisfy every equation simultaneously.

Elimination simplifies the system without changing its solution set.

What Is a Linear System?

A linear system is a collection of linear equations in the same variables.

A general system of \(m\) equations in \(n\) unknowns can be written

\[ \begin{aligned} a_{11}x_1+\cdots+a_{1n}x_n&=b_1,\\ a_{21}x_1+\cdots+a_{2n}x_n&=b_2,\\ &\vdots\\ a_{m1}x_1+\cdots+a_{mn}x_n&=b_m. \end{aligned} \]

Matrix Form of a Linear System

The same system can be written compactly as

\[ \boxed{Ax=b}, \]

where

\[ A= \begin{bmatrix} a_{11}&\cdots&a_{1n}\\ \vdots&\ddots&\vdots\\ a_{m1}&\cdots&a_{mn} \end{bmatrix}, \qquad x= \begin{bmatrix} x_1\\ \vdots\\ x_n \end{bmatrix}, \qquad b= \begin{bmatrix} b_1\\ \vdots\\ b_m \end{bmatrix}. \]

This connects solving systems directly to matrix representations.

The Coefficient Matrix

The matrix \(A\) contains the coefficients of the unknown variables.

For

\[ \begin{aligned} x+2y&=5,\\ 3x-y&=4, \end{aligned} \]

the coefficient matrix is

\[ \boxed{ A= \begin{bmatrix} 1&2\\ 3&-1 \end{bmatrix}}. \]

The Augmented Matrix

To solve a system, it is often convenient to attach the right-hand side column to the coefficient matrix.

The augmented matrix of

\[ Ax=b \]

is

\[ \boxed{[A\mid b]}. \]

For the example above,

\[ \boxed{ \left[ \begin{array}{cc|c} 1&2&5\\ 3&-1&4 \end{array} \right]}. \]

Why Use an Augmented Matrix?

The variable names do not change during elimination.

Once the column order is established, we can work entirely with the numerical coefficients.

This makes row operations easier to organize, especially for large multidimensional systems.

Equivalent Systems

Two systems are equivalent if they have exactly the same solution set.

Elimination works by repeatedly replacing a system with an equivalent system that is easier to solve.

Elementary row operations preserve equivalence.

The Three Elementary Row Operations

There are three permitted elementary row operations.

  1. Interchange two rows.
  2. Multiply a row by a nonzero scalar.
  3. Add a scalar multiple of one row to another row.

These operations do not change the solution set of the system.

Row Interchange

Swapping two equations does not change which values satisfy the system.

In matrix notation,

\[ \boxed{R_i\leftrightarrow R_j}. \]

Row interchange is often used to move a convenient pivot into position.

Row Scaling

Multiplying an equation by a nonzero number preserves its solution set.

In matrix notation,

\[ \boxed{R_i\leftarrow cR_i,\qquad c\neq0}. \]

Scaling is commonly used to turn a pivot into \(1\).

Row Replacement

We may add a scalar multiple of one row to another:

\[ \boxed{R_i\leftarrow R_i+cR_j}. \]

This is the main elimination step because it allows selected entries to be turned into zero.

What Is a Pivot?

A pivot is a leading nonzero entry used to eliminate other entries in its column.

Pivot positions identify variables that are constrained directly by the system.

Columns without pivots correspond to free variables.

Row Echelon Form

A matrix is in row echelon form when:

The pivots create a staircase pattern.

Example of Row Echelon Form

\[ \boxed{ \begin{bmatrix} 1&2&-1&4\\ 0&3&5&2\\ 0&0&7&1\\ 0&0&0&0 \end{bmatrix}} \]

is in row echelon form.

Its pivot columns are the first, second, and third columns.

Reduced Row Echelon Form

A matrix is in reduced row echelon form, or RREF, if it is in row echelon form and additionally:

Example of Reduced Row Echelon Form

\[ \boxed{ \begin{bmatrix} 1&0&2&3\\ 0&1&-1&4\\ 0&0&0&0 \end{bmatrix}} \]

is in reduced row echelon form.

The first two columns contain pivots.

Gaussian Elimination

Gaussian elimination uses elementary row operations to reduce a matrix to row echelon form.

After reaching echelon form, the remaining equations are solved by back substitution.

Gauss-Jordan Elimination

Gauss-Jordan elimination continues beyond echelon form until reduced row echelon form is reached.

Once RREF is obtained, the solution structure can usually be read directly without back substitution.

Example: Gaussian Elimination

Solve

\[ \begin{aligned} x+2y&=5,\\ 3x-y&=4. \end{aligned} \]

Begin with

\[ \left[ \begin{array}{cc|c} 1&2&5\\ 3&-1&4 \end{array} \right]. \]

Eliminate the \(3\) below the first pivot:

\[ R_2\leftarrow R_2-3R_1. \]

This gives

\[ \left[ \begin{array}{cc|c} 1&2&5\\ 0&-7&-11 \end{array} \right]. \]

Back Substitution

From

\[ -7y=-11, \]

we obtain

\[ y=\frac{11}{7}. \]

Substitute into the first equation:

\[ x+2\left(\frac{11}{7}\right)=5. \]

Therefore,

\[ x=\frac{13}{7}. \]

The solution is

\[ \boxed{ (x,y)= \left( \frac{13}{7}, \frac{11}{7} \right)}. \]

Three Possible Solution Types

A linear system can have:

  1. exactly one solution;
  2. infinitely many solutions;
  3. no solution.

Elimination reveals which case occurs.

Unique Solutions

A system has a unique solution when every variable column contains a pivot and no contradiction appears.

In RREF, the solution often looks like

\[ \left[ \begin{array}{ccc|c} 1&0&0&a\\ 0&1&0&b\\ 0&0&1&c \end{array} \right]. \]

Then

\[ \boxed{x_1=a,\qquad x_2=b,\qquad x_3=c}. \]

Inconsistent Systems

A system is inconsistent if elimination produces a row of the form

\[ \boxed{ \begin{bmatrix} 0&0&\cdots&0\mid c \end{bmatrix}, \qquad c\neq0}. \]

This row represents the impossible equation

\[ 0=c. \]

Therefore the system has no solution.

Example: No Solution

Consider

\[ \begin{aligned} x+y&=2,\\ 2x+2y&=5. \end{aligned} \]

The augmented matrix is

\[ \left[ \begin{array}{cc|c} 1&1&2\\ 2&2&5 \end{array} \right]. \]

Apply

\[ R_2\leftarrow R_2-2R_1. \]

We obtain

\[ \left[ \begin{array}{cc|c} 1&1&2\\ 0&0&1 \end{array} \right]. \]

The second row means \(0=1\), so the system is inconsistent.

Free Variables

A variable whose column does not contain a pivot is called a free variable.

Free variables can vary independently and are usually represented by parameters.

When a consistent system has at least one free variable, it has infinitely many solutions.

Example: Infinitely Many Solutions

Suppose elimination gives

\[ \left[ \begin{array}{ccc|c} 1&0&2&3\\ 0&1&-1&4 \end{array} \right]. \]

The variables \(x_1\) and \(x_2\) are pivot variables, while \(x_3\) is free.

Let

\[ x_3=t. \]

Then

\[ x_1=3-2t, \qquad x_2=4+t. \]

Parametric Vector Form

The solution above becomes

\[ \begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix} = \begin{bmatrix} 3-2t\\ 4+t\\ t \end{bmatrix}. \]

Separate the fixed and variable parts:

\[ \boxed{ \begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix} = \begin{bmatrix} 3\\ 4\\ 0 \end{bmatrix} + t \begin{bmatrix} -2\\ 1\\ 1 \end{bmatrix}}. \]

Geometrically, the solution set is a line in \(\mathbb{R}^3\).

Homogeneous Systems

A system is homogeneous when

\[ \boxed{Ax=0}. \]

Every homogeneous system has at least the trivial solution

\[ x=0. \]

Therefore a homogeneous system can never be inconsistent.

Nontrivial Homogeneous Solutions

A homogeneous system has nontrivial solutions when at least one variable is free.

Equivalently,

\[ Ax=0 \]

has a nonzero solution when the columns of \(A\) are linearly dependent.

Homogeneous Solutions Form a Subspace

The solution set of

\[ Ax=0 \]

is the null space of \(A\).

Therefore it is a subspace:

\[ \boxed{\operatorname{Null}(A)=\{x:Ax=0\}}. \]

Nonhomogeneous Solution Sets Are Usually Affine

For

\[ Ax=b, \qquad b\neq0, \]

a consistent solution set generally does not pass through the origin.

If \(x_p\) is one particular solution, then every solution can be written

\[ \boxed{x=x_p+x_h}, \]

where \(x_h\) is a solution of the homogeneous system

\[ Ax_h=0. \]

Particular Plus Homogeneous Structure

Suppose

\[ Ax_p=b \]

and

\[ Ax_h=0. \]

Then

\[ A(x_p+x_h)=Ax_p+Ax_h=b. \]

Thus adding any null-space vector to a particular solution produces another solution.

Pivot Variables and Free Variables

A pivot variable corresponds to a pivot column.

A free variable corresponds to a nonpivot column.

Pivot variables are determined in terms of the free variables.

Free variables control the dimensions of the solution family.

Number of Free Variables

If a matrix \(A\) has \(n\) columns and rank \(r\), then the number of free variables in the homogeneous system is

\[ \boxed{n-r}. \]

This is exactly the nullity:

\[ \operatorname{nullity}(A)=n-\operatorname{rank}(A). \]

Elimination and Rank

The number of pivots in an echelon form of \(A\) is the rank of \(A\).

\[ \boxed{\operatorname{rank}(A)=\text{number of pivot positions}}. \]

Row reduction therefore provides a direct computational method for finding rank.

Row Operations Preserve Rank

Elementary row operations do not change the rank of a matrix.

They may alter the entries dramatically, but they preserve the essential number of independent equations.

This is why echelon form can reveal the rank of the original matrix.

Row Operations Do Not Preserve Column Vectors Individually

Row reduction preserves the solution set of an augmented system and the linear dependence relationships needed for solving.

However, the actual columns of the row-reduced matrix are not the same vectors as the original matrix columns.

This distinction becomes important when selecting a basis for the original column space.

Pivot Columns and the Original Matrix

Row reduction identifies which column positions contain pivots.

The corresponding columns of the original matrix form a basis for its column space.

We do not use the pivot columns of the reduced matrix as a basis for the original column space.

Example: Identify Pivot Columns

Suppose

\[ A= \begin{bmatrix} 1&2&3\\ 0&1&1\\ 1&3&4 \end{bmatrix} \]

reduces to a matrix with pivots in columns \(1\) and \(2\).

Then columns \(1\) and \(2\) of the original matrix form a basis for \(\operatorname{Col}(A)\).

Consistency and the Augmented Matrix

The system

\[ Ax=b \]

is consistent exactly when the augmented column does not create a new pivot beyond the coefficient matrix.

Equivalently,

\[ \boxed{ \operatorname{rank}(A) = \operatorname{rank}([A\mid b])}. \]

Geometric Interpretation in Two Variables

Each linear equation in two variables describes a line.

Two equations can represent:

Geometric Interpretation in Three Variables

Each linear equation in three variables describes a plane.

Several planes may intersect in:

Elimination determines the algebraic structure behind these geometric possibilities.

Underdetermined Systems

A system with fewer independent equations than unknowns is underdetermined.

Such systems often contain free variables and infinitely many solutions when consistent.

The solution set can have one or more dimensions.

Overdetermined Systems

A system with more equations than unknowns is overdetermined.

It may still have a solution if some equations are redundant or mutually compatible.

Overdetermined systems frequently arise in data fitting and measurement problems.

Square Systems

A system with the same number of equations as unknowns is represented by a square coefficient matrix.

A square system can still have one solution, no solution, or infinitely many solutions.

Squareness alone does not guarantee invertibility.

Elimination and Invertibility

For an \(n\times n\) matrix \(A\), if row reduction produces a pivot in every column, then

\[ \operatorname{rank}(A)=n. \]

In that case,

\[ Ax=b \]

has a unique solution for every \(b\in\mathbb{R}^n\), and \(A\) is invertible.

Elimination Can Compute an Inverse

For an invertible square matrix \(A\), form

\[ [A\mid I]. \]

Row-reduce until the left side becomes \(I\):

\[ [A\mid I] \longrightarrow [I\mid A^{-1}]. \]

Thus Gauss-Jordan elimination can be used to compute an inverse matrix.

Example: A Three-Variable System

Solve

\[ \begin{aligned} x+y+z&=6,\\ 2x-y+z&=3,\\ x+2y-z&=2. \end{aligned} \]

Write the augmented matrix:

\[ \left[ \begin{array}{ccc|c} 1&1&1&6\\ 2&-1&1&3\\ 1&2&-1&2 \end{array} \right]. \]

First Elimination Stage

Apply

\[ R_2\leftarrow R_2-2R_1, \qquad R_3\leftarrow R_3-R_1. \]

We obtain

\[ \left[ \begin{array}{ccc|c} 1&1&1&6\\ 0&-3&-1&-9\\ 0&1&-2&-4 \end{array} \right]. \]

Swap rows \(2\) and \(3\) to use the simpler pivot:

\[ R_2\leftrightarrow R_3. \]

Second Elimination Stage

After the swap,

\[ \left[ \begin{array}{ccc|c} 1&1&1&6\\ 0&1&-2&-4\\ 0&-3&-1&-9 \end{array} \right]. \]

Now apply

\[ R_3\leftarrow R_3+3R_2. \]

This gives

\[ \left[ \begin{array}{ccc|c} 1&1&1&6\\ 0&1&-2&-4\\ 0&0&-7&-21 \end{array} \right]. \]

Back Substitute the Three-Variable System

From the last row,

\[ -7z=-21, \]

so

\[ z=3. \]

From the second row,

\[ y-2z=-4, \]

so

\[ y=2. \]

From the first row,

\[ x+y+z=6, \]

so

\[ x=1. \]

Therefore,

\[ \boxed{(x,y,z)=(1,2,3)}. \]

Numerical Stability and Pivot Choice

In exact symbolic work, any convenient nonzero pivot may be used.

In numerical computation with floating-point arithmetic, pivot selection matters because dividing by very small numbers can magnify rounding error.

Practical algorithms often use partial pivoting, choosing a larger available pivot within the current column.

Partial Pivoting

Partial pivoting swaps the current row with another row whose entry in the pivot column has larger magnitude.

This does not change the exact solution set, but it can improve numerical reliability.

The conceptual elimination process remains the same.

Elimination as Factorization

Gaussian elimination can be organized into matrix factorizations such as

\[ A=LU \]

under suitable conditions, where \(L\) is lower triangular and \(U\) is upper triangular.

This viewpoint allows the same coefficient matrix to be reused efficiently for several different right-hand sides.

Why Triangular Systems Are Easy to Solve

An upper triangular system can be solved from the bottom upward.

Each successive equation introduces one additional unknown.

This is why Gaussian elimination aims to create an upper-triangular or echelon structure.

Systems as Column-Combination Problems

The matrix equation

\[ Ax=b \]

can also be read as

\[ x_1a_1+\cdots+x_na_n=b, \]

where \(a_1,\ldots,a_n\) are the columns of \(A\).

Therefore solving a system asks whether \(b\) lies in the span of the columns and, if so, which coefficients produce it.

Consistency as Span Membership

The system

\[ Ax=b \]

is consistent exactly when

\[ \boxed{b\in\operatorname{Col}(A)}. \]

This unifies elimination with the earlier concepts of linear combinations and span.

Uniqueness as Independence

If

\[ Ax=b \]

has a solution and the columns of \(A\) are linearly independent, then the coefficient vector \(x\) is unique.

If the columns are dependent, multiple coefficient vectors may generate the same output.

Common Error: Performing a Column Operation

Standard elimination uses row operations on an augmented matrix.

Arbitrary column operations change the roles of variables and generally do not preserve the system in the same straightforward way.

Unless a specialized method explicitly requires them, use row operations.

Common Error: Scaling a Row by Zero

Multiplying a row by zero destroys information.

Row scaling is allowed only by a

\[ \boxed{\text{nonzero scalar}}. \]

Otherwise the new system need not be equivalent to the original system.

Common Error: Changing Only Part of a Row

A row operation must be applied to every entry in the row, including the augmented entry.

For example, if

\[ R_2\leftarrow R_2-3R_1, \]

every coefficient and the right-hand side entry must be updated.

Common Error: Ignoring a Contradiction Row

A row such as

\[ \begin{bmatrix} 0&0&0\mid5 \end{bmatrix} \]

is not a harmless zero row.

It represents

\[ 0=5, \]

so the system has no solution.

Common Error: Setting Free Variables Equal to Zero Automatically

A free variable is allowed to vary.

Setting it to zero gives only one particular solution, not the entire solution set.

To describe all solutions, introduce parameters.

Common Error: Confusing Zero Rows with Inconsistency

A row

\[ \begin{bmatrix} 0&0&0\mid0 \end{bmatrix} \]

simply means one equation was redundant.

It does not make the system inconsistent.

Common Error: Using Reduced Columns as Original Column-Space Bases

Row operations change the actual column vectors.

Use RREF to identify pivot positions, then return to the original matrix and select the corresponding original columns.

Common Error: Assuming More Equations Mean More Information

Several equations may be linearly dependent.

For example, one equation may be a scalar multiple or linear combination of the others.

Rank, not raw equation count, measures the number of independent linear constraints.

A Reliable Procedure for Gaussian Elimination

  1. Write the augmented matrix.
  2. Choose the leftmost available nonzero pivot column.
  3. Move a nonzero entry into the pivot position if necessary.
  4. Use row replacement to create zeros below the pivot.
  5. Move to the next row and next pivot column.
  6. Continue until row echelon form is reached.
  7. Check for contradiction rows.
  8. Back substitute to solve for pivot variables.
  9. Parameterize any free variables.

A Reliable Procedure for Gauss-Jordan Elimination

  1. Reduce the augmented matrix to row echelon form.
  2. Scale each pivot row so its pivot is \(1\).
  3. Use row replacement to create zeros above each pivot.
  4. Continue until reduced row echelon form is reached.
  5. Read pivot variables and free variables directly.
  6. Write the complete solution set in scalar or vector form.

A Reliable Procedure for Classifying a System

  1. Row-reduce the augmented matrix.
  2. If a row gives \(0=c\) with \(c\neq0\), there is no solution.
  3. If the system is consistent and every variable column has a pivot, the solution is unique.
  4. If the system is consistent and at least one variable is free, there are infinitely many solutions.

Example: Full Free-Variable Workflow

Suppose

\[ \left[ \begin{array}{cccc|c} 1&0&2&-1&4\\ 0&1&-3&2&1\\ 0&0&0&0&0 \end{array} \right] \]

is the RREF of a system in variables

\[ x_1,x_2,x_3,x_4. \]

The pivot variables are \(x_1,x_2\), while \(x_3,x_4\) are free.

Let

\[ x_3=s,\qquad x_4=t. \]

Then

\[ x_1=4-2s+t, \] \[ x_2=1+3s-2t. \]

Vector Form of the Free-Variable Solution

The solution becomes

\[ \begin{bmatrix} x_1\\ x_2\\ x_3\\ x_4 \end{bmatrix} = \begin{bmatrix} 4\\ 1\\ 0\\ 0 \end{bmatrix} + s \begin{bmatrix} -2\\ 3\\ 1\\ 0 \end{bmatrix} + t \begin{bmatrix} 1\\ -2\\ 0\\ 1 \end{bmatrix}. \]

The two free parameters show that the solution set has two independent directions.

Why Systems and Elimination Matter in Multidimensional Models

Many scientific, engineering, financial, and data problems reduce to coupled linear constraints.

Elimination reveals whether those constraints are compatible, whether the solution is unique, which variables are free, and how many independent directions remain.

The same computational structure supports matrix inversion, rank calculations, null spaces, least-squares methods, eigenvalue algorithms, and numerical simulation.

Practice 1: Augmented Matrix

Write the augmented matrix for

\[ \begin{aligned} x+2y&=7,\\ 3x-y&=5. \end{aligned} \]

Practice 2: Row Operation

Apply

\[ R_2\leftarrow R_2-2R_1 \]

to

\[ \left[ \begin{array}{cc|c} 1&3&4\\ 2&7&9 \end{array} \right]. \]

Practice 3: Identify a Contradiction

What does the row

\[ \begin{bmatrix} 0&0&0\mid2 \end{bmatrix} \]

tell you about the system?

Practice 4: Free Variable

In

\[ \left[ \begin{array}{ccc|c} 1&0&4&2\\ 0&1&-3&5 \end{array} \right], \]

which variable is free?

Practice 5: Parameterize a Solution

Solve

\[ \left[ \begin{array}{ccc|c} 1&0&2&3\\ 0&1&-1&4 \end{array} \right] \]

in parametric vector form.

Practice 6: Homogeneous System

Why is

\[ Ax=0 \]

always consistent?

Practice 7: Rank from Pivots

A row-echelon form contains three pivots. What is the rank of the matrix?

Practice 8: Nullity

A matrix has \(6\) columns and rank \(4\). What is its nullity?

Practice 9: Solution Classification

A consistent system has four variables and only three pivot columns. How many solutions does it have?

Practice 10: Column-Space Interpretation

What does consistency of

\[ Ax=b \]

say about \(b\) and the columns of \(A\)?

Practice 11: Unique Solution Condition

An \(n\times n\) matrix has a pivot in every column. What can you conclude about

\[ Ax=b \]

for arbitrary \(b\in\mathbb{R}^n\)?

Practice 12: Identify the Error

A student obtains a free variable \(x_3\), sets \(x_3=0\), finds one solution, and reports that as the complete answer.

Explain the error.

Practice Answers

Practice 1

\[ \boxed{ \left[ \begin{array}{cc|c} 1&2&7\\ 3&-1&5 \end{array} \right]}. \]

Practice 2

Compute

\[ R_2-2R_1: \] \[ \begin{bmatrix} 2&7&9 \end{bmatrix} - 2 \begin{bmatrix} 1&3&4 \end{bmatrix} = \begin{bmatrix} 0&1&1 \end{bmatrix}. \]

Therefore,

\[ \boxed{ \left[ \begin{array}{cc|c} 1&3&4\\ 0&1&1 \end{array} \right]}. \]

Practice 3

The row represents

\[ 0=2, \]

which is impossible. Therefore the system is inconsistent and has no solution.

Practice 4

The first and second columns contain pivots, so

\[ \boxed{x_3\text{ is free}}. \]

Practice 5

Let

\[ x_3=t. \]

Then

\[ x_1=3-2t, \qquad x_2=4+t. \]

Thus

\[ \boxed{ \begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix} = \begin{bmatrix} 3\\ 4\\ 0 \end{bmatrix} + t \begin{bmatrix} -2\\ 1\\ 1 \end{bmatrix}}. \]

Practice 6

The zero vector always satisfies

\[ A0=0. \]

Therefore every homogeneous system has at least the trivial solution.

Practice 7

\[ \boxed{\operatorname{rank}(A)=3}. \]

Practice 8

By rank-nullity,

\[ 6=4+\operatorname{nullity}(A). \]

Therefore,

\[ \boxed{\operatorname{nullity}(A)=2}. \]

Practice 9

One variable is free. Since the system is consistent, it has

\[ \boxed{\text{infinitely many solutions}}. \]

Practice 10

Consistency means that \(b\) can be written as a linear combination of the columns of \(A\):

\[ \boxed{b\in\operatorname{Col}(A)}. \]

Practice 11

A pivot in every column of a square \(n\times n\) matrix implies full rank and invertibility.

Therefore

\[ \boxed{Ax=b\text{ has exactly one solution for every }b}. \]

Practice 12

Setting \(x_3=0\) produces only one particular solution. Because \(x_3\) is free, it must be assigned a parameter so that all possible solutions are represented.

What to Remember

A linear system can be written as

\[ \boxed{Ax=b}. \]

The core computational strategy is:

\[ \boxed{\text{simplify the system while preserving its solution set}}. \]

Where We Go Next

Elimination tells us whether a square system has full rank and whether its equations produce a unique solution. The next lesson introduces a scalar quantity that summarizes key invertibility and geometric information about square matrices.

The next lesson, Determinants, develops determinant computation, geometric scaling, orientation, invertibility tests, and the structural role determinants play in linear algebra.