The Central Idea
Suppose a finite Markov chain has transition matrix \(P\), and probability distributions are written as row vectors.
A probability vector
\[ \boldsymbol{\pi} = \begin{bmatrix} \pi_1 & \pi_2 & \cdots & \pi_m \end{bmatrix} \]is stationary when
\[ \boxed{\boldsymbol{\pi}P=\boldsymbol{\pi}}. \]The transition changes individual sample paths, but it leaves the overall state probabilities unchanged.
What Stationary Means
If
\[ \boldsymbol{\pi}_0=\boldsymbol{\pi} \]and \(\boldsymbol{\pi}\) is stationary, then
\[ \boldsymbol{\pi}_1 = \boldsymbol{\pi}_0P = \boldsymbol{\pi}. \]Applying the transition again gives
\[ \boldsymbol{\pi}_2 = \boldsymbol{\pi}P = \boldsymbol{\pi}. \]Therefore,
\[ \boxed{\boldsymbol{\pi}_n=\boldsymbol{\pi}\quad\text{for all }n\ge0}. \]A Stationary Distribution Is Still a Probability Distribution
Solving
\[ \boldsymbol{\pi}P=\boldsymbol{\pi} \]is not enough by itself. The vector must also satisfy
\[ \pi_i\ge0 \]for every state \(i\), and
\[ \boxed{\sum_{i=1}^{m}\pi_i=1}. \]These conditions ensure that the entries can be interpreted as probabilities.
A Two-State Example
Consider
\[ P= \begin{bmatrix} 0.8 & 0.2\\ 0.3 & 0.7 \end{bmatrix}. \]Let
\[ \boldsymbol{\pi} = \begin{bmatrix} a & b \end{bmatrix}. \]The stationary condition is
\[ \begin{bmatrix} a & b \end{bmatrix} \begin{bmatrix} 0.8 & 0.2\\ 0.3 & 0.7 \end{bmatrix} = \begin{bmatrix} a & b \end{bmatrix}. \]Expanding the Stationary Equation
Matrix multiplication gives
\[ \begin{bmatrix} 0.8a+0.3b & 0.2a+0.7b \end{bmatrix} = \begin{bmatrix} a & b \end{bmatrix}. \]Therefore,
\[ 0.8a+0.3b=a \]and
\[ 0.2a+0.7b=b. \]We also require
\[ a+b=1. \]Solving the Two-State Example
From
\[ 0.8a+0.3b=a, \]we obtain
\[ 0.3b=0.2a. \]Hence,
\[ 3b=2a. \]Combining this with \(a+b=1\) gives
\[ a=0.6,\qquad b=0.4. \]Thus,
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} 0.6 & 0.4 \end{bmatrix} }. \]Checking the Stationary Distribution
Verify directly:
\[ \begin{bmatrix} 0.6 & 0.4 \end{bmatrix} \begin{bmatrix} 0.8 & 0.2\\ 0.3 & 0.7 \end{bmatrix} = \begin{bmatrix} 0.60 & 0.40 \end{bmatrix}. \]Because the vector is reproduced exactly, it is stationary.
Why the Equations Are Dependent
In a transition matrix, every row sums to \(1\). As a result, the equations in
\[ \boldsymbol{\pi}P=\boldsymbol{\pi} \]are not all independent.
For a finite chain, one equation can often be replaced by the normalization condition
\[ \sum_i\pi_i=1. \]This produces a solvable system without losing the probability requirement.
Stationary Equations as Balance Equations
The equation
\[ \pi_j = \sum_i\pi_i p_{ij} \]says that the stationary probability of state \(j\) equals the total probability flowing into \(j\) after one step.
This is why stationary equations are often called balance equations.
Probability Flow into a State
Suppose there are three states \(A,B,C\). At stationarity,
\[ \pi_B = \pi_Ap_{AB} + \pi_Bp_{BB} + \pi_Cp_{CB}. \]Every possible current state contributes some probability to the next-step probability of being in \(B\).
Probability Flow Is Not Physical Motion
The phrase probability flow is a useful interpretation, but it should not be taken too literally.
Individual sample paths move from one state to another. The entries \(\pi_i\) describe proportions or probabilities across repeated realizations of the stochastic system.
Stationarity means those aggregate proportions are preserved.
Three-State Example
Consider
\[ P= \begin{bmatrix} 0.5 & 0.3 & 0.2\\ 0.2 & 0.6 & 0.2\\ 0.1 & 0.3 & 0.6 \end{bmatrix}. \]Let
\[ \boldsymbol{\pi} = \begin{bmatrix} a & b & c \end{bmatrix}. \]Then
\[ \boldsymbol{\pi}P=\boldsymbol{\pi} \]produces
\[ 0.5a+0.2b+0.1c=a, \] \[ 0.3a+0.6b+0.3c=b, \] \[ 0.2a+0.2b+0.6c=c. \]Solving the Three-State Example
Combine two independent balance equations with
\[ a+b+c=1. \]Solving gives
\[ a=\frac{2}{9},\qquad b=\frac{4}{9},\qquad c=\frac{1}{3}. \]Therefore,
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} \frac{2}{9} & \frac{4}{9} & \frac{1}{3} \end{bmatrix} }. \]Checking the Three-State Distribution
The entries are nonnegative and satisfy
\[ \frac{2}{9}+\frac{4}{9}+\frac{1}{3}=1. \]Multiplying by \(P\) returns the same vector:
\[ \begin{bmatrix} \frac{2}{9} & \frac{4}{9} & \frac{1}{3} \end{bmatrix} P = \begin{bmatrix} \frac{2}{9} & \frac{4}{9} & \frac{1}{3} \end{bmatrix}. \]Thus the vector is stationary.
Stationarity Versus a Known Current State
A stationary distribution is not the same thing as saying that the chain is fixed in one state.
Under a stationary distribution, individual realizations may continue to move among states.
What remains unchanged is the probability distribution across states.
Example: Motion Under Stationarity
In the two-state chain
\[ P= \begin{bmatrix} 0.8 & 0.2\\ 0.3 & 0.7 \end{bmatrix}, \]the stationary distribution is
\[ \begin{bmatrix} 0.6 & 0.4 \end{bmatrix}. \]Even at stationarity, transitions such as
\[ A\to B \]and
\[ B\to A \]continue to occur. Their aggregate effects balance.
Stationarity Versus Convergence
A stationary distribution and a limiting distribution are related but distinct ideas.
A stationary distribution satisfies
\[ \boldsymbol{\pi}P=\boldsymbol{\pi}. \]Convergence asks whether, for an initial distribution \(\boldsymbol{\mu}\),
\[ \boldsymbol{\mu}P^n \]approaches some distribution as \(n\to\infty\).
A Chain Can Have a Stationary Distribution Without Converging
Consider
\[ P= \begin{bmatrix} 0 & 1\\ 1 & 0 \end{bmatrix}. \]The stationary distribution is
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} \frac12 & \frac12 \end{bmatrix} }. \]But if the chain starts in state \(A\), its distributions alternate:
\[ \begin{bmatrix}1&0\end{bmatrix}, \quad \begin{bmatrix}0&1\end{bmatrix}, \quad \begin{bmatrix}1&0\end{bmatrix}, \quad\ldots \]They do not converge to the stationary distribution.
Periodicity Can Prevent Convergence
The preceding chain alternates deterministically between two states.
This is an example of periodic behavior.
Periodicity can cause probability distributions to oscillate even when a stationary distribution exists.
Irreducibility and Uniqueness
For a finite Markov chain, irreducibility has an important consequence:
A finite irreducible Markov chain has a unique stationary distribution.
Every state receives positive probability in that distribution.
Aperiodicity and Convergence
For a finite Markov chain that is both irreducible and aperiodic,
\[ \boldsymbol{\mu}P^n \to \boldsymbol{\pi} \]for every initial probability distribution \(\boldsymbol{\mu}\), where \(\boldsymbol{\pi}\) is the unique stationary distribution.
This is one of the central long-run results for finite Markov chains.
What Aperiodic Means Informally
Aperiodicity means that the chain is not forced to revisit states only at multiples of some fixed cycle length greater than \(1\).
A self-transition with positive probability often helps break strict periodic cycling, although the full definition is based on possible return times.
Finite Chains Always Have at Least One Stationary Distribution
Every finite Markov chain has at least one stationary distribution.
What can vary is whether that distribution is unique and whether arbitrary initial distributions converge to it.
Reducibility and periodicity are two major structural reasons those stronger conclusions may fail.
Reducible Chains Can Have Multiple Stationary Distributions
Consider
\[ P= \begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix}. \]Both states are absorbing.
Every probability vector
\[ \boldsymbol{\pi} = \begin{bmatrix} a & 1-a \end{bmatrix}, \qquad 0\le a\le1, \]satisfies
\[ \boldsymbol{\pi}P=\boldsymbol{\pi}. \]Therefore, this chain has infinitely many stationary distributions.
Absorbing States Produce Stationary Distributions
If state \(j\) is absorbing, then the probability vector concentrated entirely on that state is stationary.
For example, if \(C\) is absorbing in a three-state chain, then
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} 0 & 0 & 1 \end{bmatrix} } \]is stationary.
Closed Classes and Stationary Mass
In a reducible finite chain, stationary probability must ultimately be supported on closed recurrent portions of the state space rather than on states that continually leak probability away.
Different closed classes can support different stationary distributions, which is why reducible chains may fail to have a unique stationary distribution.
Long-Run Proportions
Under suitable conditions, the stationary probability \(\pi_i\) can be interpreted as the long-run proportion of time the chain spends in state \(i\).
For a finite irreducible chain, long-run empirical state frequencies converge to the stationary probabilities, even though periodicity may prevent the one-time distribution \(\boldsymbol{\mu}P^n\) from converging.
Distributional Convergence and Time Averages Are Different
Two distinct long-run questions are:
- Does the distribution of \(X_n\) converge as \(n\to\infty\)?
- Does the fraction of time spent in each state converge?
These questions are related, but they are not identical.
Periodic irreducible chains provide an important example where time averages can stabilize even though the distribution at an exact time continues to oscillate.
Stationary Distributions and Expected Long-Run Quantities
Suppose each state \(i\) has an associated numerical value \(g(i)\).
Under a stationary distribution, the expected value of \(g(X)\) is
\[ \boxed{ E_{\pi}[g(X)] = \sum_i \pi_i g(i) }. \]This provides a way to translate stationary state probabilities into long-run average costs, rewards, loads, failure rates, or other modeled quantities.
Example: Long-Run Average Cost
Suppose a two-state system has stationary distribution
\[ \boldsymbol{\pi} = \begin{bmatrix} 0.6 & 0.4 \end{bmatrix}. \]Let the cost associated with state \(A\) be \(2\) and the cost associated with state \(B\) be \(5\).
The stationary expected cost is
\[ (0.6)(2)+(0.4)(5) = 1.2+2 = \boxed{3.2}. \]Eigenvector Interpretation
The equation
\[ \boldsymbol{\pi}P=\boldsymbol{\pi} \]can be written
\[ \boldsymbol{\pi}P = 1\boldsymbol{\pi}. \]Thus, under the row-vector convention, a stationary distribution is a normalized left eigenvector of \(P\) corresponding to eigenvalue \(1\).
Why the Eigenvalue \(1\) Appears
A stochastic matrix preserves total probability.
This structure guarantees that \(1\) is an eigenvalue of a finite transition matrix.
Stationary distributions identify the probability vectors that are fixed by the transition transformation.
Solving with \(P-I\)
The stationary equation can be rearranged:
\[ \boldsymbol{\pi}P-\boldsymbol{\pi}=0. \]Factoring the row vector gives
\[ \boxed{ \boldsymbol{\pi}(P-I)=0 }. \]This is a homogeneous linear system together with the normalization condition
\[ \sum_i\pi_i=1. \]Row Reduction as a Computational Method
For larger finite chains, the stationary equations can be solved using standard linear-system methods.
A common procedure is:
- form the equations from \(\boldsymbol{\pi}P=\boldsymbol{\pi}\);
- replace one dependent equation with \(\sum_i\pi_i=1\);
- solve the resulting linear system by elimination or row reduction;
- verify nonnegativity and normalization;
- multiply by \(P\) to check the result.
Two-State Shortcut
Consider the general two-state matrix
\[ P= \begin{bmatrix} 1-a & a\\ b & 1-b \end{bmatrix}, \]where \(a,b>0\).
Stationarity requires
\[ \pi_Aa=\pi_Bb. \]Together with
\[ \pi_A+\pi_B=1, \]this gives
\[ \boxed{ \pi_A=\frac{b}{a+b}, \qquad \pi_B=\frac{a}{a+b} }. \]Interpreting the Two-State Shortcut
At stationarity, the expected probability flow from \(A\) to \(B\) is
\[ \pi_Aa, \]while the flow from \(B\) to \(A\) is
\[ \pi_Bb. \]In a two-state chain these must match.
Detailed Balance
A stronger pairwise balance condition is
\[ \boxed{ \pi_i p_{ij} = \pi_j p_{ji} } \]for every pair of states \(i,j\).
When a probability distribution satisfies these detailed balance equations, it is stationary.
Not every stationary distribution must satisfy detailed balance, so detailed balance is a sufficient condition rather than the definition of stationarity.
Reversible Chains
A Markov chain that has a stationary distribution satisfying detailed balance is called reversible with respect to that distribution.
Reversibility means that, under stationarity, the probability flow between every pair of states balances in both directions.
This idea becomes especially useful in more advanced stochastic modeling.
Stationary Does Not Mean Uniform
A stationary distribution does not usually assign equal probability to every state.
For the chain
\[ P= \begin{bmatrix} 0.8 & 0.2\\ 0.3 & 0.7 \end{bmatrix}, \]the stationary distribution is
\[ \begin{bmatrix} 0.6 & 0.4 \end{bmatrix}, \]not
\[ \begin{bmatrix} 0.5 & 0.5 \end{bmatrix}. \]When the Uniform Distribution Is Stationary
For an \(m\)-state chain, the uniform distribution is
\[ \boldsymbol{u} = \begin{bmatrix} \frac1m & \frac1m & \cdots & \frac1m \end{bmatrix}. \]If \(P\) is doubly stochastic, meaning both every row and every column sum to \(1\), then the uniform distribution is stationary.
Example: A Doubly Stochastic Matrix
Consider
\[ P= \begin{bmatrix} 0.7 & 0.3\\ 0.3 & 0.7 \end{bmatrix}. \]Both rows and both columns sum to \(1\).
Therefore,
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} 0.5 & 0.5 \end{bmatrix} } \]is stationary.
Stationarity and Simulation
A Markov chain can be simulated by repeatedly sampling the next state from the transition probabilities associated with the current state.
For an irreducible finite chain, a long simulated path can be used to estimate stationary probabilities by recording the fraction of time spent in each state.
Empirical State Frequencies
Suppose a simulated chain produces states
\[ X_0,X_1,\ldots,X_N. \]The empirical frequency of state \(i\) is
\[ \widehat{\pi}_i = \frac{\#\{n: X_n=i\}}{N+1}. \]Under appropriate conditions and for a sufficiently long run, these frequencies approximate the stationary probabilities.
Initial Transients
A simulation may begin far from stationarity.
Early observations can therefore reflect the chosen initial state more strongly than the long-run behavior of the chain.
The period during which the initial condition still has substantial influence is often called a transient or burn-in period in simulation contexts.
Stationary Models in Applications
Stationary distributions can represent long-run proportions in systems such as:
- customer status models;
- machine reliability states;
- inventory levels;
- queue-length models;
- credit-rating transitions;
- population-state models;
- random walks on networks;
- communication and routing systems.
Example: Customer Status
Suppose a customer is either Active or Inactive and
\[ P= \begin{bmatrix} 0.9 & 0.1\\ 0.4 & 0.6 \end{bmatrix}. \]The stationary equations give
\[ 0.1\pi_A=0.4\pi_I. \]Thus,
\[ \pi_A=4\pi_I. \]With \(\pi_A+\pi_I=1\),
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} 0.8 & 0.2 \end{bmatrix} }. \]Interpreting the Customer Example
Under the assumptions of the model, the stationary distribution indicates a long-run balance of approximately
\[ 80\% \]Active and
\[ 20\% \]Inactive.
This does not mean that every customer spends exactly \(80\%\) of every finite observation window in the Active state. It is a probabilistic long-run statement under the model.
Example: Absorbing Failure
Suppose
\[ P= \begin{bmatrix} 0.95 & 0.05\\ 0 & 1 \end{bmatrix}, \]where the second state represents permanent failure.
The stationary distribution is
\[ \boxed{ \begin{bmatrix} 0 & 1 \end{bmatrix} }. \]Any stationary mass assigned to the working state would leak toward failure and would not be replenished.
Stationary Distribution Versus Quasi-Stationary Ideas
In systems with absorbing states, the ordinary stationary distribution may concentrate entirely on absorption.
More advanced models sometimes study the conditional distribution among nonabsorbing states given that absorption has not yet occurred. Such quasi-stationary ideas are useful but are beyond the scope of this introductory lesson.
Common Error: Solving Only \(\boldsymbol{\pi}P=\boldsymbol{\pi}\)
The stationary equation usually leaves a scaling freedom because it is homogeneous after rearrangement.
The normalization condition
\[ \sum_i\pi_i=1 \]is essential for obtaining a probability distribution.
Common Error: Accepting Negative Entries
A solution to the linear equations is not a probability distribution if any entry is negative.
Always check
\[ \pi_i\ge0. \]Stationary distributions must satisfy both the algebraic and probability requirements.
Common Error: Assuming Stationary Means Absorbing
A stationary distribution can remain unchanged even while the chain continually moves among states.
An absorbing state is one special mechanism that produces a stationary distribution, but stationarity is much broader.
Common Error: Assuming Stationary Means Unique
Some finite chains have one stationary distribution, while others have many.
Irreducibility guarantees uniqueness for a finite chain. Reducible chains can have multiple stationary distributions.
Common Error: Assuming Stationary Means Limiting
A stationary distribution may exist even when
\[ \boldsymbol{\mu}P^n \]does not converge from a particular initial distribution.
Periodic chains demonstrate this distinction clearly.
Common Error: Assuming the Limit Depends on the Starting State
For a finite irreducible aperiodic chain, the limiting distribution is the unique stationary distribution and does not depend on the initial distribution.
Outside those conditions, however, the starting state or initial distribution can matter substantially.
Common Error: Mixing Row and Column Conventions
This lesson uses
\[ \boldsymbol{\pi}P=\boldsymbol{\pi} \]because probability vectors are rows and the transition matrix is row-stochastic.
Sources that use column probability vectors typically write the corresponding stationary equation in transposed form.
Both approaches are valid when used consistently.
A Reliable Procedure for Finding a Stationary Distribution
- Record the state order used by the transition matrix.
- Write an unknown probability vector \(\boldsymbol{\pi}\).
- Form the equation \(\boldsymbol{\pi}P=\boldsymbol{\pi}\).
- Expand the matrix equation into scalar balance equations.
- Use enough independent balance equations to determine the unknown ratios.
- Add the normalization condition \(\sum_i\pi_i=1\).
- Solve the resulting linear system.
- Check that every component is nonnegative.
- Verify that the components sum to \(1\).
- Multiply the result by \(P\) to confirm that it is unchanged.
A Reliable Procedure for Interpreting Long-Run Behavior
- Find the stationary distribution or stationary distributions.
- Check whether the chain is irreducible.
- Check for periodic structure when distributional convergence matters.
- Do not assume stationarity automatically implies convergence.
- Distinguish exact-time distributions from long-run time averages.
- Translate stationary probabilities into expected long-run quantities only when the model assumptions support that interpretation.
Example: Full Workflow
Let
\[ P= \begin{bmatrix} 0.75 & 0.25\\ 0.50 & 0.50 \end{bmatrix}. \]Write
\[ \boldsymbol{\pi} = \begin{bmatrix} a & b \end{bmatrix}. \]The first stationary equation is
\[ 0.75a+0.50b=a. \]Therefore,
\[ 0.50b=0.25a, \]so
\[ a=2b. \]With
\[ a+b=1, \]we obtain
\[ a=\frac23,\qquad b=\frac13. \]Hence,
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} \frac23 & \frac13 \end{bmatrix} }. \]The chain is finite and irreducible. It also has positive self-transition probabilities, so it is aperiodic. Therefore, distributions converge to this stationary distribution from any initial distribution.
Practice 1: Verify Stationarity
Let
\[ P= \begin{bmatrix} 0.8 & 0.2\\ 0.3 & 0.7 \end{bmatrix}. \]Verify that
\[ \boldsymbol{\pi} = \begin{bmatrix} 0.6 & 0.4 \end{bmatrix} \]is stationary.
Practice 2: Find a Two-State Stationary Distribution
Find the stationary distribution for
\[ P= \begin{bmatrix} 0.9 & 0.1\\ 0.2 & 0.8 \end{bmatrix}. \]Practice 3: Use Normalization
Suppose the stationary equations imply
\[ a=3b. \]Find \(a\) and \(b\).
Practice 4: Identify an Absorbing Stationary Distribution
Consider
\[ P= \begin{bmatrix} 0.7 & 0.3\\ 0 & 1 \end{bmatrix}. \]Give one stationary distribution.
Practice 5: Multiple Stationary Distributions
Consider
\[ P= \begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix}. \]Describe all stationary distributions.
Practice 6: Stationary or Limiting?
For
\[ P= \begin{bmatrix} 0 & 1\\ 1 & 0 \end{bmatrix}, \]does the stationary distribution
\[ \begin{bmatrix} 1/2 & 1/2 \end{bmatrix} \]imply that a chain starting from state \(A\) converges to that distribution?
Practice 7: Uniform Stationarity
Is the uniform distribution stationary for
\[ P= \begin{bmatrix} 0.6 & 0.4\\ 0.4 & 0.6 \end{bmatrix}? \]Practice 8: Long-Run Expected Value
A system has stationary distribution
\[ \boldsymbol{\pi} = \begin{bmatrix} 0.7 & 0.3 \end{bmatrix}. \]The state values are \(4\) and \(10\). Find the stationary expected value.
Practice 9: Interpret Irreducibility
What does finite-state irreducibility guarantee about the stationary distribution?
Practice 10: Identify the Error
A student solves
\[ \boldsymbol{\pi}P=\boldsymbol{\pi} \]and obtains
\[ \boldsymbol{\pi} = \begin{bmatrix} 6 & 4 \end{bmatrix}. \]The student declares this the stationary distribution. Explain what remains to be done.
Practice Answers
Practice 1
\[ \begin{bmatrix} 0.6 & 0.4 \end{bmatrix} \begin{bmatrix} 0.8 & 0.2\\ 0.3 & 0.7 \end{bmatrix} = \begin{bmatrix} 0.6 & 0.4 \end{bmatrix}. \]Therefore, the vector is stationary.
Practice 2
Let
\[ \boldsymbol{\pi} = \begin{bmatrix} a & b \end{bmatrix}. \]Balance gives
\[ 0.1a=0.2b, \]so
\[ a=2b. \]With \(a+b=1\),
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} \frac23 & \frac13 \end{bmatrix} }. \]Practice 3
Since
\[ a=3b \]and
\[ a+b=1, \]we have
\[ 4b=1. \]Therefore,
\[ \boxed{ a=\frac34,\qquad b=\frac14 }. \]Practice 4
The second state is absorbing, so
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} 0 & 1 \end{bmatrix} } \]is stationary.
Practice 5
Every probability vector is unchanged by the identity matrix. Thus all vectors of the form
\[ \boxed{ \begin{bmatrix} a & 1-a \end{bmatrix}, \qquad 0\le a\le1 } \]are stationary.
Practice 6
No. Starting from \(A\), the distribution alternates between
\[ \begin{bmatrix} 1 & 0 \end{bmatrix} \]and
\[ \begin{bmatrix} 0 & 1 \end{bmatrix}. \]The chain has a stationary distribution but the exact-time distributions do not converge because the chain is periodic.
Practice 7
Yes. The matrix is doubly stochastic, so
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} 0.5 & 0.5 \end{bmatrix} } \]is stationary.
Practice 8
\[ (0.7)(4)+(0.3)(10) = 2.8+3 = \boxed{5.8}. \]Practice 9
A finite irreducible Markov chain has a unique stationary distribution, and every state receives positive stationary probability.
Practice 10
The vector must be normalized so that its entries sum to \(1\).
Since
\[ 6+4=10, \]divide by \(10\):
\[ \boxed{ \boldsymbol{\pi} = \begin{bmatrix} 0.6 & 0.4 \end{bmatrix} }. \]The result should then be checked by verifying \(\boldsymbol{\pi}P=\boldsymbol{\pi}\).
What to Remember
A stationary distribution is a probability vector left unchanged by the transition matrix:
\[ \boxed{\boldsymbol{\pi}P=\boldsymbol{\pi}}. \]- The entries of \(\boldsymbol{\pi}\) must be nonnegative.
- The entries must sum to \(1\).
- Stationary equations can be interpreted as probability-balance equations.
- A stationary distribution does not mean that individual sample paths stop moving.
- Every finite Markov chain has at least one stationary distribution.
- A finite irreducible chain has a unique stationary distribution.
- A finite irreducible aperiodic chain converges to its stationary distribution from any initial distribution.
- Periodic chains can have a stationary distribution without exact-time convergence.
- Reducible chains can have multiple stationary distributions.
- An absorbing state supports a stationary distribution concentrated entirely on that state.
- Under suitable conditions, stationary probabilities describe long-run proportions of time spent in the states.
- Stationary expected values are weighted averages using the stationary probabilities.
- In linear-algebra language, a stationary distribution is a normalized left eigenvector associated with eigenvalue \(1\).
The essential distinction is: stationarity describes a distribution that is already in balance; convergence describes whether other starting distributions approach that balance.
Where We Go Next
Markov chains describe probabilistic motion among states, and stationary distributions describe balanced long-run state probabilities. One of the most important concrete families of Markov chains is built by repeatedly taking random steps through a numerical or spatial state space.
The next lesson, Random Walks, develops this step-by-step model and connects state transitions to position, displacement, boundaries, hitting behavior, and diffusion-like randomness.