The Central Idea
Suppose an event \(A\) has probability
\[ P(A)=p. \]After \(n\) repeated trials, let \(X_n\) be the number of times \(A\) occurs.
The observed relative frequency is
\[ \frac{X_n}{n}. \]In many standard repeated-trial models,
\[ \boxed{ \frac{X_n}{n} \text{ tends to become close to } p \text{ as }n\text{ becomes large} }. \]Long-Run Frequency Interpretation
One way to interpret a probability such as
\[ P(A)=0.30 \]is that, under repeated comparable trials, the proportion of trials in which \(A\) occurs is expected to stabilize near
\[ 0.30. \]This does not mean exactly \(30\%\) of every finite batch must produce \(A\).
Relative Frequency
If an event occurs \(x\) times in \(n\) trials, then its relative frequency is
\[ \boxed{ \frac{x}{n} }. \]Long-run frequency reasoning studies how this proportion behaves as more trials are accumulated.
Example: Fair Coin Tosses
For a fair coin,
\[ P(H)=\frac12. \]After a small number of tosses, the observed head proportion might be
\[ \frac7{10}=0.70. \]After many more tosses, the running proportion might be
\[ \frac{5032}{10000}=0.5032. \]The later proportion is much closer to the theoretical value
\[ 0.5. \]Small Samples Can Be Highly Variable
In only \(4\) fair coin tosses, the head proportion might be
\[ 0,\quad\frac14,\quad\frac12,\quad\frac34,\quad1. \]These possibilities are widely separated.
A small sample therefore provides a coarse and unstable estimate of the underlying probability.
Larger Samples Usually Produce More Stable Proportions
As the number of trials increases, the set of plausible relative-frequency values becomes more concentrated around the model probability in many standard random systems.
This does not mean randomness disappears.
It means the proportion tends to fluctuate less dramatically.
The Law of Large Numbers
The law of large numbers formalizes the long-run stabilization of averages and relative frequencies.
In a standard repeated independent-trial setting with event probability \(p\),
\[ \boxed{ \frac{X_n}{n} \to p } \]in the appropriate probabilistic sense as
\[ n\to\infty. \]At an introductory level, the key message is that sufficiently long-run relative frequencies tend to remain close to the model probability.
The Law Does Not Predict the Next Trial
The law of large numbers concerns aggregate behavior across many trials.
It does not say which outcome will occur next.
If a fair coin has produced ten tails in a row, the next toss is still modeled with
\[ P(H)=\frac12 \]under independence.
Long-Run Stabilization Is Not a Balancing Force
A fair random process does not actively compensate for previous outcomes.
Long-run frequencies stabilize because many new random outcomes are added, not because the process remembers earlier imbalances and corrects them.
Example: Ten Tails in a Row
Suppose a fair coin has produced
\[ TTTTTTTTTT. \]The next toss still satisfies
\[ P(H)=\frac12. \]The previous ten tails do not increase the probability of heads on the next independent toss.
The Gambler's Fallacy
The gambler's fallacy is the mistaken belief that a random process must soon compensate for a recent imbalance.
For example:
“There have been many tails, so heads is now due.”
In independent fair tosses, this reasoning is incorrect.
Running Relative Frequency
A running relative frequency after trial \(n\) is
\[ \boxed{ \widehat p_n=\frac{X_n}{n} }. \]Recording
\[ \widehat p_1,\widehat p_2,\widehat p_3,\ldots \]shows how the empirical proportion changes as data accumulate.
Example: Running Head Proportion
Suppose a sequence of fair coin tosses begins
\[ H,T,H,H,T,T,H,T,\ldots \]The running head proportions after the first few tosses are
\[ 1,\quad \frac12,\quad \frac23,\quad \frac34,\quad \frac35,\quad \frac12,\quad \frac47,\quad \frac12,\ldots \]The proportion moves as each new observation changes both the numerator and denominator.
Running Proportions Need Not Move Monotonically
A running proportion can move closer to the theoretical probability on one trial and farther away on the next.
Long-run convergence is not the same as monotonic convergence.
Fluctuation remains part of the process.
Expected Count
If an event has probability \(p\) on each of \(n\) comparable trials, then the expected number of occurrences is
\[ \boxed{np}. \]This provides a benchmark for the count.
It does not require the actual count to equal \(np\).
Example: Expected Sixes
A fair die is rolled
\[ 600 \]times.
Since
\[ P(6)=\frac16, \]the expected number of sixes is
\[ 600\left(\frac16\right)=\boxed{100}. \]The actual count might be \(94\), \(103\), \(111\), or another nearby value.
Expected Count Is Not a Guaranteed Count
The expected value
\[ np \]describes the center of the repeated-trial model.
It does not impose an exact quota.
Even when
\[ np \]is an integer, the observed count can differ.
Absolute Error
Suppose the theoretical probability is \(p\) and the empirical proportion is \(\widehat p\).
The absolute error is
\[ \boxed{ |\widehat p-p| }. \]This measures the distance between empirical and theoretical probability.
Example: Absolute Error
Suppose
\[ p=0.50 \]and a simulation produces
\[ \widehat p=0.487. \]Then
\[ |\widehat p-p| = |0.487-0.500| = \boxed{0.013}. \]Relative Frequency Can Improve While Count Error Grows
A larger experiment may have a larger difference in raw counts but a smaller difference in proportions.
For example, compare:
\[ 6\text{ heads out of }10 \]with
\[ 50{,}100\text{ heads out of }100{,}000. \]The count errors from the expected values are \(1\) and \(100\), but the proportions are
\[ 0.60 \]and
\[ 0.501. \]The second relative frequency is much closer to \(0.5\).
Why Proportions Stabilize More Than Counts
As more trials are added, random deviations in the count can continue to grow in absolute size.
But the denominator
\[ n \]also grows.
The ratio
\[ \frac{X_n}{n} \]can therefore become increasingly stable even while the raw count difference remains nonzero.
Probability Describes Repeated Structure, Not Exact Scheduling
A probability of
\[ 0.20 \]does not mean an event must occur exactly once every five trials.
The event might occur twice in one group of five and zero times in another.
The probability concerns the long-run proportion under repeated comparable trials.
Example: A \(20\%\) Event
Suppose
\[ P(A)=0.20. \]In \(5\) trials, the event could occur:
\[ 0,\ 1,\ 2,\ 3,\ 4,\text{ or }5 \]times.
The model does not force exactly one occurrence in each block of five.
Long-Run Frequency Requires a Stable Repetition Model
For long-run frequency interpretation to be meaningful, the repeated trials must be comparable.
If the probability changes systematically from trial to trial, a single stable limiting proportion may no longer represent one fixed event probability.
Example: Changing Probability Across Trials
Suppose a machine's defect probability begins at
\[ 0.01 \]but gradually rises as the machine wears down.
Treating all items as identical repeated trials with one fixed
\[ p \]may be inappropriate.
Long-run frequency must be interpreted within the actual model.
Independence Supports Standard Long-Run Models
Repeated independent trials with a common event probability provide the standard setting for introductory long-run frequency results.
Independence prevents earlier outcomes from systematically altering later trial probabilities.
Identically Distributed Trials
In many repeated-trial models, each trial uses the same probability distribution.
For a Bernoulli-type event,
\[ P(A)=p \]on every trial.
Together with independence, this produces the familiar repeated-trial framework.
Long-Run Frequency and Simulation
Simulation provides a practical way to observe stabilization.
Generate many trials, record the running relative frequency, and compare it with the theoretical probability.
The graph often shows large early swings followed by progressively smaller relative fluctuations.
Example: Simulated Fair Coin
Suppose a simulation records the following cumulative results:
| Trials | Heads | Head Proportion |
|---|---|---|
| 10 | 7 | 0.700 |
| 100 | 54 | 0.540 |
| 1,000 | 493 | 0.493 |
| 10,000 | 5,008 | 0.5008 |
The proportions illustrate increasing stability near
\[ 0.5. \]One Simulation Path Is Not a Proof
A single simulation run can illustrate long-run behavior but cannot prove a mathematical theorem.
Another random run will produce different intermediate values.
The law of large numbers is a theoretical statement about probability, not merely an observed pattern from one computer experiment.
Different Runs Produce Different Paths
Two simulations of the same fair coin model might produce:
\[ 0.4972 \]and
\[ 0.5038 \]after \(10{,}000\) tosses.
Both can be fully consistent with
\[ p=0.5. \]Convergence Does Not Mean Eventual Equality
A running relative frequency does not need to become exactly equal to
\[ p \]and remain there.
It can continue to move above and below \(p\) while staying increasingly close in relative terms.
Long Runs Can Still Contain Streaks
A long random sequence can contain:
- several heads in a row;
- several failures in a row;
- clusters;
- temporary imbalances.
Such local patterns do not contradict long-run stabilization.
Randomness Does Not Mean Visual Alternation
People sometimes expect a random sequence to alternate frequently:
\[ H,T,H,T,H,T,\ldots \]But true random sequences naturally contain runs such as
\[ H,H,H,H \]or
\[ T,T,T. \]Randomness includes clustering.
Long-Run Frequency and Rare Events
If
\[ P(A)=0.001, \]then the event is rare but not impossible.
Over a very large number of trials, the event may occur many times while still representing only about one occurrence per thousand trials in proportion.
Example: Rare Defect
Suppose a component has defect probability
\[ 0.002. \]In
\[ 100{,}000 \]comparable items, the expected defect count is
\[ 100000(0.002)=\boxed{200}. \]A rare event can therefore generate a substantial count in a large population.
Probability Zero and Long-Run Frequency
In finite discrete models, an impossible event has probability zero and never occurs.
In continuous models, an individual outcome can have probability zero while remaining possible.
Thus, long-run frequency intuition must be applied carefully in continuous probability.
Probability One and Long-Run Frequency
In finite elementary models, an event with probability one occurs on every trial.
In more advanced infinite models, probability one can coexist with a nonempty probability-zero exceptional set.
Introductory frequency interpretations should therefore remain tied to the model being used.
Empirical Frequency Can Be Used to Estimate Unknown Probability
If the true event probability is not known, a long sequence of observations can provide an estimate:
\[ \boxed{ \widehat p=\frac{x}{n} }. \]Greater sample size generally improves the stability of this estimate under an appropriate stable model.
Example: Estimating a Defect Probability
Suppose \(41\) defects are observed among \(2000\) items.
The empirical estimate is
\[ \widehat p = \frac{41}{2000} = \boxed{0.0205}. \]This estimate can be compared with engineering targets or earlier model assumptions.
Long-Run Frequency Does Not Eliminate Sampling Error
A large sample usually improves precision, but the empirical frequency is still random.
Different large samples can produce slightly different estimates.
Statistical inference later quantifies how much uncertainty remains.
Frequency Stability and Model Checking
If a model predicts
\[ p=0.25, \]but repeated large datasets consistently produce relative frequencies near
\[ 0.60, \]the model may be inappropriate.
Long-run empirical behavior can therefore provide evidence about model quality.
Persistent Disagreement Suggests More Than Random Noise
Small discrepancies between theory and data are expected.
Large persistent discrepancies can suggest:
- an incorrect probability assumption;
- dependence between trials;
- a changing process;
- selection bias;
- measurement error;
- an incorrect simulation or data model.
Long-Run Frequency Is Conditional on the Model
A statement such as
\[ P(A)=0.7 \]only has meaning relative to a specified random process.
Long-run frequency does not rescue an ambiguous or poorly specified experiment.
The trial structure, population, random mechanism, and event definition must remain clear.
Stationarity as a Modeling Idea
A process is informally called stationary when important probabilistic features remain stable over time.
Introductory repeated-trial models often assume at least a stable event probability from one trial to the next.
If probabilities drift, long-run averages can reflect a mixture of changing conditions rather than one fixed probability.
Example: Seasonal Probability
Suppose the probability of rain varies by season.
A single long-run frequency over the entire year may combine very different seasonal probabilities.
A better model may condition on season rather than treating every day as an identical trial.
Independent Trials Are Not the Only Possible Long-Run Models
More advanced probability theory contains long-run results for dependent processes as well.
However, independence provides the cleanest introductory setting.
The key lesson is not that long-run frequency requires independence in every conceivable model, but that assumptions must match the theorem being used.
The Difference Between Probability and Certainty
A probability close to \(1\) means an event is very likely in the model.
It does not mean the event must occur on the next trial.
Likewise, a probability close to \(0\) does not make the event impossible.
Example: \(99\%\) Success
Suppose
\[ P(\text{success})=0.99. \]A single trial can still fail.
Over many comparable trials, the success proportion is expected to be near
\[ 0.99, \]but individual failures remain part of the model.
Long-Run Frequency and Risk
Small probabilities can matter when exposure is repeated many times.
If an adverse event has probability
\[ p \]per trial, then repeated exposure can produce a meaningful expected number of occurrences
\[ np. \]Long-run thinking is therefore central to reliability, insurance, quality control, and risk analysis.
Example: Repeated Low-Risk Exposure
Suppose an event has probability
\[ 0.0005 \]per independent trial.
Across
\[ 1{,}000{,}000 \]trials, the expected count is
\[ 1000000(0.0005)=\boxed{500}. \]A very small individual probability can still correspond to many events at large scale.
Common Error: Expecting Exact Proportions in Small Samples
A probability of
\[ 0.5 \]does not require exactly five successes in every ten trials.
Small-sample variation is normal.
Common Error: Believing the Next Trial Must Restore Balance
Long-run stabilization does not imply short-run compensation.
In independent trials, the next probability remains the same regardless of recent streaks.
Common Error: Confusing Expected Count with Guaranteed Count
If
\[ np=40, \]the observed count need not be exactly \(40\).
Expected value is a model benchmark, not a deterministic prediction.
Common Error: Thinking More Trials Remove All Randomness
A larger sample usually stabilizes proportions, but different large samples can still differ.
Randomness remains present at every finite sample size.
Common Error: Assuming More Trials Fix Bias
If the experiment or simulation is biased, more trials can make the wrong long-run frequency more stable.
Sample size reduces random variation, not systematic error.
Common Error: Ignoring a Changing Process
If the probability changes over time, combining all observations into one relative frequency can hide important structure.
Check whether the repeated-trial assumptions remain reasonable.
Common Error: Treating a Streak as Evidence of Dependence
Independent random sequences naturally produce streaks.
A short run of repeated outcomes does not by itself prove that trials are dependent.
Dependence must be evaluated using broader evidence or model structure.
A Reliable Procedure for Long-Run Frequency Reasoning
- Identify the event and its model probability \(p\).
- Confirm that the repeated trials are comparable.
- Determine whether independence or another stability assumption is appropriate.
- Track the event count \(X_n\).
- Compute the running relative frequency \(X_n/n\).
- Compare empirical proportions with \(p\), not individual trial outcomes.
- Expect short-run fluctuation.
- Do not interpret streaks as automatic evidence of compensation or dependence.
- Use \(np\) as an expected-count benchmark, not a guaranteed count.
- Reconsider the model if large discrepancies persist across substantial data.
Example: Full Long-Run Workflow
A production model predicts a defect probability of
\[ p=0.03. \]Among \(5000\) produced items, \(162\) are defective.
The empirical defect proportion is
\[ \widehat p = \frac{162}{5000} = 0.0324. \]The expected count is
\[ np = 5000(0.03) = 150. \]The observed proportion is close to the model probability:
\[ |\widehat p-p| = |0.0324-0.03| = \boxed{0.0024}. \]This comparison is descriptive; formal model assessment would require statistical inference.
Practice 1: Relative Frequency
An event occurs \(84\) times in \(300\) trials.
Find its relative frequency.
Practice 2: Expected Count
An event has probability
\[ 0.18. \]What is its expected count in \(500\) trials?
Practice 3: Absolute Error
A theoretical probability is
\[ 0.40 \]and an empirical proportion is
\[ 0.427. \]Find the absolute error.
Practice 4: Fair Coin Interpretation
A fair coin produces \(8\) heads in its first \(10\) tosses.
Does this contradict
\[ P(H)=0.5? \]Practice 5: Gambler's Fallacy
A fair coin has landed tails six times in a row.
What is the probability of heads on the next independent toss?
Practice 6: Rare Event Count
An event has probability
\[ 0.004. \]What is its expected count in \(25{,}000\) trials?
Practice 7: Empirical Estimate
A process produces \(73\) successes in \(160\) trials.
Estimate the success probability empirically.
Practice 8: Running Frequency
After \(200\) trials, an event has occurred \(92\) times.
After \(300\) trials, it has occurred \(139\) times.
Find the running relative frequency at both points.
Practice 9: Long-Run Stability
A model has
\[ P(A)=0.25. \]Which empirical result is relatively closer to the model:
\[ 3\text{ occurrences in }10\text{ trials} \]or
\[ 2520\text{ occurrences in }10000\text{ trials}? \]Practice 10: Changing Process
A machine's defect probability rises as the machine heats during the day.
Is it automatically appropriate to treat all items as independent trials with one constant probability \(p\)?
Practice 11: Streak Interpretation
A fair die produces four sixes in a row.
Does that streak by itself prove the rolls are dependent?
Practice 12: Identify the Error
A student says:
“If an event has probability \(0.2\), it must happen exactly once in every five trials.”
Explain the error.
Practice Answers
Practice 1
\[ \frac{84}{300} = \boxed{0.28}. \]Practice 2
\[ np = 500(0.18) = \boxed{90}. \]Practice 3
\[ |0.427-0.400| = \boxed{0.027}. \]Practice 4
No. A finite sample can deviate substantially from the theoretical probability. Eight heads in ten tosses is possible under a fair-coin model.
Practice 5
\[ \boxed{\frac12}. \]Independence means previous tails do not change the next-toss probability.
Practice 6
\[ np = 25000(0.004) = \boxed{100}. \]Practice 7
\[ \widehat p = \frac{73}{160} = \boxed{0.45625}. \]Practice 8
After \(200\) trials:
\[ \frac{92}{200} = \boxed{0.46}. \]After \(300\) trials:
\[ \frac{139}{300} \approx \boxed{0.4633}. \]Practice 9
The first proportion is
\[ \frac3{10}=0.30, \]with absolute error
\[ 0.05. \]The second is
\[ \frac{2520}{10000}=0.252, \]with absolute error
\[ 0.002. \]Therefore, the second result is much closer to the model probability.
Practice 10
No. If the defect probability changes over time, a constant-\(p\) repeated trial model may be inappropriate. The changing conditions should be modeled explicitly.
Practice 11
No. Independent random sequences can contain streaks. A short streak alone does not establish dependence.
Practice 12
Probability \(0.2\) describes a long-run proportion under repeated comparable trials. It does not impose an exact schedule of one occurrence in each block of five trials.
What to Remember
If an event has model probability \(p\), then in many standard repeated-trial settings its running relative frequency
\[ \frac{X_n}{n} \]tends to become increasingly stable near
\[ p. \]The expected number of occurrences in \(n\) trials is
\[ \boxed{np}. \]- Long-run stabilization does not imply exact short-run balance.
- The next independent trial is not influenced by a need to “catch up.”
- Running proportions can fluctuate above and below the theoretical probability.
- Larger samples generally produce more stable relative frequencies.
- Expected count is a benchmark, not a guaranteed count.
- Long-run reasoning depends on the assumptions of the probability model.
- Persistent disagreement between data and theory can signal model problems.
The most important habit is: interpret probability through repeated structure without demanding short-run regularity from random outcomes.
Where We Go Next
Long-run frequency explains how theoretical probability can appear in repeated observations, but that interpretation is only as good as the model generating the trials.
The next lesson, Model Assumptions and Limitations, examines the assumptions behind probability models, how idealizations can fail, and how to recognize when a probability calculation is mathematically correct but poorly matched to the real system.