The Central Idea
Probability begins with a sample space \(S\), events \(A\subseteq S\), and a rule that assigns a number to each event.
That rule is written
\[ P(A). \]The axioms of probability specify the structural requirements that make \(P\) a valid probability measure.
\[ \boxed{ \text{probability model} = \text{sample space} + \text{events} + \text{valid probability assignment} }. \]Probability as a Function
A probability assignment can be viewed as a function that takes an event as input and returns a real number.
\[ A\longmapsto P(A). \]If \(A\) is the event “roll an even number,” then \(P(A)\) represents the probability assigned to that event.
The value \(P(A)\) depends on the probability model, not merely on the name of the event.
The Three Probability Axioms
A probability measure \(P\) satisfies three fundamental axioms.
- Probabilities are nonnegative.
- The entire sample space has probability \(1\).
- Probabilities add across pairwise disjoint events.
These are commonly called the Kolmogorov axioms.
Axiom 1: Nonnegativity
Every event has a probability greater than or equal to zero:
\[ \boxed{P(A)\ge 0}. \]A negative probability is not permitted in an ordinary probability model.
For example,
\[ P(A)=-0.2 \]cannot be a valid probability assignment.
Why Nonnegativity Matters
Probability measures the weight assigned to an event within a model. Negative weight would contradict the usual interpretation of probability as a quantity representing possibility or likelihood.
Nonnegativity also supports later consequences such as
\[ P(A)\le 1. \]That upper bound is derived rather than stated separately as an axiom.
Axiom 2: Normalization
The entire sample space contains every modeled outcome, so its probability is
\[ \boxed{P(S)=1}. \]This axiom normalizes the total probability in the model.
The number \(1\) represents certainty within the stated sample space.
Probability One Means Certain Within the Model
If an event is the entire sample space, then it must occur whenever the experiment produces a modeled outcome.
Therefore,
\[ P(S)=1. \]This statement is always relative to the model. The sample space itself encodes what outcomes are being treated as possible.
Axiom 3: Countable Additivity
Suppose
\[ A_1,A_2,A_3,\ldots \]are pairwise disjoint events, meaning
\[ A_i\cap A_j=\varnothing \qquad\text{whenever }i\ne j. \]Then
\[ \boxed{ P\left(\bigcup_{i=1}^{\infty}A_i\right) = \sum_{i=1}^{\infty}P(A_i) }. \]This is the countable additivity axiom.
What Pairwise Disjoint Means
A collection of events is pairwise disjoint if every distinct pair has empty intersection.
For three events \(A\), \(B\), and \(C\), this requires
\[ A\cap B=\varnothing, \] \[ A\cap C=\varnothing, \]and
\[ B\cap C=\varnothing. \]No outcome belongs to more than one event in the collection.
Finite Additivity Follows Immediately
Countable additivity includes finite collections as a special case.
If \(A\) and \(B\) are disjoint, then
\[ A\cap B=\varnothing \]and
\[ \boxed{P(A\cup B)=P(A)+P(B)}. \]This is finite additivity for two mutually exclusive events.
Example: Disjoint Die Events
Suppose a die model assigns
\[ P(\{1\})=\frac16 \]and
\[ P(\{2\})=\frac16. \]Because
\[ \{1\}\cap\{2\}=\varnothing, \]additivity gives
\[ P(\{1,2\}) = \frac16+\frac16 = \boxed{\frac13}. \]The Axioms Do Not Require Equal Likelihood
The axioms require valid total structure, but they do not require individual outcomes to have equal probability.
For
\[ S=\{A,B,C\}, \]a model such as
\[ P(\{A\})=0.2,\qquad P(\{B\})=0.3,\qquad P(\{C\})=0.5 \]is valid because the probabilities are nonnegative and sum to \(1\).
Example: A Loaded Three-Outcome Model
Let
\[ S=\{a,b,c\}. \]Assign
\[ P(\{a\})=\frac12, \qquad P(\{b\})=\frac13, \qquad P(\{c\})=\frac16. \]These singleton events are disjoint and
\[ \frac12+\frac13+\frac16=1. \]Therefore,
\[ P(S)=1, \]as required.
Deriving the Probability of the Empty Set
The empty set is disjoint from every event, including the sample space.
Since
\[ S\cup\varnothing=S, \]finite additivity gives
\[ P(S) = P(S)+P(\varnothing). \]Because \(P(S)=1\),
\[ 1=1+P(\varnothing), \]so
\[ \boxed{P(\varnothing)=0}. \]The Impossible Event Has Probability Zero
The empty event contains no outcomes, so its probability is
\[ \boxed{0}. \]Thus,
\[ P(\varnothing)=0. \]This is a theorem derived from the axioms rather than an additional axiom.
Complementary Events Partition the Sample Space
For any event \(A\),
\[ A\cap A^c=\varnothing \]and
\[ A\cup A^c=S. \]Therefore, additivity gives
\[ P(A)+P(A^c)=P(S)=1. \]The Complement Rule
Rearranging
\[ P(A)+P(A^c)=1 \]gives
\[ \boxed{P(A^c)=1-P(A)}. \]This important rule is a direct consequence of the probability axioms.
A later lesson will develop complement rules more extensively.
Every Probability Is at Most One
Because
\[ P(A^c)\ge0, \]the complement rule gives
\[ 1-P(A)\ge0. \]Therefore,
\[ P(A)\le1. \]Combining this with nonnegativity,
\[ \boxed{0\le P(A)\le1}. \]Probability Values Lie in the Unit Interval
Every event probability belongs to
\[ \boxed{[0,1]}. \]A value near \(0\) represents an event assigned little probability, while a value near \(1\) represents an event assigned substantial probability.
Values below \(0\) or above \(1\) are invalid in ordinary probability.
Subset Monotonicity
Suppose
\[ A\subseteq B. \]Then \(B\) can be decomposed into the disjoint pieces
\[ A \]and
\[ B\setminus A. \]Thus,
\[ P(B)=P(A)+P(B\setminus A). \]Since probabilities are nonnegative,
\[ \boxed{P(A)\le P(B)}. \]Why Monotonicity Makes Sense
If every outcome that makes \(A\) occur also makes \(B\) occur, then \(B\) cannot have less probability than \(A\).
In set language,
\[ A\subseteq B \]implies
\[ \boxed{P(A)\le P(B)}. \]This is another consequence of the axioms.
Probability of a Difference When One Event Is Contained in Another
If
\[ A\subseteq B, \]then
\[ B=A\cup(B\setminus A), \]where the union is disjoint.
Therefore,
\[ P(B)=P(A)+P(B\setminus A), \]so
\[ \boxed{P(B\setminus A)=P(B)-P(A)}. \]General Difference Formula
For arbitrary events \(A\) and \(B\),
\[ A=(A\cap B)\cup(A\setminus B), \]and the two pieces are disjoint.
Thus,
\[ P(A)=P(A\cap B)+P(A\setminus B), \]giving
\[ \boxed{P(A\setminus B)=P(A)-P(A\cap B)}. \]Why We Cannot Usually Add Overlapping Events Directly
If \(A\) and \(B\) overlap, then
\[ P(A)+P(B) \]counts the probability of the overlap \(A\cap B\) twice.
Therefore, the simple rule
\[ P(A\cup B)=P(A)+P(B) \]applies only when
\[ A\cap B=\varnothing. \]The General Addition Relationship
Decompose \(A\cup B\) into disjoint pieces. The resulting relationship is
\[ \boxed{ P(A\cup B) = P(A)+P(B)-P(A\cap B) }. \]The subtraction removes the overlap that was counted twice.
This relationship will be developed systematically in the later lesson on complement, addition, and multiplication rules.
Example: Overlapping Events
Suppose
\[ P(A)=0.6, \qquad P(B)=0.5, \qquad P(A\cap B)=0.3. \]Then
\[ P(A\cup B) = 0.6+0.5-0.3 = \boxed{0.8}. \]Simply adding \(0.6+0.5\) would incorrectly produce \(1.1\).
Probability of a Finite Partition
Suppose
\[ A_1,A_2,\ldots,A_n \]are pairwise disjoint and together fill the sample space:
\[ A_1\cup A_2\cup\cdots\cup A_n=S. \]Then
\[ \boxed{ P(A_1)+P(A_2)+\cdots+P(A_n)=1 }. \]This is the finite partition form of normalization and additivity.
Singleton Probabilities in a Finite Sample Space
Suppose
\[ S=\{s_1,s_2,\ldots,s_n\}. \]The singleton events
\[ \{s_1\},\{s_2\},\ldots,\{s_n\} \]are pairwise disjoint and their union is \(S\).
Therefore,
\[ \boxed{ \sum_{i=1}^{n}P(\{s_i\})=1 }. \]Building a Finite Probability Model from Outcome Weights
For a finite sample space, one common way to define a probability model is to assign each outcome a nonnegative weight
\[ p_i=P(\{s_i\}) \]such that
\[ p_i\ge0 \]and
\[ \sum_{i=1}^{n}p_i=1. \]Then the probability of any event \(A\) is the sum of the weights of the outcomes contained in \(A\).
Example: A Finite Unequal-Probability Model
Let
\[ S=\{a,b,c,d\} \]with
\[ P(\{a\})=0.1,\quad P(\{b\})=0.2,\quad P(\{c\})=0.3,\quad P(\{d\})=0.4. \]Then for
\[ A=\{a,c,d\}, \]finite additivity gives
\[ P(A)=0.1+0.3+0.4=\boxed{0.8}. \]Checking a Proposed Finite Model
Suppose
\[ S=\{a,b,c\} \]and someone proposes
\[ P(\{a\})=0.4,\qquad P(\{b\})=0.5,\qquad P(\{c\})=0.3. \]The total is
\[ 0.4+0.5+0.3=1.2. \]Because the singleton events partition \(S\), their probabilities must sum to \(1\). Therefore this assignment is invalid.
Another Invalid Model: Negative Probability
Suppose
\[ P(\{a\})=0.7,\qquad P(\{b\})=0.4,\qquad P(\{c\})=-0.1. \]The values sum to \(1\), but the model still fails because
\[ P(\{c\})<0. \]Satisfying normalization alone is not enough. All axioms must hold.
Zero Probability Does Not Always Mean Impossible
In a finite sample space in which every individual outcome has positive probability, an event has probability zero only when it contains no outcomes.
In continuous probability models, however, an individual outcome can have probability zero even though it is possible.
This distinction becomes important when continuous probability distributions are studied.
Example: A Continuous Outcome with Probability Zero
Suppose a point is selected uniformly from the interval
\[ [0,1]. \]In the standard continuous model, the probability of selecting exactly
\[ x=\frac12 \]is
\[ P\left(\left\{\frac12\right\}\right)=0. \]Yet \(\frac12\) is still an outcome in the sample space.
Probability One Does Not Always Mean Logical Necessity
In some infinite or continuous models, an event can have probability \(1\) while its complement is nonempty.
Probability \(1\) means the complement has probability \(0\):
\[ P(A)=1 \quad\Longrightarrow\quad P(A^c)=0. \]Whether a probability-zero complement is literally impossible depends on the structure of the model.
Countably Infinite Sample Spaces
The axioms also apply when the sample space contains countably many outcomes.
Suppose
\[ S=\{1,2,3,\ldots\} \]and assign
\[ P(\{n\})=\frac{1}{2^n}. \]Then
\[ \sum_{n=1}^{\infty}\frac{1}{2^n}=1, \]so the singleton probabilities can define a valid probability model.
Example: Probability of an Infinite Event
Continue with
\[ P(\{n\})=\frac{1}{2^n}. \]Let \(A\) be the event that \(n\ge3\):
\[ A=\{3,4,5,\ldots\}. \]Then countable additivity gives
\[ P(A) = \sum_{n=3}^{\infty}\frac{1}{2^n} = \boxed{\frac14}. \]Why Countable Additivity Is Stronger Than Finite Additivity
Finite additivity tells us how probabilities behave when finitely many disjoint events are combined.
Countable additivity extends this structure to sequences of disjoint events:
\[ A_1,A_2,A_3,\ldots. \]This additional requirement is essential for the standard mathematical theory of probability on infinite sample spaces.
Continuity from Below
Suppose
\[ A_1\subseteq A_2\subseteq A_3\subseteq\cdots \]is an increasing sequence of events.
Then standard probability measures satisfy
\[ \boxed{ P\left(\bigcup_{n=1}^{\infty}A_n\right) = \lim_{n\to\infty}P(A_n) }. \]This useful property follows from countable additivity.
Example: Increasing Events
Let
\[ S=\{1,2,3,\ldots\} \]and define
\[ A_n=\{1,2,\ldots,n\}. \]Then
\[ A_1\subseteq A_2\subseteq\cdots \]and
\[ \bigcup_{n=1}^{\infty}A_n=S. \]Therefore,
\[ \lim_{n\to\infty}P(A_n)=P(S)=1. \]Continuity from Above
Suppose
\[ A_1\supseteq A_2\supseteq A_3\supseteq\cdots \]is a decreasing sequence of events.
Then
\[ \boxed{ P\left(\bigcap_{n=1}^{\infty}A_n\right) = \lim_{n\to\infty}P(A_n) }. \]This is another consequence of the probability axioms.
The Union Bound
For arbitrary events \(A\) and \(B\),
\[ P(A\cup B) = P(A)+P(B)-P(A\cap B). \]Because
\[ P(A\cap B)\ge0, \]it follows that
\[ \boxed{P(A\cup B)\le P(A)+P(B)}. \]This inequality is often called a union bound.
Countable Union Bound
More generally, for events
\[ A_1,A_2,\ldots, \]not necessarily disjoint,
\[ \boxed{ P\left(\bigcup_{i=1}^{\infty}A_i\right) \le \sum_{i=1}^{\infty}P(A_i) }. \]When the events overlap, the sum on the right may count some outcomes more than once, producing an upper bound rather than an equality.
Probability Models Must Be Internally Consistent
Once probabilities are assigned, every derived event probability must remain compatible with the axioms.
For example, if
\[ A\subseteq B, \]then any proposed model with
\[ P(A)>P(B) \]is inconsistent with monotonicity and therefore cannot be valid.
Example: Detecting an Inconsistency
Suppose
\[ A\subseteq B, \]but someone proposes
\[ P(A)=0.7 \]and
\[ P(B)=0.5. \]Because subsets cannot have larger probability than the events containing them,
\[ 0.7\le0.5 \]would be required, which is false.
The proposed assignments are therefore invalid.
Example: Recovering a Missing Probability
Suppose
\[ S=\{a,b,c,d\} \]and
\[ P(\{a\})=0.15, \qquad P(\{b\})=0.25, \qquad P(\{c\})=0.40. \]Because the singleton outcomes partition \(S\),
\[ 0.15+0.25+0.40+P(\{d\})=1. \]Therefore,
\[ \boxed{P(\{d\})=0.20}. \]Example: Recovering a Complement Probability
Suppose
\[ P(A)=0.37. \]Then
\[ P(A^c)=1-P(A) \]gives
\[ \boxed{P(A^c)=0.63}. \]The two probabilities sum to \(1\) because \(A\) and \(A^c\) partition the sample space.
Example: A Partition with Three Events
Suppose \(A\), \(B\), and \(C\) are pairwise disjoint and
\[ A\cup B\cup C=S. \]If
\[ P(A)=0.2 \]and
\[ P(B)=0.35, \]then
\[ P(C)=1-0.2-0.35=\boxed{0.45}. \]Probability Is Not Determined by Event Size Alone
Two events containing the same number of outcomes can have different probabilities if the underlying outcomes are not equally likely.
Therefore,
\[ |A|=|B| \]does not imply
\[ P(A)=P(B) \]without additional assumptions.
Equal Outcome Counts Require an Equal-Likelihood Assumption
The familiar formula
\[ \frac{\text{favorable outcomes}}{\text{possible outcomes}} \]is not one of the axioms.
It applies when the relevant elementary outcomes are equally likely.
The next lesson develops this condition carefully.
Axioms Versus Modeling Assumptions
The axioms state what every valid probability measure must satisfy.
Modeling assumptions determine which particular probability measure is appropriate for a real or hypothetical experiment.
For example, the axioms do not tell us whether a coin should have
\[ P(H)=\frac12 \]or
\[ P(H)=0.7. \]That depends on the model of the coin and the experiment.
The Axioms Constrain Models Rather Than Create Them
A valid probability model must obey the axioms, but several different models can obey them on the same sample space.
For
\[ S=\{H,T\}, \]all of the following assignments satisfy the axioms:
\[ P(H)=0.5,\qquad P(T)=0.5, \] \[ P(H)=0.7,\qquad P(T)=0.3, \]and
\[ P(H)=0.1,\qquad P(T)=0.9. \]Evidence or assumptions are needed to choose among them.
Common Error: Treating \(P(S)=1\) as the Only Requirement
A set of assigned values can sum to \(1\) and still be invalid.
For example,
\[ 0.8+0.4-0.2=1, \]but a probability assignment containing \(-0.2\) violates nonnegativity.
All probability axioms must hold simultaneously.
Common Error: Allowing Probability Greater Than One
A proposed value such as
\[ P(A)=1.12 \]cannot be valid.
From the axioms,
\[ \boxed{0\le P(A)\le1}. \]Common Error: Adding Overlapping Events as Though They Were Disjoint
The equality
\[ P(A\cup B)=P(A)+P(B) \]requires
\[ A\cap B=\varnothing. \]If the events overlap, the overlap must be accounted for separately.
Common Error: Assuming Probability Zero Means Impossible
The implication
\[ A=\varnothing \quad\Longrightarrow\quad P(A)=0 \]is always true.
The reverse implication
\[ P(A)=0 \quad\Longrightarrow\quad A=\varnothing \]is not true in every probability model, especially in continuous settings.
Common Error: Assuming Probability One Means the Event Equals the Sample Space
Certainly,
\[ A=S \quad\Longrightarrow\quad P(A)=1. \]But in some infinite models,
\[ P(A)=1 \]can occur even when
\[ A\ne S. \]The complement may be nonempty but have probability zero.
Common Error: Assuming the Axioms Guarantee Fairness
A fair coin model and a biased coin model can both satisfy the probability axioms.
The axioms guarantee mathematical consistency, not physical fairness.
Fairness and symmetry are additional modeling assumptions.
A Reliable Procedure for Checking a Finite Probability Model
- Identify the sample space.
- Check that every assigned probability is nonnegative.
- Check that the probabilities of disjoint elementary outcomes sum to \(1\).
- Use additivity to construct probabilities of compound events.
- Verify that every event probability lies between \(0\) and \(1\).
- Check subset relationships for monotonicity.
- Do not assume equal likelihood unless it is justified separately.
A Reliable Procedure for Using the Axioms
- Translate the probability question into events.
- Identify whether relevant events are disjoint.
- Use \(P(S)=1\) when the events form a partition.
- Use \(P(A^c)=1-P(A)\) when a complement is easier to evaluate.
- Use additivity directly only for disjoint events.
- Use monotonicity to check whether proposed values are plausible.
- Confirm that all resulting probabilities remain in \([0,1]\).
Example: Full Axiom Check
Let
\[ S=\{a,b,c,d\} \]with proposed singleton probabilities
\[ 0.10,\qquad0.25,\qquad0.30,\qquad0.35. \]First, all values are nonnegative.
Second,
\[ 0.10+0.25+0.30+0.35=1. \]Because the singleton events are disjoint and partition \(S\), these assignments define a valid finite probability model.
Example: Computing a Compound Event
Continue with
\[ P(\{a\})=0.10,\quad P(\{b\})=0.25,\quad P(\{c\})=0.30,\quad P(\{d\})=0.35. \]Let
\[ A=\{a,c,d\}. \]Because singleton outcomes are disjoint,
\[ P(A)=0.10+0.30+0.35=\boxed{0.75}. \]Practice 1: Probability Range
Which of the following could be a probability?
\[ -0.2,\qquad0,\qquad0.47,\qquad1,\qquad1.08. \]Practice 2: Empty Event
What is
\[ P(\varnothing)? \]Practice 3: Complement
If
\[ P(A)=0.64, \]find \(P(A^c)\).
Practice 4: Finite Additivity
Suppose \(A\cap B=\varnothing\),
\[ P(A)=0.28 \]and
\[ P(B)=0.41. \]Find \(P(A\cup B)\).
Practice 5: Partition
Events \(A\), \(B\), and \(C\) are pairwise disjoint and partition the sample space. If
\[ P(A)=0.15 \]and
\[ P(B)=0.50, \]find \(P(C)\).
Practice 6: Check a Proposed Model
A sample space has three outcomes with assigned probabilities
\[ 0.4,\qquad0.35,\qquad0.25. \]Is this assignment valid?
Practice 7: Check Another Proposed Model
A sample space has three outcomes with assigned probabilities
\[ 0.6,\qquad0.5,\qquad-0.1. \]Is this assignment valid?
Practice 8: Monotonicity
Suppose
\[ A\subseteq B \]and
\[ P(B)=0.72. \]Could \(P(A)=0.81\)? Explain.
Practice 9: Difference
Suppose
\[ A\subseteq B, \qquad P(A)=0.30, \qquad P(B)=0.74. \]Find
\[ P(B\setminus A). \]Practice 10: Overlapping Events
Suppose
\[ P(A)=0.55, \qquad P(B)=0.48, \qquad P(A\cap B)=0.20. \]Find \(P(A\cup B)\).
Practice 11: Missing Outcome Probability
Let
\[ S=\{a,b,c,d\} \]with
\[ P(\{a\})=0.18,\qquad P(\{b\})=0.22,\qquad P(\{c\})=0.31. \]Find \(P(\{d\})\).
Practice 12: Identify the Error
A student says:
“The sample space has four outcomes, so each outcome must have probability \(1/4\).”
Explain why this conclusion does not follow from the probability axioms.
Practice Answers
Practice 1
Valid probability values must lie in \([0,1]\). Therefore,
\[ \boxed{0,\quad0.47,\quad1} \]could be probabilities.
Practice 2
\[ \boxed{P(\varnothing)=0}. \]Practice 3
\[ P(A^c)=1-0.64=\boxed{0.36}. \]Practice 4
Since \(A\) and \(B\) are disjoint,
\[ P(A\cup B)=0.28+0.41=\boxed{0.69}. \]Practice 5
\[ P(C)=1-0.15-0.50=\boxed{0.35}. \]Practice 6
Yes. Every probability is nonnegative and
\[ 0.4+0.35+0.25=1. \]Thus the singleton assignments satisfy the finite-model requirements.
Practice 7
No. Although
\[ 0.6+0.5-0.1=1, \]the value \(-0.1\) violates nonnegativity.
Practice 8
No. Since
\[ A\subseteq B, \]monotonicity requires
\[ P(A)\le P(B). \]Therefore \(0.81\) cannot be the probability of \(A\) when \(P(B)=0.72\).
Practice 9
\[ P(B\setminus A) = P(B)-P(A) = 0.74-0.30 = \boxed{0.44}. \]Practice 10
\[ P(A\cup B) = 0.55+0.48-0.20 = \boxed{0.83}. \]Practice 11
\[ P(\{d\}) = 1-0.18-0.22-0.31 = \boxed{0.29}. \]Practice 12
The axioms require nonnegative probabilities totaling \(1\), but they do not require equal probabilities. Equal likelihood is an additional assumption. A four-outcome model could assign unequal nonnegative probabilities that sum to \(1\).
What to Remember
The probability axioms are
\[ \boxed{P(A)\ge0}, \] \[ \boxed{P(S)=1}, \]and, for pairwise disjoint events \(A_1,A_2,\ldots\),
\[ \boxed{ P\left(\bigcup_{i=1}^{\infty}A_i\right) = \sum_{i=1}^{\infty}P(A_i) }. \]Important consequences include
\[ \boxed{P(\varnothing)=0}, \] \[ \boxed{0\le P(A)\le1}, \] \[ \boxed{P(A^c)=1-P(A)}, \]and
\[ \boxed{ A\subseteq B \Longrightarrow P(A)\le P(B) }. \]- The axioms enforce consistency, not fairness.
- Disjoint event probabilities add directly.
- Overlapping events cannot usually be added without correction.
- Equal outcome counts do not imply equal probabilities.
- Probability-zero events need not be impossible in continuous models.
The most important habit is: check the structure of the events before applying arithmetic to their probabilities.
Where We Go Next
The axioms establish what every valid probability model must satisfy, but they do not determine when different outcomes should receive the same probability.
The next lesson, Equally Likely Outcomes, develops the symmetry assumptions that justify assigning equal probability to elementary outcomes and using counting ratios to compute event probabilities.