The Central Idea
Suppose \(H\) represents a hypothesis or underlying state and \(E\) represents observed evidence.
Bayesian updating compares:
- the probability of \(H\) before seeing \(E\);
- the probability of observing \(E\) if \(H\) is true;
- the probability of \(H\) after observing \(E\).
These correspond to:
\[ \boxed{\text{prior}} \qquad \boxed{\text{likelihood}} \qquad \boxed{\text{posterior}}. \]The Prior
The prior probability is the probability assigned to a hypothesis before the new evidence is incorporated.
For a hypothesis \(H\),
\[ \boxed{P(H)} \]is its prior probability.
A prior may come from historical frequency, previous data, a model, an earlier analysis, or another justified source of information.
The Likelihood
The likelihood measures how compatible the observed evidence is with a particular hypothesis.
For evidence \(E\) and hypothesis \(H\),
\[ \boxed{P(E\mid H)} \]is the probability of observing \(E\) if \(H\) is true.
When the evidence is fixed and hypotheses are being compared, \(P(E\mid H)\) is treated as a function of the hypothesis.
The Posterior
The posterior probability is the probability assigned to the hypothesis after the evidence has been incorporated.
It is written
\[ \boxed{P(H\mid E)}. \]The posterior is the updated probability of \(H\) given the observed evidence \(E\).
The Bayesian Update
Bayes' theorem connects the three quantities:
\[ \boxed{ P(H\mid E) = \frac{P(E\mid H)P(H)}{P(E)} }. \]In words:
\[ \boxed{ \text{posterior} = \frac{\text{likelihood}\times\text{prior}} {\text{evidence probability}} }. \]The Unnormalized Update
Before dividing by \(P(E)\), the product
\[ P(E\mid H)P(H) \]measures the joint probability
\[ P(H\cap E). \]This quantity combines how common the hypothesis was before the evidence with how compatible the evidence is with that hypothesis.
Why Normalization Is Needed
The quantity
\[ P(E\mid H)P(H) \]is not yet a conditional probability over the hypotheses.
To obtain a posterior, we divide by the total probability of the observed evidence:
\[ P(E). \]This rescales the competing hypothesis weights so that the resulting posterior probabilities sum to \(1\).
Total Probability Provides the Normalizing Constant
Suppose
\[ H_1,H_2,\ldots,H_n \]form a partition of the possible hypotheses.
Then
\[ P(E) = \sum_{i=1}^{n}P(H_i)P(E\mid H_i). \]Therefore,
\[ \boxed{ P(H_k\mid E) = \frac{P(E\mid H_k)P(H_k)} {\sum_{i=1}^{n}P(E\mid H_i)P(H_i)} }. \]A Three-Part Vocabulary
For a fixed hypothesis \(H\) and observed evidence \(E\):
- prior: \(P(H)\);
- likelihood: \(P(E\mid H)\);
- posterior: \(P(H\mid E)\).
The direction of conditioning is essential.
A First Numerical Example
Suppose a hypothesis has prior probability
\[ P(H)=0.30. \]Suppose the observed evidence has probability
\[ P(E\mid H)=0.80 \]if \(H\) is true, and the total evidence probability is
\[ P(E)=0.50. \]Then
\[ P(H\mid E) = \frac{0.80(0.30)}{0.50} = \boxed{0.48}. \]Identify the Three Quantities
In the previous example:
\[ \boxed{P(H)=0.30} \]is the prior,
\[ \boxed{P(E\mid H)=0.80} \]is the likelihood of the observed evidence under \(H\), and
\[ \boxed{P(H\mid E)=0.48} \]is the posterior.
Likelihood Is Not the Posterior
A common confusion is to treat
\[ P(E\mid H) \]as though it were
\[ P(H\mid E). \]These probabilities reverse the direction of conditioning.
Likelihood asks how probable the evidence is under a hypothesis. Posterior probability asks how probable the hypothesis is after the evidence is observed.
Likelihood Is Used to Compare Hypotheses
Suppose the same evidence \(E\) can arise under hypotheses \(H_1\) and \(H_2\).
If
\[ P(E\mid H_1)>P(E\mid H_2), \]then the evidence is more compatible with \(H_1\) than with \(H_2\).
However, the posterior probabilities also depend on the priors.
A Stronger Likelihood Does Not Always Produce a Larger Posterior
Suppose
\[ P(H_1)=0.01, \qquad P(H_2)=0.99, \]while
\[ P(E\mid H_1)=0.90 \]and
\[ P(E\mid H_2)=0.10. \]Although \(E\) is much more likely under \(H_1\), the prior for \(H_1\) is extremely small.
The posterior must combine both pieces of information.
Calculate the Competing Joint Weights
For \(H_1\),
\[ P(E\mid H_1)P(H_1) = 0.90(0.01) = 0.009. \]For \(H_2\),
\[ P(E\mid H_2)P(H_2) = 0.10(0.99) = 0.099. \]The second path remains much larger despite its lower likelihood.
Normalize the Competing Weights
The total evidence probability is
\[ P(E)=0.009+0.099=0.108. \]Therefore,
\[ P(H_1\mid E) = \frac{0.009}{0.108} = \boxed{\frac{1}{12}} \approx \boxed{0.0833}. \]The evidence raises the probability of \(H_1\) from \(1\%\) to about \(8.33\%\), but \(H_1\) still remains less probable than \(H_2\).
Prior and Evidence Work Together
A posterior is not determined by the prior alone.
It is also not determined by the likelihood alone.
Bayesian updating combines:
\[ \boxed{ \text{previous probability information} + \text{new evidence} }. \]Example: Manufacturing Source
Suppose a factory receives \(70\%\) of parts from Supplier 1 and \(30\%\) from Supplier 2:
\[ P(S_1)=0.70, \qquad P(S_2)=0.30. \]Their defect rates are
\[ P(D\mid S_1)=0.02 \]and
\[ P(D\mid S_2)=0.06. \]Identify the Priors and Likelihoods
Before observing whether a part is defective, the supplier probabilities are the priors:
\[ P(S_1)=0.70, \qquad P(S_2)=0.30. \]Once the evidence
\[ D=\{\text{defective}\} \]is observed, the likelihoods are
\[ P(D\mid S_1)=0.02, \qquad P(D\mid S_2)=0.06. \]Find the Evidence Probability
Total probability gives
\[ P(D) = 0.70(0.02)+0.30(0.06). \]Therefore,
\[ P(D)=0.014+0.018=\boxed{0.032}. \]Find the Posterior for Supplier 2
The posterior is
\[ P(S_2\mid D) = \frac{P(D\mid S_2)P(S_2)}{P(D)}. \]Substitute:
\[ P(S_2\mid D) = \frac{0.06(0.30)}{0.032}. \]Thus,
\[ \boxed{ P(S_2\mid D)=0.5625 }. \]The supplier probability updates from a prior of \(30\%\) to a posterior of \(56.25\%\) after observing a defect.
Posterior Probabilities Across All Hypotheses Sum to One
In the two-supplier example,
\[ P(S_1\mid D)+P(S_2\mid D)=1. \]Since
\[ P(S_2\mid D)=0.5625, \]we have
\[ P(S_1\mid D)=1-0.5625=\boxed{0.4375}. \]Normalization distributes all posterior probability across the complete set of competing hypotheses.
Example: Three Competing Hypotheses
Suppose three possible sources have priors
\[ P(H_1)=0.50, \qquad P(H_2)=0.30, \qquad P(H_3)=0.20. \]For observed evidence \(E\),
\[ P(E\mid H_1)=0.10, \] \[ P(E\mid H_2)=0.20, \] \[ P(E\mid H_3)=0.50. \]Compute the Unnormalized Weights
For \(H_1\),
\[ 0.50(0.10)=0.05. \]For \(H_2\),
\[ 0.30(0.20)=0.06. \]For \(H_3\),
\[ 0.20(0.50)=0.10. \]These are the joint probabilities of each hypothesis with the evidence.
Normalize the Three Weights
The evidence probability is
\[ P(E)=0.05+0.06+0.10=0.21. \]Therefore,
\[ P(H_1\mid E)=\frac{0.05}{0.21}\approx0.2381, \] \[ P(H_2\mid E)=\frac{0.06}{0.21}\approx0.2857, \] \[ P(H_3\mid E)=\frac{0.10}{0.21}\approx0.4762. \]The posterior probabilities sum to \(1\).
The Evidence Can Change the Ranking of Hypotheses
Before observing \(E\),
\[ H_1 \]had the largest prior probability:
\[ P(H_1)=0.50. \]After observing \(E\),
\[ H_3 \]has the largest posterior probability:
\[ P(H_3\mid E)\approx0.4762. \]Evidence can therefore change both the numerical probabilities and the ordering of competing explanations.
Posterior as Updated Prior
After evidence has been incorporated, a posterior can become the starting probability for a later update.
Conceptually:
\[ \text{prior} \longrightarrow \text{posterior after first evidence} \longrightarrow \text{new prior for later evidence}. \]This idea will become central in the later lesson on sequential updating.
Example: First Evidence Updates a Hypothesis
Suppose
\[ P(H)=0.20. \]After observing evidence \(E_1\), suppose Bayes' theorem gives
\[ P(H\mid E_1)=0.50. \]If later evidence \(E_2\) is observed, the updated value
\[ 0.50 \]can serve as the starting probability for the next Bayesian update, provided the second update is modeled consistently.
Likelihood as a Function of the Hypothesis
Once evidence \(E\) has been observed, it is fixed.
We can then compare how the quantity
\[ P(E\mid H) \]changes as \(H\) changes.
This is why likelihood is often described as a function of the hypothesis or model parameter rather than as a probability distribution over hypotheses.
Likelihood Values Need Not Sum to One Across Hypotheses
Suppose
\[ P(E\mid H_1)=0.80 \]and
\[ P(E\mid H_2)=0.70. \]Their sum is
\[ 1.50. \]This is not a contradiction.
These are probabilities of the same evidence under different hypotheses, not probabilities assigned across mutually exclusive hypotheses.
Posterior Probabilities Do Sum to One Across a Complete Hypothesis Set
If
\[ H_1,\ldots,H_n \]form a complete partition of the possible hypotheses, then
\[ \boxed{ \sum_{i=1}^{n}P(H_i\mid E)=1 }. \]This is one of the key distinctions between likelihood values and posterior probabilities.
Prior Probabilities Also Sum to One Across a Complete Hypothesis Set
Before the evidence is observed,
\[ \boxed{ \sum_{i=1}^{n}P(H_i)=1 } \]whenever \(H_1,\ldots,H_n\) form a partition.
Bayesian updating transforms one probability distribution across the hypotheses into another.
A Compact Update Table
For several hypotheses, it is useful to organize the calculation in four columns:
| Hypothesis | Prior | Likelihood | Prior × Likelihood |
|---|---|---|---|
| \(H_1\) | \(P(H_1)\) | \(P(E\mid H_1)\) | \(P(H_1)P(E\mid H_1)\) |
| \(H_2\) | \(P(H_2)\) | \(P(E\mid H_2)\) | \(P(H_2)P(E\mid H_2)\) |
Divide each final-column entry by the sum of that column to obtain the posteriors.
Example: Update Table
Suppose
| Hypothesis | Prior | Likelihood | Prior × Likelihood |
|---|---|---|---|
| \(H_1\) | \(0.60\) | \(0.20\) | \(0.12\) |
| \(H_2\) | \(0.40\) | \(0.50\) | \(0.20\) |
The normalizing total is
\[ 0.12+0.20=0.32. \]Find the Posterior from the Update Table
For \(H_1\),
\[ P(H_1\mid E) = \frac{0.12}{0.32} = \boxed{0.375}. \]For \(H_2\),
\[ P(H_2\mid E) = \frac{0.20}{0.32} = \boxed{0.625}. \]Evidence has shifted the balance from \(H_1\) toward \(H_2\).
Bayesian Updating as Reweighting
Each hypothesis begins with a prior weight.
The likelihood scales that weight according to how well the hypothesis predicts or supports the observed evidence.
Normalization then converts the adjusted weights into posterior probabilities.
In shorthand:
\[ \boxed{ \text{posterior} \propto \text{likelihood}\times\text{prior} }. \]What the Proportionality Symbol Means
The expression
\[ P(H\mid E)\propto P(E\mid H)P(H) \]means that, for fixed evidence \(E\), posterior probabilities are proportional to prior-times-likelihood values.
To obtain actual probabilities, the values must still be normalized so that the posterior distribution sums to \(1\).
Evidence Favoring a Hypothesis
If evidence is relatively more probable under \(H\) than under competing hypotheses, the evidence tends to increase the posterior probability of \(H\).
The amount of change depends on both:
- how strongly the evidence distinguishes \(H\);
- the prior probability assigned to \(H\).
Evidence Opposing a Hypothesis
If the observed evidence is relatively unlikely under \(H\) but more likely under competing hypotheses, the posterior probability of \(H\) can fall below its prior.
Bayesian updating can therefore strengthen or weaken a hypothesis.
Uninformative Evidence
Suppose
\[ P(E\mid H_1) = P(E\mid H_2) = \cdots = P(E\mid H_n). \]Then the evidence treats every hypothesis equally.
In that case, posterior probabilities remain proportional to the priors, and after normalization the posterior distribution equals the prior distribution.
Example: Uninformative Evidence
Suppose
\[ P(H_1)=0.70, \qquad P(H_2)=0.30, \]and
\[ P(E\mid H_1)=P(E\mid H_2)=0.40. \]The unnormalized weights are
\[ 0.70(0.40)=0.28 \]and
\[ 0.30(0.40)=0.12. \]Their total is \(0.40\), so
\[ P(H_1\mid E)=0.70, \qquad P(H_2\mid E)=0.30. \]The evidence produces no update.
Example: Strong Evidence Against the Initially Favored Hypothesis
Suppose
\[ P(H_1)=0.80, \qquad P(H_2)=0.20, \]but
\[ P(E\mid H_1)=0.05, \qquad P(E\mid H_2)=0.80. \]The unnormalized weights are
\[ 0.80(0.05)=0.04 \]and
\[ 0.20(0.80)=0.16. \]The posterior probabilities are therefore
\[ P(H_1\mid E)=0.20 \]and
\[ P(H_2\mid E)=0.80. \]Strong evidence can reverse the initial ranking.
Example: Risk Classification
Suppose \(10\%\) of borrowers eventually default:
\[ P(D)=0.10. \]A model labels borrowers high risk with probabilities
\[ P(H\mid D)=0.80 \]and
\[ P(H\mid D^c)=0.15. \]Here \(P(D)\) is the prior for default, while the two high-risk classification rates act as likelihoods for the observation \(H\).
Compute the Posterior Default Probability
First,
\[ P(H) = 0.10(0.80)+0.90(0.15) = 0.215. \]Then
\[ P(D\mid H) = \frac{0.10(0.80)}{0.215} \approx \boxed{0.3721}. \]The default probability updates from a prior of \(10\%\) to a posterior of about \(37.21\%\) after the high-risk classification is observed.
The Prior Is Context-Dependent
A prior is not necessarily a universal constant.
It may differ across populations, time periods, operating environments, or decision contexts.
If the underlying population changes, then the relevant prior may also change.
The Likelihood Model Is Also Context-Dependent
The probability
\[ P(E\mid H) \]depends on how the evidence is generated.
If measurement procedures, testing technology, operating conditions, or sampling processes change, the likelihood model may change as well.
The Posterior Depends on the Entire Model
A posterior probability is only as reliable as the assumptions and inputs used to calculate it.
Errors in priors, likelihoods, classifications, or evidence modeling can propagate into the posterior.
Prior Does Not Mean Guess
The word prior refers to timing relative to the current evidence.
A prior may be based on substantial historical evidence.
It simply represents the probability distribution before the particular evidence currently being incorporated.
Posterior Does Not Mean Final Forever
A posterior is final only relative to the evidence used in the current update.
New evidence can lead to another update.
Bayesian reasoning is therefore naturally sequential.
Likelihood Is Not a Probability Distribution over Hypotheses
When comparing hypotheses using fixed evidence,
\[ P(E\mid H_i) \]gives one probability for each hypothesis.
These values do not generally sum to \(1\), because each one comes from a different conditional probability model.
The posterior probabilities, after normalization, do sum to \(1\).
Common Error: Calling \(P(H\mid E)\) the Likelihood
The likelihood associated with fixed evidence \(E\) is based on
\[ P(E\mid H). \]The quantity
\[ P(H\mid E) \]is the posterior probability of the hypothesis after observing the evidence.
Common Error: Ignoring the Prior
Comparing only
\[ P(E\mid H_1) \]and
\[ P(E\mid H_2) \]does not generally determine the posterior probabilities.
The prior weights
\[ P(H_1),\qquad P(H_2) \]must also be included.
Common Error: Forgetting Normalization
The values
\[ P(E\mid H_i)P(H_i) \]are unnormalized weights.
They become posterior probabilities only after dividing by
\[ \sum_iP(E\mid H_i)P(H_i). \]Common Error: Expecting Likelihoods to Sum to One
Likelihood values are probabilities of the evidence under different hypotheses.
They need not sum to \(1\).
Priors and posteriors across a complete set of mutually exclusive hypotheses do sum to \(1\).
Common Error: Confusing an Update with Certainty
A posterior probability can increase substantially without reaching \(1\).
Evidence may favor a hypothesis while leaving meaningful probability on competing hypotheses.
Common Error: Ignoring Alternative Hypotheses
The denominator in a Bayesian update must account for every hypothesis in the partition that can produce the observed evidence.
Leaving out a plausible competing path can distort the posterior.
Common Error: Treating the Prior as Immutable
Priors are starting probabilities for a given update.
Once evidence is incorporated, the posterior can become the starting point for later reasoning.
A Reliable Procedure for Two Hypotheses
- Identify the hypotheses \(H\) and \(H^c\).
- Write their prior probabilities.
- Identify the observed evidence \(E\).
- Write \(P(E\mid H)\) and \(P(E\mid H^c)\).
- Multiply each prior by its likelihood.
- Add the resulting weights to find \(P(E)\).
- Divide each desired weight by \(P(E)\).
- Check that the posterior probabilities sum to \(1\).
- Compare the posterior with the prior to describe the direction of the update.
A Reliable Procedure for Several Hypotheses
- List the mutually exclusive, collectively exhaustive hypotheses.
- Record each prior \(P(H_i)\).
- Record each likelihood \(P(E\mid H_i)\).
- Compute \(P(H_i)P(E\mid H_i)\) for every hypothesis.
- Add those products to obtain the normalizing constant \(P(E)\).
- Divide each product by \(P(E)\) to obtain the posterior.
- Verify that the posterior probabilities sum to \(1\).
- Interpret how the evidence reweighted the hypotheses.
Example: Full Two-Hypothesis Workflow
Suppose
\[ P(H_1)=0.40, \qquad P(H_2)=0.60, \]and the observed evidence has likelihoods
\[ P(E\mid H_1)=0.75, \qquad P(E\mid H_2)=0.25. \]The unnormalized weights are
\[ 0.40(0.75)=0.30 \]and
\[ 0.60(0.25)=0.15. \]Therefore,
\[ P(E)=0.45. \]The posteriors are
\[ P(H_1\mid E)=\frac{0.30}{0.45}=\boxed{\frac{2}{3}} \]and
\[ P(H_2\mid E)=\frac{0.15}{0.45}=\boxed{\frac{1}{3}}. \]Example: Full Three-Hypothesis Workflow
Suppose
\[ P(H_1)=0.20,\qquad P(H_2)=0.50,\qquad P(H_3)=0.30, \]with likelihoods
\[ P(E\mid H_1)=0.70, \] \[ P(E\mid H_2)=0.20, \] \[ P(E\mid H_3)=0.40. \]The unnormalized weights are
\[ 0.14,\qquad0.10,\qquad0.12. \]Their sum is
\[ P(E)=0.36. \]Therefore,
\[ P(H_1\mid E)=\frac{0.14}{0.36}\approx0.3889, \] \[ P(H_2\mid E)=\frac{0.10}{0.36}\approx0.2778, \] \[ P(H_3\mid E)=\frac{0.12}{0.36}\approx0.3333. \]The evidence moves \(H_1\) from the smallest prior to the largest posterior.
Practice 1: Identify the Vocabulary
Suppose
\[ P(H)=0.25, \qquad P(E\mid H)=0.80, \qquad P(H\mid E)=0.50. \]Identify the prior, likelihood, and posterior.
Practice 2: One Bayesian Update
Suppose
\[ P(H)=0.40, \qquad P(E\mid H)=0.75, \qquad P(E)=0.50. \]Find \(P(H\mid E)\).
Practice 3: Two Hypotheses
Suppose
\[ P(H_1)=0.60, \qquad P(H_2)=0.40, \]and
\[ P(E\mid H_1)=0.20, \qquad P(E\mid H_2)=0.50. \]Find both posterior probabilities.
Practice 4: Three Hypotheses
Suppose the priors are
\[ 0.50,\qquad0.30,\qquad0.20, \]and the likelihoods for observed evidence are
\[ 0.10,\qquad0.20,\qquad0.50. \]Find the posterior probability of the third hypothesis.
Practice 5: Uninformative Evidence
Suppose
\[ P(H_1)=0.65, \qquad P(H_2)=0.35, \]and
\[ P(E\mid H_1)=P(E\mid H_2)=0.40. \]Find the posterior probabilities.
Practice 6: Evidence Changes the Ranking
Suppose
\[ P(H_1)=0.80, \qquad P(H_2)=0.20, \]but
\[ P(E\mid H_1)=0.10, \qquad P(E\mid H_2)=0.90. \]Find the posterior probability of \(H_2\).
Practice 7: Manufacturing
Supplier 1 provides \(75\%\) of parts with a \(1\%\) defect rate. Supplier 2 provides \(25\%\) with a \(5\%\) defect rate.
Given a defective part, find the posterior probability that it came from Supplier 2.
Practice 8: Likelihoods Need Not Sum to One
Suppose
\[ P(E\mid H_1)=0.80 \]and
\[ P(E\mid H_2)=0.70. \]Is there a contradiction because the two values sum to \(1.50\)? Explain.
Practice 9: Posterior Becomes New Prior
A hypothesis begins with prior probability \(0.20\). After observing evidence \(E_1\), its posterior becomes \(0.55\).
What probability should serve as the starting probability for a later, properly modeled update using new evidence \(E_2\)?
Practice 10: Identify the Error
A student computes
\[ P(E\mid H_1)P(H_1)=0.18 \]and immediately states
\[ P(H_1\mid E)=0.18. \]Explain what is missing.
Practice Answers
Practice 1
The prior is
\[ \boxed{P(H)=0.25}. \]The likelihood is
\[ \boxed{P(E\mid H)=0.80}. \]The posterior is
\[ \boxed{P(H\mid E)=0.50}. \]Practice 2
\[ P(H\mid E) = \frac{0.75(0.40)}{0.50} = \boxed{0.60}. \]Practice 3
The unnormalized weights are
\[ 0.60(0.20)=0.12 \]and
\[ 0.40(0.50)=0.20. \]The total is
\[ 0.32. \]Therefore,
\[ \boxed{P(H_1\mid E)=0.375} \]and
\[ \boxed{P(H_2\mid E)=0.625}. \]Practice 4
The unnormalized weights are
\[ 0.50(0.10)=0.05, \] \[ 0.30(0.20)=0.06, \] \[ 0.20(0.50)=0.10. \]The total is
\[ 0.21. \]Therefore,
\[ \boxed{ P(H_3\mid E)=\frac{0.10}{0.21}\approx0.4762 }. \]Practice 5
Because the likelihood is the same under both hypotheses, the evidence does not change their relative weights.
Therefore,
\[ \boxed{P(H_1\mid E)=0.65} \]and
\[ \boxed{P(H_2\mid E)=0.35}. \]Practice 6
The unnormalized weights are
\[ 0.80(0.10)=0.08 \]and
\[ 0.20(0.90)=0.18. \]The total is
\[ 0.26. \]Therefore,
\[ \boxed{ P(H_2\mid E)=\frac{0.18}{0.26}\approx0.6923 }. \]Practice 7
The defect probability is
\[ P(D) = 0.75(0.01)+0.25(0.05) = 0.0075+0.0125 = 0.0200. \]Therefore,
\[ P(S_2\mid D) = \frac{0.25(0.05)}{0.0200} = \boxed{0.625}. \]Practice 8
No. Likelihoods are probabilities of the same evidence under different hypotheses. They are not probabilities distributed across the hypotheses, so they do not need to sum to \(1\).
Practice 9
The posterior after the first update becomes the starting probability for the next consistent update:
\[ \boxed{0.55}. \]Practice 10
The value
\[ 0.18 \]is an unnormalized prior-times-likelihood weight, equal to the joint probability \(P(H_1\cap E)\).
To obtain the posterior, it must be divided by the total evidence probability:
\[ P(H_1\mid E) = \frac{0.18}{P(E)}. \]What to Remember
Bayesian updating separates probability reasoning into three roles:
\[ \boxed{\text{prior}=P(H)} \] \[ \boxed{\text{likelihood}=P(E\mid H)} \] \[ \boxed{\text{posterior}=P(H\mid E)}. \]Bayes' theorem connects them:
\[ \boxed{ P(H\mid E) = \frac{P(E\mid H)P(H)}{P(E)} }. \]Equivalently,
\[ \boxed{ \text{posterior} \propto \text{likelihood}\times\text{prior} }. \]- The prior describes probability before the current evidence.
- The likelihood measures how compatible the evidence is with a hypothesis.
- The posterior describes probability after the evidence is incorporated.
- Prior-times-likelihood values must be normalized.
- Likelihood values do not need to sum to \(1\) across hypotheses.
- Posterior probabilities do sum to \(1\) across a complete hypothesis set.
- A posterior can serve as a later prior when new evidence arrives.
The central habit is: separate what was believed before the evidence, how the evidence behaves under each possibility, and what should be believed after the update.
Where We Go Next
Prior, likelihood, and posterior describe the basic architecture of a Bayesian update. The next step is to compare how strongly evidence favors one hypothesis over another without yet performing the full normalization.
The next lesson, Odds and Likelihood Ratios, develops odds-based Bayesian updating and shows how evidence can multiply prior odds into posterior odds.