The Central Idea
The mathematical form used to describe a probability distribution depends on whether the random variable is discrete or continuous.
For a discrete random variable \(X\), we use a probability mass function:
\[ \boxed{p_X(x)=P(X=x)}. \]For a continuous random variable \(X\), we use a probability density function \(f_X(x)\), and interval probabilities are found by integration:
\[ \boxed{P(a\le X\le b)=\int_a^b f_X(x)\,dx}. \]The two functions serve parallel purposes, but they must be interpreted differently.
A Probability Distribution
A probability distribution describes how the total probability of \(1\) is distributed across the possible values of a random variable.
For a discrete variable, probability is concentrated at separate points. For a continuous variable, probability is spread across intervals.
In either case, the entire distribution must account for total probability
\[ \boxed{1}. \]Probability Mass Functions
If \(X\) is discrete, its probability mass function, or PMF, assigns a probability to each possible value:
\[ p_X(x)=P(X=x). \]The notation \(p_X\) emphasizes that the function belongs to the random variable \(X\). When the random variable is clear, the shorter notation \(p(x)\) is also common.
The Two Requirements for a PMF
A function \(p(x)\) is a valid probability mass function if it satisfies two conditions.
First, every probability must be nonnegative:
\[ \boxed{p(x)\ge 0}. \]Second, the probabilities over all possible values must sum to \(1\):
\[ \boxed{\sum_x p(x)=1}. \]These conditions encode the basic probability axioms for a discrete distribution.
Example: A Fair Six-Sided Die
Let \(X\) be the number shown when a fair six-sided die is rolled.
The possible values are
\[ 1,2,3,4,5,6. \]Since the die is fair,
\[ p_X(x)=\frac16 \]for \(x\in\{1,2,3,4,5,6\}\), and
\[ p_X(x)=0 \]for all other values.
Checking the Die PMF
Each probability is nonnegative:
\[ \frac16\ge0. \]The total probability is
\[ \sum_{x=1}^{6}p_X(x) = 6\left(\frac16\right) = 1. \]Therefore the function satisfies both PMF requirements.
A PMF Can Be Written as a Table
Suppose \(X\) has possible values \(0\), \(1\), \(2\), and \(3\) with the following probabilities.
| \(x\) | \(p_X(x)\) |
|---|---|
| \(0\) | \(0.10\) |
| \(1\) | \(0.25\) |
| \(2\) | \(0.40\) |
| \(3\) | \(0.25\) |
The probabilities sum to
\[ 0.10+0.25+0.40+0.25=1. \]Therefore the table defines a valid PMF.
Finding a Single Discrete Probability
From the preceding table,
\[ \boxed{P(X=2)=p_X(2)=0.40}. \]A PMF gives the probability of an individual discrete value directly.
Finding a Discrete Event Probability
To find the probability of an event containing several possible values, add the corresponding PMF values.
For example,
\[ P(X\ge2) = P(X=2)+P(X=3). \]Using the table,
\[ P(X\ge2)=0.40+0.25=0.65. \]Thus,
\[ \boxed{P(X\ge2)=0.65}. \]General Discrete Event Rule
If \(A\) is a set of possible values of a discrete random variable \(X\), then
\[ \boxed{P(X\in A)=\sum_{x\in A}p_X(x)}. \]Discrete probability is accumulated by summing the probability masses at the values belonging to the event.
Why It Is Called Probability Mass
The word mass reflects the idea that probability is concentrated at individual points.
If
\[ p_X(4)=0.18, \]then the single point \(X=4\) carries probability \(0.18\).
This pointwise interpretation is characteristic of discrete random variables.
Example: Number of Heads in Two Tosses
Toss two fair coins and let \(X\) be the number of heads.
The outcomes are
\[ HH,\quad HT,\quad TH,\quad TT. \]The random variable values are
\[ X(HH)=2,\qquad X(HT)=1,\qquad X(TH)=1,\qquad X(TT)=0. \]Therefore,
\[ p_X(0)=\frac14, \qquad p_X(1)=\frac12, \qquad p_X(2)=\frac14. \]Probability Density Functions
If \(X\) is continuous, its distribution is described by a probability density function, or PDF, denoted by
\[ f_X(x). \]A density does not assign probability directly to a single point. Instead, probability over an interval is represented by area under the density curve.
The Two Requirements for a PDF
A function \(f(x)\) is a valid probability density function if it satisfies two conditions.
First, the density must be nonnegative:
\[ \boxed{f(x)\ge0}. \]Second, the total area under the density over all possible values must equal \(1\):
\[ \boxed{\int_{-\infty}^{\infty}f(x)\,dx=1}. \]These are the continuous counterparts of the PMF requirements.
Density Is Not Probability at a Point
For a continuous random variable, it is generally incorrect to write
\[ f_X(x)=P(X=x). \]In fact,
\[ \boxed{P(X=x)=0} \]for every exact value \(x\) in a continuous model.
The value \(f_X(x)\) describes local probability density, not point probability.
Probability as Area
If \(X\) is continuous with density \(f_X(x)\), then
\[ \boxed{P(a\le X\le b)=\int_a^b f_X(x)\,dx}. \]Geometrically, this probability is the area under the graph of \(f_X(x)\) between \(x=a\) and \(x=b\).
Open and Closed Endpoints
Since exact points have probability zero for a continuous random variable,
\[ P(aExample: A Uniform Density
Suppose \(X\) has density
\[ f_X(x)= \begin{cases} \frac14, & 0\le x\le4,\\ 0, & \text{otherwise}. \end{cases} \]The density is constant over the interval \([0,4]\).
Check the total area:
\[ \int_0^4 \frac14\,dx = \frac14(4) = 1. \]Therefore the function is a valid PDF.
Finding an Interval Probability
Using the uniform density
\[ f_X(x)=\frac14 \]on \(0\le x\le4\), find
\[ P(1\le X\le3). \]Integrate over the interval:
\[ P(1\le X\le3) = \int_1^3 \frac14\,dx. \]Therefore,
\[ \boxed{P(1\le X\le3)=\frac12}. \]Area Interpretation of the Uniform Example
The interval from \(1\) to \(3\) has width
\[ 3-1=2. \]The density has height
\[ \frac14. \]The probability is therefore the rectangular area
\[ 2\left(\frac14\right)=\frac12. \]Integration generalizes this area calculation to densities that are not constant.
A Density Can Exceed 1
A probability must lie between \(0\) and \(1\), but a probability density value can be greater than \(1\).
For example, consider
\[ f_X(x)= \begin{cases} 2, & 0\le x\le\frac12,\\ 0, & \text{otherwise}. \end{cases} \]The total area is
\[ 2\left(\frac12\right)=1. \]So this is a valid density even though its height is \(2\).
Why Density Can Exceed 1
The value of a density is not itself a probability. Probability comes from area:
\[ \text{probability} = \text{accumulated density over an interval}. \]A tall density over a sufficiently narrow interval can still have total area no greater than \(1\).
PMF and PDF: The Structural Parallel
Discrete and continuous distributions share the same basic probability structure.
| Discrete | Continuous |
|---|---|
| Probability mass function \(p_X(x)\) | Probability density function \(f_X(x)\) |
| \(p_X(x)\ge0\) | \(f_X(x)\ge0\) |
| \(\sum_x p_X(x)=1\) | \(\int_{-\infty}^{\infty}f_X(x)\,dx=1\) |
| Probabilities are found by sums. | Probabilities are found by integrals. |
| \(P(X=x)\) may be positive. | \(P(X=x)=0\). |
Sums and Integrals Play Parallel Roles
For a discrete random variable,
\[ P(X\in A)=\sum_{x\in A}p_X(x). \]For a continuous random variable over an interval,
\[ P(a\le X\le b)=\int_a^b f_X(x)\,dx. \]A sum accumulates probability mass at separate points. An integral accumulates probability density across a continuous range.
Support of a Distribution
The support identifies the values where the distribution places probability or density.
For a PMF, the support is typically the set of values where
\[ p_X(x)>0. \]For a continuous density, the support is associated with the region where the density is positive, together with any relevant boundary values.
Values Outside the Support
A distribution assigns no probability mass or density outside its support.
For example, if \(X\) counts the number of heads in two coin tosses, then
\[ p_X(5)=0. \]If a continuous variable is supported only on \(0\le x\le4\), then
\[ f_X(x)=0 \]outside that interval.
Finding an Unknown Constant in a PMF
Suppose a discrete random variable has PMF
\[ p_X(x)=cx \]for
\[ x\in\{1,2,3\}. \]Since total probability must equal \(1\),
\[ c(1)+c(2)+c(3)=1. \]Thus,
\[ 6c=1 \]and
\[ \boxed{c=\frac16}. \]The Resulting PMF
Substituting
\[ c=\frac16 \]gives
\[ p_X(x)=\frac{x}{6}, \qquad x\in\{1,2,3\}. \]Therefore,
\[ p_X(1)=\frac16, \qquad p_X(2)=\frac13, \qquad p_X(3)=\frac12. \]These probabilities are nonnegative and sum to \(1\).
Finding an Unknown Constant in a PDF
Suppose
\[ f_X(x)=cx \]for \(0\le x\le2\), and \(f_X(x)=0\) otherwise.
Total area must equal \(1\):
\[ \int_0^2 cx\,dx=1. \]Evaluate:
\[ c\left[\frac{x^2}{2}\right]_0^2=1. \]Therefore,
\[ 2c=1 \]and
\[ \boxed{c=\frac12}. \]The Resulting Density
The normalized density is
\[ f_X(x)= \begin{cases} \frac{x}{2}, & 0\le x\le2,\\ 0, & \text{otherwise}. \end{cases} \]For example,
\[ P(0\le X\le1) = \int_0^1 \frac{x}{2}\,dx = \left[\frac{x^2}{4}\right]_0^1 = \frac14. \]Thus,
\[ \boxed{P(0\le X\le1)=\frac14}. \]Complement Probabilities with a PMF
The complement rule can reduce the amount of summation required.
For a discrete random variable,
\[ P(X\notin A)=1-P(X\in A). \]For example,
\[ P(X\ge3)=1-P(X<3) \]may be easier to compute when only a few values lie below \(3\).
Complement Probabilities with a PDF
The same rule applies to continuous distributions:
\[ P(X\notin A)=1-P(X\in A). \]For example,
\[ P(X>b)=1-P(X\le b). \]Whether the variable is discrete or continuous, complements are governed by the same probability law.
Graphing a PMF
A PMF is often graphed using separate points or vertical bars at the possible values of \(X\).
The height at each value \(x\) represents
\[ p_X(x)=P(X=x). \]The gaps between admissible values reinforce the discrete structure of the variable.
Graphing a PDF
A PDF is represented by a curve or piecewise-defined graph over a continuous range.
The height of the curve indicates density. Probability is represented by area beneath the curve over the relevant interval.
The graph should therefore be read in terms of accumulated area rather than isolated heights.
Higher Density Means Greater Local Concentration
If one region of a continuous density is higher than another, probability is more concentrated per unit of the horizontal variable in that region.
This does not mean that a single point in the higher region has positive probability. Individual points still have probability zero.
Density and Units
A PMF value is itself a probability and therefore has no physical unit.
A probability density generally carries reciprocal units so that integrating density over the horizontal variable produces a dimensionless probability.
For example, if \(X\) is measured in hours, a density may be interpreted as probability per hour.
A PMF Is Not a Frequency Table
A frequency table records how often values occurred in observed data. A PMF describes probabilities in a probability model.
Relative frequencies from data may be used to estimate probabilities, but an observed frequency and a theoretical probability are not automatically the same object.
A PDF Is Not a Histogram
A histogram summarizes observed data by placing observations into bins. A PDF is a mathematical function describing a continuous probability distribution.
A suitably scaled histogram may resemble an underlying density, but the two should not be confused.
Checking Whether a Proposed PMF Is Valid
Suppose
\[ p_X(0)=0.2,\qquad p_X(1)=0.5,\qquad p_X(2)=0.4. \]Every value is nonnegative, but
\[ 0.2+0.5+0.4=1.1. \]Because the probabilities do not sum to \(1\), this is not a valid PMF.
Negative Values Make a PMF Invalid
Suppose a proposed PMF includes
\[ p_X(2)=-0.1. \]This immediately violates
\[ p_X(x)\ge0. \]Therefore the function cannot represent a probability distribution, regardless of whether the listed values happen to sum to \(1\).
Checking Whether a Proposed PDF Is Valid
Suppose
\[ f_X(x)= \begin{cases} 3x^2, & 0\le x\le1,\\ 0, & \text{otherwise}. \end{cases} \]The function is nonnegative on its support.
Its total area is
\[ \int_0^1 3x^2\,dx = \left[x^3\right]_0^1 = 1. \]Therefore it is a valid PDF.
A Nonnegative Function May Still Fail as a PDF
Consider
\[ f(x)= \begin{cases} x, & 0\le x\le1,\\ 0, & \text{otherwise}. \end{cases} \]The function is nonnegative, but
\[ \int_0^1 x\,dx=\frac12. \]Since the total area is not \(1\), the function is not a normalized probability density.
Common Error: Using a PMF Formula for a Continuous Variable
If \(X\) is continuous, writing
\[ P(X=x)=f_X(x) \]confuses density with probability.
The correct interpretation is that interval probability comes from integrating the density:
\[ P(a\le X\le b)=\int_a^b f_X(x)\,dx. \]Common Error: Integrating a PMF
A PMF belongs to a discrete random variable, so probability is accumulated by summation:
\[ P(X\in A)=\sum_{x\in A}p_X(x). \]Integration is the natural accumulation operation for continuous density, not for ordinary discrete probability mass.
Common Error: Assuming Density Height Is Probability
If
\[ f_X(2)=0.7, \]this does not mean
\[ P(X=2)=0.7. \]For a continuous random variable,
\[ P(X=2)=0. \]The value \(0.7\) describes the density at \(x=2\).
Common Error: Requiring a PDF to Stay Below 1
A probability must not exceed \(1\), but a density value can exceed \(1\).
What must equal \(1\) is the total area:
\[ \int_{-\infty}^{\infty}f_X(x)\,dx=1. \]Confusing density height with probability leads to an unnecessary and incorrect restriction.
Common Error: Forgetting Values Outside the Support
A piecewise PMF or PDF should specify what happens outside the listed support.
For example,
\[ f_X(x)= \begin{cases} \frac14, & 0\le x\le4,\\ 0, & \text{otherwise} \end{cases} \]makes clear that the density vanishes outside \([0,4]\).
A Reliable Procedure for a PMF
- Identify the possible values of the discrete random variable.
- Determine or write the probability assigned to each value.
- Check that every probability is nonnegative.
- Check that all probabilities sum to \(1\).
- For an event, add the PMF values belonging to that event.
- Use complements when they reduce the amount of calculation.
A Reliable Procedure for a PDF
- Identify the support of the continuous random variable.
- Check that the proposed density is nonnegative on its support.
- Check that the total area under the density is \(1\).
- Translate the desired event into an interval or collection of intervals.
- Integrate the density over that region.
- Use complements when they simplify the calculation.
- Remember that isolated points have probability zero.
Practice 1: Check a PMF
A discrete random variable has
\[ p_X(0)=0.15,\qquad p_X(1)=0.35,\qquad p_X(2)=0.30,\qquad p_X(3)=0.20. \]Is this a valid PMF?
Practice 2: Discrete Event Probability
Using the PMF from Practice 1, find
\[ P(X\ge2). \]Practice 3: Unknown PMF Constant
Suppose
\[ p_X(x)=c(x+1) \]for \(x\in\{0,1,2\}\).
Find \(c\).
Practice 4: Check a PDF
Consider
\[ f_X(x)= \begin{cases} \frac12, & 0\le x\le2,\\ 0, & \text{otherwise}. \end{cases} \]Verify that this is a valid PDF.
Practice 5: Continuous Interval Probability
Using the density from Practice 4, find
\[ P(0.5\le X\le1.5). \]Practice 6: Exact Continuous Value
Using the density from Practice 4, find
\[ P(X=1). \]Practice 7: Unknown PDF Constant
Suppose
\[ f_X(x)= \begin{cases} cx^2, & 0\le x\le1,\\ 0, & \text{otherwise}. \end{cases} \]Find \(c\).
Practice 8: Use the Normalized Density
Using the density from Practice 7, find
\[ P\left(0\le X\le\frac12\right). \]Practice 9: Identify the Error
A student says that a continuous density cannot have a value greater than \(1\) because probabilities cannot exceed \(1\).
Explain the error.
Practice 10: PMF or PDF?
Decide which object is appropriate in each case.
- Number of customers arriving in one hour.
- Waiting time until the next customer arrives.
- Number of defective items in a sample.
- Measured lifetime of a component.
Practice Answers
Practice 1
Every probability is nonnegative, and
\[ 0.15+0.35+0.30+0.20=1. \]Therefore, \(\boxed{\text{the PMF is valid}}\).
Practice 2
\[ P(X\ge2) = p_X(2)+p_X(3) = 0.30+0.20. \]Thus,
\[ \boxed{P(X\ge2)=0.50}. \]Practice 3
Total probability must equal \(1\):
\[ c(1)+c(2)+c(3)=1. \] \[ 6c=1. \]Therefore,
\[ \boxed{c=\frac16}. \]Practice 4
The density is nonnegative, and
\[ \int_0^2\frac12\,dx=1. \]Therefore it is a valid PDF.
Practice 5
\[ P(0.5\le X\le1.5) = \int_{0.5}^{1.5}\frac12\,dx. \]The interval has width \(1\), so
\[ \boxed{P(0.5\le X\le1.5)=\frac12}. \]Practice 6
Since \(X\) is continuous,
\[ \boxed{P(X=1)=0}. \]Practice 7
\[ \int_0^1 cx^2\,dx=1. \] \[ c\left[\frac{x^3}{3}\right]_0^1=1. \] \[ \frac{c}{3}=1. \]Therefore,
\[ \boxed{c=3}. \]Practice 8
The normalized density is \(f_X(x)=3x^2\) on \([0,1]\). Therefore,
\[ P\left(0\le X\le\frac12\right) = \int_0^{1/2}3x^2\,dx. \] \[ = \left[x^3\right]_0^{1/2} = \frac18. \]Thus,
\[ \boxed{\frac18}. \]Practice 9
A density value is not itself a probability. A PDF may exceed \(1\) over part of its support as long as it remains nonnegative and its total area is \(1\).
Practice 10
- PMF.
- PDF.
- PMF.
- PDF.
What to Remember
For a discrete random variable,
\[ \boxed{p_X(x)=P(X=x)} \]with
\[ p_X(x)\ge0 \]and
\[ \sum_x p_X(x)=1. \]For a continuous random variable,
\[ f_X(x)\ge0 \]and
\[ \int_{-\infty}^{\infty}f_X(x)\,dx=1. \]Interval probabilities are found from
\[ \boxed{P(a\le X\le b)=\int_a^b f_X(x)\,dx}. \]The essential distinction is: PMF values are probabilities at discrete points; PDF values are densities whose areas produce continuous probabilities.
Where We Go Next
PMFs and PDFs describe how probability is distributed locally. Another function summarizes how much probability has accumulated up to a given value.
The next lesson, Cumulative Distributions, develops the cumulative distribution function and shows how it provides a unified description of both discrete and continuous random variables.