CDF method, change of variables, LOTUS, moments, skewness, kurtosis, moment generating functions, and the distributional algebra of sums — the analytical backbone of distribution theory
Probability models do not always arrive in convenient form. A random variable may measure voltage, but the quantity of interest is power, which depends on the square of voltage. A model may describe individual component lifetimes, but the question concerns the total lifetime of a system built from many components. Data science pipelines routinely apply logarithms, normalizations, and other functions to raw measurements before any analysis begins. In every case, the underlying question is the same: if a random variable $X$ has a known distribution, what is the distribution of $Y = g(X)$, and what can we say about its expected value, its spread, and its shape?
This chapter develops the tools for answering that question systematically. The first half studies transformation methods: how to derive the distribution of a function of one or more random variables, starting with direct probability arguments and extending to the change-of-variables technique that uses derivatives and integrals. The second half studies moment methods: how to describe and compare distributions through their moments, and how moment generating functions provide an algebraically powerful way to compute moments, identify distributions, and analyze sums of independent random variables.
Together, transformations and moment methods form the analytical backbone of distribution theory. They are the techniques that turn raw probability models into answers about real quantities in science, engineering, finance, and data analysis. They are also the tools that will make the central limit theorem and the sampling distributions of Chapter 7 feel mathematically grounded rather than mysterious.
Students who have studied Chapters 3 through 5 carefully are well prepared. This chapter draws directly on probability density functions, cumulative distribution functions, expected value as an integral, variance, and the basic properties of joint distributions. The calculus of derivatives, integrals, and the chain rule will appear throughout. The chapter moves carefully and illustrates every technique with worked examples before asking students to apply it independently.
An engineer models voltage $V$ across a resistor as a random variable. The power dissipated is $P = V^2/R$, where $R$ is fixed. If the distribution of $V$ is known, what is the distribution of $P$? Can we compute $E[P]$ without finding the distribution of $P$ first? These are the questions this chapter teaches you to answer — and the answers turn out to be far more elegant than you might expect.
In Chapters 3 and 4, random variables were introduced as numerical functions on outcomes. We studied their distributions directly: the probability mass function or probability density function was given, and we computed probabilities, expected values, and variances from it. But in practice, the random variable whose distribution we can specify is often not the one we actually need.
Consider a few examples. A physicist measures the speed $V$ of a gas molecule, which follows a distribution related to the Maxwell–Boltzmann distribution. The kinetic energy is $E = \frac{1}{2}mV^2$. The physicist needs the distribution of $E$, not $V$. An electrical engineer models the voltage $V$ across a resistor as a random variable. The power dissipated is $P = V^2/R$. The engineer needs the distribution of $P$. A statistician models the lifetime $T$ of a component as an exponential random variable. The log-lifetime $W = \ln T$ is more symmetric and easier to work with in certain analyses. In each case, a new random variable $Y = g(X)$ is defined as a function of a known random variable $X$, and the question is how to find the distribution of $Y$.
The probability that $Y$ falls in some set $B$ equals the probability that $X$ falls in the set of all $x$-values that $g$ maps into $B$:
$$P(Y \in B) = P(X \in g^{-1}(B))$$
where $g^{-1}(B) = \{x : g(x) \in B\}$ is the preimage of $B$ under $g$.
The CDF of $Y$ can always be written in terms of the CDF or PDF of $X$, because
$$F_Y(y) = P(Y \leq y) = P(g(X) \leq y).$$
Differentiating $F_Y(y)$ with respect to $y$, when possible, gives the PDF of $Y$.
There is no single method that works for every $g$ and every type of $X$. Section 6.2 handles discrete random variables directly. Section 6.3 uses the CDF method for continuous random variables. Section 6.4 develops the change-of-variables method using derivatives. Section 6.5 extends these ideas to two-variable transformations.
When $X$ is a discrete random variable, transforming it is conceptually straightforward. If $Y = g(X)$, then for each possible value $y$ that $Y$ can take:
$$p_Y(y) = P(Y = y) = P(g(X) = y) = \sum_{\{x\,:\,g(x)=y\}} p_X(x)$$
To find the probability that $Y = y$, add up the probabilities of all values of $X$ that $g$ maps to $y$. This method requires no calculus — only careful bookkeeping over the preimage.
p_Y(0) = P(X=0) = 1/5
p_Y(1) = P(X=-1) + P(X=1) = 1/5 + 1/5 = 2/5
p_Y(4) = P(X=-2) + P(X=2) = 1/5 + 1/5 = 2/5
p_Y(y) = P(n - X = y) = P(X = n - y) = \binom{n}{n-y} p^{n-y}(1-p)^y
Since $\binom{n}{n-y} = \binom{n}{y}$:
p_Y(y) = \binom{n}{y}(1-p)^y p^{n-y}
The events $\{X = x_1\}$ and $\{X = x_2\}$ are mutually exclusive when $x_1 \neq x_2$, so $P(Y = y) = P\!\left(\bigcup_{\{x: g(x)=y\}} \{X=x\}\right) = \sum_{\{x: g(x)=y\}} P(X = x)$. No approximation is involved — the direct method is exact.
Let $X$ be a continuous random variable with PDF $f_X(x)$ and CDF $F_X(x)$. Define $Y = g(X)$. To find the PDF of $Y$:
Step 1. Express $F_Y(y) = P(Y \leq y) = P(g(X) \leq y)$ in terms of $F_X$ or as an integral of $f_X$. The key is translating $\{g(X) \leq y\}$ into an equivalent condition on $X$.
Step 2. Differentiate: $f_Y(y) = \frac{d}{dy} F_Y(y)$, wherever the derivative exists.
F_Y(y) = P(Y ≤ y) = P(√X ≤ y) = P(X ≤ y²) = F_X(y²) = y²
f_Y(y) = d/dy [y²] = 2y, 0 < y < 1F_Y(y) = P(Y ≤ y) = P(e^X ≤ y) = P(X ≤ ln y) = Φ((ln y − μ)/σ)
where $\Phi$ is the standard normal CDF.
f_Y(y) = φ((ln y − μ)/σ) · 1/(σy)
Substituting the standard normal PDF $\phi(z) = \frac{1}{\sqrt{2\pi}}e^{-z^2/2}$:
f_Y(y) = 1/(yσ√(2π)) · exp(−(ln y − μ)²/(2σ²)), y > 0
F_Y(y) = P(1 − e^{−λX} ≤ y) = P(e^{−λX} ≥ 1−y) = P(X ≤ −ln(1−y)/λ)
= F_X(−ln(1−y)/λ) = 1 − e^{ln(1−y)} = y
f_Y(y) = d/dy [y] = 1, 0 < y < 1The result that $F_X(X) \sim \text{Uniform}(0,1)$ is fundamental to simulation: to generate a random variable $X$ with CDF $F_X$, generate $U \sim \text{Uniform}(0,1)$ and set $X = F_X^{-1}(U)$. This is the inverse transform method used in every statistical computing package.
Let $X$ be a continuous random variable with PDF $f_X(x)$ supported on an interval $I$. Let $g: I \to \mathbb{R}$ be a strictly monotone, differentiable function, and define $Y = g(X)$. Then $Y$ has PDF
$$f_Y(y) = f_X\!\left(g^{-1}(y)\right) \cdot \left|\frac{d}{dy} g^{-1}(y)\right|$$
for $y$ in the range of $g$. The term $|dg^{-1}/dy|$ is the Jacobian of the transformation. It corrects for how the transformation stretches or compresses probability, exactly as the substitution rule corrects for stretching in a change-of-variable integral.
When $g$ is strictly increasing, $F_Y(y) = F_X(g^{-1}(y))$, and differentiating by the chain rule gives the formula directly. When $g$ is strictly decreasing, $F_Y(y) = 1 - F_X(g^{-1}(y))$, and differentiating introduces a negative sign that is cancelled by the absolute value. Both cases are unified by the absolute value in the formula.
g⁻¹(y) = y^(1/3), d/dy g⁻¹(y) = (1/3)y^(−2/3)f_Y(y) = f_X(y^(1/3)) · (1/3)y^(−2/3) = 1 · (1/3)y^(−2/3) = (1/3)y^(−2/3), 0 < y < 1d/dy g⁻¹(y) = −e^(−y), |d/dy g⁻¹(y)| = e^(−y)
f_Y(y) = f_X(e^(−y)) · e^(−y) = e^(−e^(−y)) · e^(−y), y ∈ ℝSuppose the support of $X$ can be partitioned into intervals $I_1, I_2, \ldots, I_k$ on each of which $g$ is strictly monotone. Let $g_j^{-1}$ denote the inverse of $g$ on $I_j$. Then
$$f_Y(y) = \sum_{j=1}^{k} f_X\!\left(g_j^{-1}(y)\right) \cdot \left|\frac{d}{dy} g_j^{-1}(y)\right|.$$
f_Y(y) = 2 · (1/√(2π)) e^(−y/2) · 1/(2√y) = (1/√(2πy)) e^(−y/2), y > 0
The single-branch change-of-variables formula requires that $g$ is strictly monotone on the entire support of $X$. Applying it to $g(x) = x^2$ on all of $\mathbb{R}$ without splitting into two branches is one of the most common errors in transformation problems. Always partition the support if $g$ changes direction.
In many applications, we transform two random variables simultaneously. If $(X, Y)$ is a pair of random variables with a known joint distribution, and we define $U = g(X,Y)$ and $V = h(X,Y)$, what is the joint distribution of $(U,V)$?
When the transformation $(u,v) = (g(x,y), h(x,y))$ is one-to-one and differentiable, with inverse $x = x(u,v)$ and $y = y(u,v)$:
$$f_{U,V}(u,v) = f_{X,Y}(x(u,v),\, y(u,v)) \cdot |J|$$
where $|J|$ is the absolute value of the Jacobian determinant of the inverse transformation:
$$J = \frac{\partial(x,y)}{\partial(u,v)} = \begin{vmatrix} \partial x/\partial u & \partial x/\partial v \\ \partial y/\partial u & \partial y/\partial v \end{vmatrix} = \frac{\partial x}{\partial u}\frac{\partial y}{\partial v} - \frac{\partial x}{\partial v}\frac{\partial y}{\partial u}.$$
J = |1/2 · (−1/2) − (1/2) · 1/2| = |−1/4 − 1/4| = 1/2
f_{U,V}(u,v) = 1 · (1/2) = 1/2f_{X,Y}(x,y) = (1/2π) e^(−(x²+y²)/2)f_{R,Θ}(r,θ) = (1/2π) e^(−r²/2) · r = (r e^(−r²/2)) · (1/2π)
This factors into $f_R(r) = r e^{-r^2/2}$ (Rayleigh distribution) and $f_\Theta(\theta) = 1/(2\pi)$ (Uniform on $(0,2\pi)$). So $R$ and $\Theta$ are independent.
Let $X$ be a random variable and let $g$ be a function such that $E[g(X)]$ exists.
If $X$ is discrete with PMF $p_X(x)$: $E[g(X)] = \displaystyle\sum_{x} g(x)\, p_X(x)$
If $X$ is continuous with PDF $f_X(x)$: $E[g(X)] = \displaystyle\int_{-\infty}^{\infty} g(x)\, f_X(x)\, dx$
In either case, we compute $E[g(X)]$ by averaging $g(x)$ with respect to the distribution of $X$, without needing to find the distribution of $g(X)$ first.
The name "unconscious statistician" refers to the fact that students sometimes apply this formula without realizing it requires justification. It is true, and a careful proof uses the definition of expected value and properties of probability. LOTUS allows us to compute $E[X^2]$, $E[\sqrt{X}]$, $E[e^{tX}]$, and any other expected value of a function directly from the original distribution.
E[X²] = ∫₀^∞ x² · λe^(−λx) dx = λ · (2/λ³) = 2/λ²E[e^(tX)] = e^(t·0)(1−p) + e^(t·1)·p = (1−p) + pe^tFor two random variables $X$ and $Y$ with joint PDF $f_{X,Y}(x,y)$: $E[g(X,Y)] = \int\!\!\int g(x,y)\, f_{X,Y}(x,y)\, dx\, dy$. The covariance formula from Chapter 5, $\text{Cov}(X,Y) = E[(X-\mu_X)(Y-\mu_Y)]$, is exactly LOTUS applied to $g(x,y) = (x-\mu_X)(y-\mu_Y)$.
For a convex function $g$, Jensen's inequality says $E[g(X)] \geq g(E[X])$, with equality only when $X$ is constant. A common error is writing $E[1/X] = 1/E[X]$ or $E[X^2] = (E[X])^2$. Both are wrong. Use LOTUS to compute $E[g(X)]$ directly. Numerical example: if $X \sim \text{Uniform}(1,3)$, then $E[X] = 2$ but $E[1/X] = \frac{1}{2}\ln 3 \approx 0.549 \neq 0.5 = 1/E[X]$.
The word moment comes from physics, where the moment of a force measures its rotational effect. In probability, moments measure the shape of a distribution quantitatively and systematically.
The $k$-th moment of $X$ is $\mu_k' = E[X^k]$, $k = 1, 2, 3, \ldots$
The $k$-th central moment of $X$ is $\mu_k = E[(X - \mu)^k]$, where $\mu = E[X]$.
The first moment $\mu_1' = E[X]$ is the mean: center of the distribution. The second central moment $\mu_2 = \text{Var}(X)$: spread. Higher moments describe shape in finer detail.
| Moment | Formula | What It Measures | Normal Value |
|---|---|---|---|
| Mean | $E[X]$ | Center of the distribution | $\mu$ |
| Variance | $E[(X-\mu)^2]$ | Spread around the mean | $\sigma^2$ |
| Skewness | $E[(X-\mu)^3]/\sigma^3$ | Asymmetry; positive = right tail | $0$ |
| Kurtosis | $E[(X-\mu)^4]/\sigma^4$ | Tail weight; leptokurtic > 3 | $3$ |
Skewness measures asymmetry. If the distribution is symmetric about its mean, $\mu_3 = 0$ because positive and negative deviations cancel. A long right tail gives positive skewness; a long left tail gives negative skewness. Dividing by $\sigma^3$ makes skewness unitless so it can be compared across distributions of different scales.
$$\text{Skewness} = \frac{E[(X-\mu)^3]}{\sigma^3}$$
Kurtosis $= E[(X-\mu)^4]/\sigma^4$ measures tail weight relative to the center. The normal distribution has kurtosis exactly 3. Distributions with heavier tails than the normal (leptokurtic) have kurtosis greater than 3. Distributions with lighter tails (platykurtic) have kurtosis less than 3. Excess kurtosis = kurtosis $-$ 3 centers the comparison at the normal.
E[X] = 1/λ, E[X²] = 2/λ², E[X³] = 6/λ³, E[X⁴] = 24/λ⁴
Var(X) = E[X²] − (E[X])² = 2/λ² − 1/λ² = 1/λ²Skewness = μ₃/σ³ = (2/λ³)/(1/λ³) = 2
The exponential always has skewness exactly 2, regardless of $\lambda$ — confirming its long right tail.
If we know all the moments $E[X], E[X^2], E[X^3], \ldots$, can we recover the distribution? In many important cases, yes — the sequence of moments uniquely determines the distribution. This motivates the moment generating function of Section 6.8, which packages all the moments into a single analytic function.
The moment generating function (MGF) of a random variable $X$ is
$$M_X(t) = E\!\left[e^{tX}\right]$$
for all values of $t$ in an open interval around 0 where this expectation exists and is finite.
For discrete $X$: $M_X(t) = \sum_x e^{tx} p_X(x)$. For continuous $X$: $M_X(t) = \int_{-\infty}^{\infty} e^{tx} f_X(x)\, dx$.
The MGF is a function of $t$, not of $x$. The random variable $X$ lives in the expectation; $t$ is a parameter we use to extract information.
The Taylor expansion of $e^{tX}$ reveals the connection to moments:
$$e^{tX} = \sum_{k=0}^{\infty} \frac{(tX)^k}{k!} \quad \Longrightarrow \quad M_X(t) = E[e^{tX}] = \sum_{k=0}^{\infty} \frac{t^k}{k!} E[X^k] = \sum_{k=0}^{\infty} \frac{t^k}{k!} \mu_k'.$$
The MGF is a generating function for the moments: the $k$-th moment appears as the coefficient of $t^k/k!$ in the Taylor series of $M_X(t)$. Differentiating $k$ times and evaluating at $t=0$ recovers $\mu_k'$.
| Distribution | Parameter(s) | $M_X(t)$ | Domain of $t$ |
|---|---|---|---|
| Bernoulli$(p)$ | $p$ | $(1-p) + pe^t$ | All $t$ |
| Binomial$(n,p)$ | $n, p$ | $((1-p) + pe^t)^n$ | All $t$ |
| Geometric$(p)$ | $p$ | $pe^t / (1-(1-p)e^t)$ | $t < -\ln(1-p)$ |
| Poisson$(\lambda)$ | $\lambda$ | $e^{\lambda(e^t - 1)}$ | All $t$ |
| Uniform$(a,b)$ | $a, b$ | $(e^{tb} - e^{ta}) / (t(b-a))$ | $t \neq 0$ |
| Exponential$(\lambda)$ | $\lambda$ | $\lambda / (\lambda - t)$ | $t < \lambda$ |
| Normal$(\mu, \sigma^2)$ | $\mu, \sigma^2$ | $e^{\mu t + \sigma^2 t^2/2}$ | All $t$ |
| Gamma$(\alpha, \beta)$ | $\alpha, \beta$ | $(\beta/(\beta-t))^\alpha$ | $t < \beta$ |
M_X(t) = Σ_{k=0}^∞ e^(tk) · e^(−λ)λ^k/k! = e^(−λ) Σ_{k=0}^∞ (e^t λ)^k/k!
= e^(−λ) · e^(λe^t) = e^(λ(e^t − 1))
= −(x − (μ + σ²t))²/(2σ²) + μt + σ²t²/2
M_X(t) = e^(μt + σ²t²/2) · 1 = e^(μt + σ²t²/2)
If two random variables $X$ and $Y$ have the same MGF — $M_X(t) = M_Y(t)$ for all $t$ in some open interval around 0 — then $X$ and $Y$ have the same distribution.
This means: if we compute the MGF of a random variable and recognize it as the MGF of a known distribution, we have identified the distribution exactly. This is a powerful shortcut that the next two sections will use repeatedly.
If $M_X(t)$ exists in an open interval around $t = 0$, then for each positive integer $k$:
$$E[X^k] = M_X^{(k)}(0)$$
where $M_X^{(k)}(0)$ denotes the $k$-th derivative of $M_X(t)$ evaluated at $t = 0$. This follows directly from differentiating the Taylor series $M_X(t) = \sum_{k=0}^\infty \frac{t^k}{k!} E[X^k]$ term by term.
M_X'(t) = e^(λ(e^t−1)) · λe^t
At $t=0$: $M_X'(0) = 1 \cdot \lambda \cdot 1 = \lambda$. So $E[X] = \lambda$. ✓
M_X''(t) = λe^t e^(λ(e^t−1)) + λ²e^(2t) e^(λ(e^t−1))
At $t=0$: $M_X''(0) = \lambda + \lambda^2$. So $E[X^2] = \lambda + \lambda^2$.
Var(X) = E[X²] − (E[X])² = (λ + λ²) − λ² = λ ✓M_X'(t) = (μ + σ²t) e^(μt + σ²t²/2)
At $t=0$: $M_X'(0) = \mu$. So $E[X] = \mu$. ✓
M_X''(t) = σ² e^(μt + σ²t²/2) + (μ + σ²t)² e^(μt + σ²t²/2)
At $t=0$: $M_X''(0) = \sigma^2 + \mu^2$. So $E[X^2] = \mu^2 + \sigma^2$.
Var(X) = E[X²] − (E[X])² = (μ² + σ²) − μ² = σ² ✓M_X'(t) = λ(λ−t)^(−2) → M_X'(0) = 1/λ
M_X''(t) = 2λ(λ−t)^(−3) → M_X''(0) = 2/λ²
In general: $M_X^{(k)}(0) = k!/\lambda^k$, confirming the formula derived in Section 6.7.
If $X_1, X_2, \ldots, X_n$ are independent random variables, then
$$M_{X_1 + X_2 + \cdots + X_n}(t) = M_{X_1}(t) \cdot M_{X_2}(t) \cdots M_{X_n}(t).$$
Proof: By independence, $e^{tX_1}, \ldots, e^{tX_n}$ are independent, so $E[e^{t(X_1+\cdots+X_n)}] = E[e^{tX_1}] \cdots E[e^{tX_n}]$. $\square$
Combined with the uniqueness theorem, this converts the problem of finding the distribution of a sum into simple multiplication of MGFs.
M_S(t) = M_{X₁}(t) · M_{X₂}(t) = e^(λ₁(e^t−1)) · e^(λ₂(e^t−1)) = e^((λ₁+λ₂)(e^t−1))
M_{X₁+X₂}(t) = e^(μ₁t + σ₁²t²/2) · e^(μ₂t + σ₂²t²/2) = e^((μ₁+μ₂)t + (σ₁²+σ₂²)t²/2)
M_S(t) = (λ/(λ−t))^n$$M_{aX+b}(t) = E[e^{t(aX+b)}] = e^{tb} E[e^{(at)X}] = e^{tb} M_X(at).$$
Application: The standardization $Z = (X-\mu)/\sigma$ of a Normal$(\mu, \sigma^2)$ variable has $M_Z(t) = e^{-\mu t/\sigma} M_X(t/\sigma) = e^{t^2/2}$, which is the MGF of Standard Normal$(0,1)$. The standardization of any normal is standard normal — confirmed by the MGF.
The factorization $M_{X+Y}(t) = M_X(t) M_Y(t)$ requires that $X$ and $Y$ be independent. If they are correlated, the MGF of the sum is not the product of the MGFs. Always verify independence before applying the product theorem.
Moment generating functions are powerful, but they have one significant limitation: they may not exist. The MGF is $E[e^{tX}]$, and for some distributions this expectation is infinite for all $t \neq 0$. The Cauchy distribution, for example, has no finite mean or variance. Its MGF does not exist for any $t \neq 0$. Yet the Cauchy distribution is perfectly well-defined as a probability model, and we would like an analogous tool to study it.
The characteristic function of a random variable $X$ is
$$\varphi_X(t) = E\!\left[e^{itX}\right] = E[\cos(tX)] + i\, E[\sin(tX)]$$
where $i = \sqrt{-1}$ and $t \in \mathbb{R}$. Since $|e^{itX}| = 1$ for all $X$ and $t$, $\varphi_X(t)$ always exists and is finite for every distribution — no convergence concerns. When the MGF exists, $\varphi_X(t) = M_X(it)$.
The characteristic function satisfies the same uniqueness and product theorems as the MGF:
| Property | MGF | Characteristic Function |
|---|---|---|
| Uniqueness | $M_X(t) = M_Y(t)$ ⟹ same distribution | $\varphi_X(t) = \varphi_Y(t)$ ⟹ same distribution |
| Sum of independent | $M_{X+Y}(t) = M_X(t) M_Y(t)$ | $\varphi_{X+Y}(t) = \varphi_X(t)\varphi_Y(t)$ |
| Existence | May not exist (heavy tails) | Always exists for every distribution |
| Domain | Real $t$ near 0 | All real $t$, complex values |
φ_X(t) = M_X(it) = e^(it)²/2 = e^(−t²/2)
Characteristic functions are fundamental in rigorous proofs of the central limit theorem. If $S_n = X_1 + \cdots + X_n$ is a standardized sum of $n$ i.i.d. random variables, then $\varphi_{S_n/\sqrt{n}}(t) = [\varphi_X(t/\sqrt{n})]^n$. Expanding in a Taylor series in $t/\sqrt{n}$ and taking $n \to \infty$ yields $e^{-t^2/2}$, the characteristic function of the standard normal. The central limit theorem follows. Chapter 7 will develop this intuition fully.
In statistics, measurements are often nonlinear functions of estimated quantities. If $\bar{X}$ is the sample mean of $n$ observations from a distribution with mean $\mu$ and variance $\sigma^2$, then for large $n$, $\bar{X}$ is approximately normally distributed. But if we are interested in $g(\bar{X})$ — say, $\bar{X}^2$ or $1/\bar{X}$ — what is its approximate distribution? The delta method provides an answer using a first-order Taylor expansion.
If $g$ is differentiable at $\mu$ and $\bar{X}$ has $E[\bar{X}] = \mu$ and $\text{Var}(\bar{X}) = \sigma^2/n$, then
$$\text{Var}(g(\bar{X})) \approx [g'(\mu)]^2 \cdot \frac{\sigma^2}{n}.$$
This follows from the linearization $g(\bar{X}) \approx g(\mu) + g'(\mu)(\bar{X} - \mu)$ and the variance of a linear function.
Var(√X̄) ≈ (1/4)² · 0.1 = (1/16) · 0.1 = 0.00625Many natural measurements — incomes, species abundances, stock prices — span several orders of magnitude. They often follow skewed, right-tailed distributions. Log transformation compresses the upper tail, reduces skewness, and often makes the distribution approximately normal.
If $X$ is lognormal (so $\ln X \sim \text{Normal}(\mu, \sigma^2)$), we can derive the moments of $X$ using LOTUS with the MGF of the normal. Let $Y = \ln X$, so $X = e^Y$:
$$E[X] = E[e^Y] = M_Y(1) = e^{\mu + \sigma^2/2}.$$
$$\text{Var}(X) = e^{2\mu + \sigma^2}(e^{\sigma^2} - 1).$$
These formulas show how transformation methods connect the moments of a transformed variable to the MGF of the original. Note that $E[X] > e^\mu$: the mean exceeds the median ($e^\mu$) because the lognormal is right-skewed.
The variance of a random variable often depends on its mean. The Poisson distribution has $\text{Var}(X) = E[X] = \lambda$. If we observe Poisson counts with different means $\lambda_1, \lambda_2, \ldots$, their variances differ, complicating analysis. The delta method suggests finding a transformation $g$ such that $[g'(\lambda)]^2 \cdot \lambda \approx c$ for all $\lambda$, where $c$ is constant.
Solving: $g'(\lambda) = c/\sqrt{\lambda}$, giving $g(\lambda) = 2c\sqrt{\lambda}$. The square root transformation $g(X) = \sqrt{X}$ approximately stabilizes the variance of Poisson counts. After applying $g$, all transformed observations have approximately the same variance, simplifying comparison and modeling.
P(S_n > t) = 1 − F_{Gamma(n,λ)}(t)
evaluated from the incomplete gamma function. Engineers use this to compute reliability requirements.
Fix: When $g(x) = x^2$ is applied to $X \in \mathbb{R}$, $g$ is decreasing on $(-\infty, 0)$ and increasing on $(0, \infty)$. Apply the formula on each piece and sum the results. The correct PDF is $f_Y(y) = 2 \cdot f_X(\sqrt{y}) \cdot \frac{1}{2\sqrt{y}} = f_X(\sqrt{y})/\sqrt{y}$, giving the chi-squared distribution. The one-branch answer integrates to $1/2$, not $1$.
Fix: Always determine the range of $Y = g(X)$ from the range of $X$ and the function $g$, and explicitly write it. The complete answer is $f_Y(y) = 2y$ for $0 < y < 1$ (and 0 otherwise). Without the support, a PDF is incomplete and may integrate to a value other than 1.
Fix: These equalities are false unless $X$ is constant. Jensen's inequality says $E[g(X)] \geq g(E[X])$ for convex $g$. Always use LOTUS to compute $E[g(X)] = \int g(x) f_X(x)\,dx$ directly. Numerical counterexample: for $X \sim \text{Uniform}(1,3)$, $E[X] = 2$ but $E[1/X] = \frac{1}{2}\ln 3 \approx 0.549 \neq 0.5$.
Fix: The Jacobian $|dg^{-1}/dy|$ is always required. It corrects for how the transformation stretches probability. Omitting it produces a function that has the right shape but does not integrate to 1 and is not a valid PDF. Always write and compute the Jacobian term explicitly.
Fix: $M_X(2)$ is just the value of the MGF at $t = 2$; it has no direct meaning as a moment. The $k$-th moment is $M_X^{(k)}(0)$ — the $k$-th derivative evaluated at $t = 0$. For Exponential$(4)$: $M_X''(t) = 8(4-t)^{-3}$, so $E[X^2] = M_X''(0) = 8/64 = 1/8$. Check: $E[X^2] = 2/\lambda^2 = 2/16 = 1/8$. ✓
Fix: The product rule holds only for independent random variables. If $X$ and $Y$ are correlated, $M_{X+Y}(t) \neq M_X(t)M_Y(t)$ in general. Verify independence — either from the problem statement or by checking that the joint distribution factors as a product of the marginals — before applying the theorem.
Fix: The inverse of $y = e^x$ is $x = \ln y$, not $x = e^{-y}$. Always verify the inverse by checking that $g(g^{-1}(y)) = y$. Check: $e^{\ln y} = y$. ✓ Compare: $e^{e^{-y}} \neq y$ in general. A sign or exponent error in the inverse function propagates through the entire transformation calculation.
Fix: The exponential MGF requires $t < \lambda$ for the integral to converge. Evaluating the formula at $t \geq \lambda$ gives a negative or infinite result with no probabilistic meaning. Always state and check the domain of the MGF before using it. For the normal, the domain is all $t$; for the exponential, $t < \lambda$; for the gamma Gamma$(\alpha, \beta)$, $t < \beta$.
Chapter 7 takes the tools developed in Chapters 3 through 6 and uses them to study what happens when we observe a large number of independent values and ask about the behavior of their sum or average. This leads to the law of large numbers and the central limit theorem — the most important results in probability for all of statistics.
The MGF product theorem is the technical backbone of the central limit theorem. When $S_n = X_1 + \cdots + X_n$ is a standardized sum of $n$ independent, identically distributed random variables, its MGF is $[M_X(t/\sqrt{n})]^n e^{-\mu t\sqrt{n}/\sigma}$. Taylor expanding $M_X(t/\sqrt{n})$ around 0 and tracking carefully:
Let $X_1, X_2, \ldots$ be i.i.d. with mean $\mu$ and variance $\sigma^2$. Define $Z_n = (X_1 + \cdots + X_n - n\mu)/(\sigma\sqrt{n})$. Then
$$M_{Z_n}(t) = \left[M_X\!\left(\frac{t}{\sigma\sqrt{n}}\right)\right]^n e^{-\mu t\sqrt{n}/\sigma} \;\longrightarrow\; e^{t^2/2} \quad \text{as } n \to \infty.$$
Since $e^{t^2/2}$ is the MGF of Normal$(0,1)$, and MGFs uniquely determine distributions, $Z_n \to \text{Normal}(0,1)$ in distribution. This is the central limit theorem. Chapter 7 makes it precise.
The transformation methods of Sections 6.3 and 6.4 will also be used in Chapter 7 to derive sampling distributions: the chi-squared distribution (from squaring standard normals), the $t$-distribution (from the ratio of a normal and a chi-squared), and the $F$-distribution (from ratios of chi-squareds). These are the distributions underlying the inference procedures in Chapters 8 and 9.
The concept of moments supports the statement of the CLT: the theorem applies whenever the mean and variance of the summands are finite — the first two moments determine whether the theorem works. LOTUS will appear in Chapter 8 to compute the bias and variance of estimators. The techniques of this chapter are not isolated tools; they are the foundation of the mathematical structure that makes probability theory rigorous and statistics theoretically grounded.