The Central Idea

Monte Carlo methods replace difficult direct calculation with repeated randomized experiments.

The basic pattern is

\[ \boxed{ \text{random sampling} \longrightarrow \text{repeated model runs} \longrightarrow \text{statistical estimate} }. \]

As the number of simulations increases, sample-based estimates can often approach the underlying probability or expectation of interest.

What Is Monte Carlo Simulation?

Monte Carlo simulation is a computational technique that uses random or pseudorandom samples to estimate numerical quantities, probability distributions, or uncertain model outcomes.

A Monte Carlo model commonly involves:

Why Use Random Sampling?

Some mathematical problems are difficult because:

Monte Carlo methods approximate the answer by sampling many representative cases.

A Single Trial

A trial is one randomized experiment or one run of the simulation.

If the model produces output

\[ Y_r \]

on trial \(r\), then a Monte Carlo experiment with \(R\) trials produces

\[ Y_1,Y_2,\ldots,Y_R. \]

These outputs form a sample from the modeled outcome process.

Estimating an Expected Value

Suppose \(Y\) is a random output whose expected value is

\[ E[Y]. \]

After \(R\) simulated trials, the Monte Carlo estimate is

\[ \boxed{ \hat{\mu} = \frac{1}{R} \sum_{r=1}^{R}Y_r }. \]

The sample mean \(\hat{\mu}\) estimates the theoretical mean.

Estimating a Probability

Suppose we want the probability of an event \(A\).

Let

\[ I_r= \begin{cases} 1, & \text{if event }A\text{ occurs on trial }r,\\ 0, & \text{otherwise}. \end{cases} \]

Then

\[ \boxed{ \hat{P}(A) = \frac{1}{R} \sum_{r=1}^{R}I_r }. \]

This is simply the fraction of trials in which the event occurs.

Example: Coin Toss Simulation

Suppose a fair coin is tossed \(1000\) times in a simulation.

If heads occurs \(487\) times, then the estimated probability of heads is

\[ \hat{P}(\text{heads}) = \frac{487}{1000} = 0.487. \]

The exact probability is \(0.5\), but a finite simulation need not produce exactly \(500\) heads.

Sampling Error

Monte Carlo estimates vary because they are based on finite random samples.

This variability is called sampling error.

Two simulations using the same probability model may produce slightly different estimates:

\[ 0.487,\qquad 0.503,\qquad 0.495. \]

Such variation is expected and does not by itself indicate an implementation problem.

The Law of Large Numbers

Under appropriate conditions, the law of large numbers tells us that the sample mean approaches the expected value as the number of trials increases.

Symbolically,

\[ \frac{1}{R}\sum_{r=1}^{R}Y_r \longrightarrow E[Y] \]

as

\[ R\to\infty. \]

This principle provides much of the theoretical foundation for Monte Carlo estimation.

More Trials Usually Improve Precision

Increasing the number of trials generally reduces random estimation noise.

A rough and important Monte Carlo relationship is

\[ \boxed{ \text{standard error} \propto \frac{1}{\sqrt{R}} }. \]

This means that reducing Monte Carlo noise substantially can require a large increase in the number of trials.

The Square-Root Cost

If standard error behaves approximately like

\[ \frac{1}{\sqrt{R}}, \]

then doubling the number of trials does not cut the error in half.

To reduce the standard error by a factor of two, one generally needs about

\[ 2^2=4 \]

times as many trials.

Random Numbers in Computation

Computers usually generate pseudorandom numbers rather than truly random physical values.

A pseudorandom generator produces a deterministic sequence that is designed to behave statistically like random sampling for the intended application.

Monte Carlo simulation depends on the quality and appropriate use of these generated sequences.

Random Seeds

A random seed initializes a pseudorandom number generator.

Using the same seed can reproduce the same sequence of pseudorandom numbers.

This is useful for:

Uniform Random Sampling

A common primitive is a random variable

\[ U\sim\operatorname{Uniform}(0,1). \]

This means values are sampled from the interval \([0,1]\) with constant density.

Uniform random numbers can be transformed or used in decision rules to generate samples from many other distributions.

Simulating a Bernoulli Event

Suppose an event has probability \(p\).

Generate

\[ U\sim\operatorname{Uniform}(0,1). \]

Declare the event to occur if

\[ U Then the event occurs with probability \(p\).

Example: Failure Probability

Suppose a component has a \(2\%\) probability of failure during a specified period.

Let

\[ p=0.02. \]

For each simulated component, generate \(U\in[0,1]\). Record failure when

\[ U<0.02. \]

Repeating this process many times estimates the distribution of total failures.

Sampling from Other Distributions

Model inputs may follow distributions such as:

The choice of distribution should reflect the model's assumptions and evidence.

Monte Carlo with Several Random Inputs

Suppose model output is

\[ Y=f(X_1,X_2,\ldots,X_k), \]

where the \(X_i\) are uncertain inputs.

One Monte Carlo trial samples values

\[ x_1,x_2,\ldots,x_k \]

and computes

\[ y=f(x_1,x_2,\ldots,x_k). \]

Repeating the process generates an empirical distribution for \(Y\).

Dependence Between Inputs

Random inputs are not always independent.

If two variables are correlated or constrained together, sampling them independently may create unrealistic scenarios.

A correct Monte Carlo model should preserve important dependence structures among inputs when those dependencies are part of the modeled system.

Output Distributions

Monte Carlo simulation produces more than a single average.

The output sample

\[ Y_1,Y_2,\ldots,Y_R \]

can be used to study:

Estimating Variance

If the simulated outputs are \(Y_1,\ldots,Y_R\), the sample variance is

\[ s^2 = \frac{1}{R-1} \sum_{r=1}^{R} (Y_r-\bar{Y})^2. \]

This measures the spread of the simulated outcomes around their sample mean.

Quantiles

A quantile identifies an output level below which a specified fraction of simulated outcomes falls.

For example, the \(95\)th percentile is a value \(q_{0.95}\) such that approximately

\[ 95\% \]

of simulated outcomes are at or below that value.

Quantiles are especially useful for risk and uncertainty analysis.

Tail Probabilities

Suppose a threshold \(L\) is important.

The probability of exceeding that threshold can be estimated by

\[ \hat{P}(Y>L) = \frac{\text{number of trials with }Y_r>L}{R}. \]

Monte Carlo methods are often used to estimate rare or costly events, although very rare events may require specialized sampling methods.

Monte Carlo Integration

Random sampling can also estimate integrals.

If

\[ U\sim\operatorname{Uniform}(a,b), \]

then

\[ E[f(U)] = \frac{1}{b-a} \int_a^b f(x)\,dx. \]

Therefore,

\[ \boxed{ \int_a^b f(x)\,dx \approx (b-a)\frac{1}{R} \sum_{r=1}^{R}f(U_r) }. \]

Example: Estimating an Integral

Consider

\[ \int_0^1 x^2\,dx. \]

Generate

\[ U_1,\ldots,U_R \sim\operatorname{Uniform}(0,1). \]

Then estimate

\[ \int_0^1x^2\,dx \approx \frac{1}{R} \sum_{r=1}^{R}U_r^2. \]

The exact value is

\[ \frac{1}{3}, \]

so the simulation can be checked against a known result.

Geometric Probability

Monte Carlo simulation can estimate areas and volumes through geometric sampling.

If random points are sampled uniformly from a region of known area \(A\), and a fraction \(p\) lands inside a subregion, then the subregion's area is estimated by

\[ \boxed{ A_{\text{sub}} \approx pA }. \]

Estimating \(\pi\)

Consider the unit square

\[ 0\le x\le1,\qquad 0\le y\le1. \]

The quarter-circle

\[ x^2+y^2\le1 \]

has area

\[ \frac{\pi}{4}. \]

If \(M\) of \(R\) uniformly sampled points fall inside the quarter-circle, then

\[ \frac{M}{R} \approx \frac{\pi}{4}. \]

Therefore,

\[ \boxed{ \pi\approx4\frac{M}{R} }. \]

Monte Carlo Pseudocode

SET total = 0

REPEAT R times
    SAMPLE random inputs
    RUN model
    RECORD output Y
    ADD Y to total
END REPEAT

COMPUTE mean = total / R
REPORT summary statistics

More detailed implementations may also store every output for distributional analysis.

Scenario Simulation

Monte Carlo methods are useful when several future or uncertain quantities affect a result.

A trial may represent one possible scenario. Repeating the simulation creates a collection of plausible modeled scenarios rather than one fixed forecast.

This approach is common in:

Risk as a Distribution

A deterministic model may report one value.

A Monte Carlo risk model may instead report an entire distribution:

\[ Y\sim\text{modeled outcome distribution}. \]

This allows decision makers to ask not only What is the average? but also:

Monte Carlo and Agent-Based Models

Agent-based models often contain stochastic behavior, so they are frequently analyzed through repeated Monte Carlo runs.

The ABM defines one possible trajectory for a given sequence of random events. Monte Carlo repetition studies the distribution of trajectories and outcomes over many such sequences.

Monte Carlo and Parameter Uncertainty

Parameters themselves may be uncertain.

Instead of fixing

\[ \theta=\theta_0, \]

a model may sample

\[ \theta\sim p(\theta). \]

This propagates parameter uncertainty through the model to the output distribution.

Monte Carlo and Measurement Uncertainty

Observed inputs may contain measurement error.

If a measured quantity is represented as

\[ X=x_{\text{obs}}+\varepsilon, \]

where \(\varepsilon\) is a modeled error term, repeated sampling of \(\varepsilon\) can show how measurement uncertainty affects downstream results.

Confidence in the Monte Carlo Estimate

A simulation result should distinguish variation in the modeled system from uncertainty caused by using a finite number of Monte Carlo trials.

The first is part of the model's output distribution. The second is numerical sampling uncertainty in our estimate of that distribution.

Increasing \(R\) reduces the second type but does not remove the first.

Replication

A useful computational check is to repeat the full Monte Carlo experiment using different random seeds.

If estimates change dramatically from one large simulation to another, the number of trials may be insufficient or the model may have a highly variable output.

Replication helps assess Monte Carlo stability.

Batch Means and Running Estimates

A running mean after \(r\) trials can be written

\[ \bar{Y}_r = \frac{1}{r} \sum_{j=1}^{r}Y_j. \]

Plotting or inspecting \(\bar{Y}_r\) as \(r\) increases can reveal whether the estimate appears to be stabilizing.

This is not a proof of convergence, but it is a useful diagnostic.

Variance Reduction

Because ordinary Monte Carlo convergence can be slow, specialized methods can reduce estimator variance without simply increasing the number of trials.

Examples include:

These methods are more advanced, but they follow the same objective: obtain more precise information from computational sampling.

Importance Sampling

Rare events may occur too infrequently under ordinary sampling to estimate them efficiently.

Importance sampling deliberately samples more often from regions that contribute strongly to the quantity of interest and then reweights the samples appropriately.

The method can greatly improve estimation of small probabilities when designed correctly.

Stratified Sampling

Stratified sampling divides the sampling domain into subregions or strata and samples within each.

This can ensure that important portions of the domain are represented instead of relying entirely on chance allocation of samples.

The stratum estimates are then combined using appropriate weights.

Quasi-Monte Carlo Ideas

Some numerical integration methods use carefully constructed low-discrepancy point sequences rather than ordinary pseudorandom samples.

These are called quasi-Monte Carlo methods.

They aim to cover the sampling domain more evenly and can improve numerical integration for some problems.

Common Error: Too Few Trials

A small number of trials can produce a noisy estimate that appears more precise than it really is.

For example, estimating a \(1\%\) event using only \(20\) trials is generally inadequate because the event may not appear at all.

Trial count should reflect the variability and rarity of the quantity being estimated.

Common Error: Treating Random Variation as Model Error

Two Monte Carlo runs can produce different results even when both implementations are correct.

The relevant question is whether the variation is consistent with the model's stochastic structure and finite-sample uncertainty.

Common Error: Using the Wrong Input Distribution

A Monte Carlo model is only as meaningful as its input assumptions.

Sampling a normal distribution when the real quantity is bounded, heavily skewed, discrete, or strongly asymmetric may distort the output.

Probability distributions should be justified by theory, evidence, or explicit modeling assumptions.

Common Error: Ignoring Dependence

Sampling correlated variables independently can create impossible or unrealistic combinations.

If fuel price, transportation cost, and operating expense are modeled as related, their joint behavior should not be discarded without justification.

Dependence structure can materially affect the output distribution.

Common Error: Reusing Random Samples Incorrectly

Random values should be reused only when the model intends the same underlying uncertainty to affect multiple quantities.

Accidentally using one random draw for unrelated events can introduce artificial dependence.

Common Error: Reporting Only the Mean

A mean can hide important asymmetry and tail risk.

Two output distributions may have the same mean but very different:

Monte Carlo results should be summarized with statistics relevant to the modeling question.

Common Error: Confusing Simulation Precision with Model Validity

Running ten million trials can produce a numerically precise estimate of the wrong model.

More trials reduce Monte Carlo sampling error, but they do not correct:

A Reliable Monte Carlo Workflow

  1. Define the quantity to estimate.
  2. Identify all uncertain inputs.
  3. Assign appropriate probability models.
  4. Represent dependencies among inputs.
  5. Choose a random-number strategy and seed policy.
  6. Generate one trial and verify it manually where possible.
  7. Repeat the simulation for many trials.
  8. Store the outputs needed for analysis.
  9. Compute means, variances, quantiles, or event frequencies.
  10. Repeat with larger trial counts or independent seeds.
  11. Assess Monte Carlo sampling uncertainty.
  12. Interpret results in the context of model assumptions and validation.

Practice 1: Estimate a Probability

An event occurs in \(73\) of \(500\) Monte Carlo trials.

Estimate its probability.

Practice 2: Estimate a Mean

Four simulated outputs are

\[ 8,\ 11,\ 9,\ 12. \]

Compute the Monte Carlo sample mean.

Practice 3: Trial Growth

If the Monte Carlo standard error behaves like

\[ \frac{1}{\sqrt{R}}, \]

by what factor must the number of trials increase to reduce the standard error by a factor of \(3\)?

Practice 4: Bernoulli Simulation

An event has probability

\[ p=0.25. \]

A uniform random draw gives

\[ U=0.18. \]

Does the simulated event occur under the rule \(U

Practice 5: Geometric Estimate

In a unit square, \(7850\) of \(10{,}000\) sampled points land inside the quarter-circle

\[ x^2+y^2\le1. \]

Use the Monte Carlo formula to estimate \(\pi\).

Practice 6: Threshold Probability

Out of \(2000\) simulated project outcomes, \(130\) exceed a cost threshold.

Estimate the exceedance probability.

Practice 7: Dependence

Two model inputs are strongly positively correlated in reality, but a simulation samples them independently.

What modeling problem can this create?

Practice 8: One Run Versus Many

A stochastic model produces an unusually large loss in one trial.

Should that single outcome be interpreted as the expected loss?

Practice Answers

Practice 1

\[ \hat{P}(A) = \frac{73}{500} = \boxed{0.146}. \]

Practice 2

\[ \bar{Y} = \frac{8+11+9+12}{4} = \frac{40}{4} = \boxed{10}. \]

Practice 3

Reducing standard error by a factor of \(3\) requires approximately

\[ 3^2=\boxed{9} \]

times as many trials.

Practice 4

Yes. Since

\[ 0.18<0.25, \]

the simulated event occurs.

Practice 5

\[ \pi \approx 4\left(\frac{7850}{10{,}000}\right) = 4(0.785) = \boxed{3.14}. \]

Practice 6

\[ \hat{P}(Y>L) = \frac{130}{2000} = \boxed{0.065}. \]

Practice 7

Independent sampling can generate combinations that do not reflect the real joint behavior of the inputs. This can distort the simulated output distribution and risk estimates.

Practice 8

No. One trial is one possible realization. The expected loss should be estimated from many trials and summarized statistically.

What to Remember

Monte Carlo simulation estimates uncertain or difficult quantities through repeated random sampling.

The central principle is: use repeated sampling to convert uncertainty into a measurable distribution of computational outcomes.

Where We Go Next

Monte Carlo simulation uses random sampling to approximate probabilities, expectations, and uncertain outcomes. Another major family of computational methods approximates derivatives directly on discrete grids.

The next lesson, Finite-Difference Methods, develops systematic formulas for replacing derivatives with differences between nearby grid values and using those approximations to solve differential models numerically.