The Central Distinction

Verification and validation address different sources of possible error.

A useful summary is

\[ \boxed{ \text{verification: Did we solve the model right?} } \]

and

\[ \boxed{ \text{validation: Did we build the right model?} }. \]

A model can pass one and fail the other.

What Is Verification?

Verification checks whether the computational implementation correctly represents the specified mathematical model.

Verification can ask:

What Is Validation?

Validation checks whether the implemented model represents the intended real system, process, or phenomenon adequately for a stated purpose.

Validation can ask:

Verification Is About the Implementation

Suppose the intended equation is

\[ y=mx+b. \]

If the program instead computes

\[ y=m(x+b), \]

the implementation is incorrect even if some test cases happen to produce similar values.

Verification is designed to detect such mismatches between specification and computation.

Validation Is About the Model

Suppose a model correctly solves

\[ P(t)=P_0e^{rt}. \]

The implementation may be verified perfectly.

But if the real population is strongly limited by resources, migration, or changing birth rates, the exponential model may not be valid for long-term prediction.

Correct computation does not guarantee an adequate model.

Verification Before Validation

Validation should generally come after substantial verification.

If the implementation is known to contain numerical or coding errors, disagreement with data cannot be interpreted meaningfully.

A practical sequence is

\[ \boxed{ \text{model specification} \rightarrow \text{implementation} \rightarrow \text{verification} \rightarrow \text{validation} }. \]

Code Verification

Code verification checks whether the software correctly implements the numerical algorithm and mathematical specification.

Useful strategies include:

Unit Tests

A unit test checks one small component of the implementation.

Examples include testing:

Small tests isolate errors before they become hidden inside a large simulation.

Hand-Checkable Cases

Simple cases with known answers are extremely valuable.

Suppose a numerical routine is intended to compute

\[ y=2x^2+3. \]

At \(x=2\),

\[ y=2(4)+3=11. \]

The implementation should reproduce this exactly or within the expected numerical tolerance.

Analytical Benchmarks

When an exact solution is available, it provides a strong benchmark.

If the exact solution is

\[ u(x), \]

and the numerical solution is

\[ u_h(x), \]

then an error measure may be based on

\[ e(x)=u_h(x)-u(x). \]

Numerical convergence can then be studied directly.

Error Norms

A numerical error can be summarized in several ways.

A maximum error is

\[ \|e\|_\infty = \max_i|e_i|. \]

A root-mean-square style error is

\[ \|e\|_2 = \left( \sum_i e_i^2 \right)^{1/2}. \]

The appropriate measure depends on the modeling objective.

Convergence Testing

Numerical verification often studies how the solution changes under refinement.

Examples include reducing:

\[ \Delta x, \] \[ \Delta t, \]

or element size \(h\).

A properly behaving numerical method should approach a stable result under appropriate refinement.

Observed Order of Accuracy

Suppose the error behaves like

\[ E_h\approx Ch^p. \]

If the mesh spacing is halved,

\[ \frac{E_h}{E_{h/2}} \approx 2^p. \]

Therefore, the observed order can be estimated by

\[ \boxed{ p \approx \log_2 \left( \frac{E_h}{E_{h/2}} \right) }. \]

Agreement with the expected order provides strong numerical verification evidence.

Grid Independence

A computed quantity should often be checked on multiple spatial resolutions.

Suppose

\[ Q_h,\qquad Q_{h/2},\qquad Q_{h/4} \]

are results on increasingly fine meshes.

If the changes become small, the quantity may be approaching a grid-independent value.

Time-Step Independence

The same principle applies to time-dependent simulations.

Compare results using

\[ \Delta t,\qquad \frac{\Delta t}{2},\qquad \frac{\Delta t}{4}. \]

If the solution changes materially as the step is refined, the original time step may be too coarse.

Conservation Checks

If the mathematical model conserves a quantity, the numerical implementation should be checked for appropriate conservation.

For example, if total mass should satisfy

\[ M(t)=M_0, \]

then a numerical monitor may compute

\[ \Delta M^n=M^n-M^0. \]

Unexpected drift can reveal implementation or discretization problems.

Dimensional Verification

Units can expose errors that numerical tests miss.

In

\[ d=vt, \]

if \(v\) has units

\[ \frac{\text{length}}{\text{time}}, \]

then \(t\) must carry time units so that \(d\) has length units.

Dimensionally inconsistent terms should never be added.

Boundary Verification

Boundary conditions are common sources of implementation error.

A verified model should check that:

Limiting Cases

A model can often be simplified by choosing special parameter values.

For example, if an interaction coefficient is

\[ \alpha=0, \]

then the interaction may disappear entirely.

The implementation should reduce to the expected simpler model in such a limit.

Symmetry Checks

If the mathematical problem is symmetric, the numerical solution may be expected to preserve that symmetry.

Unexpected asymmetry can indicate:

Manufactured Solutions

A manufactured solution is a deliberately chosen function used to test a numerical solver.

Instead of solving a difficult original problem, choose a smooth function

\[ u_{\text{test}}(x) \]

and derive the forcing or source term that makes it an exact solution of a modified problem.

The numerical method can then be checked against this known answer.

Independent Implementations

A powerful verification strategy is to compare two independently written implementations of the same mathematical model.

Agreement does not prove correctness, but independent agreement reduces the chance that one implementation-specific error dominates the result.

Validation Data

Validation requires observations, experiments, historical records, or other credible reference information.

Useful validation data should be relevant to the intended use of the model.

Matching an unrelated statistic does not validate the aspects of the model that matter for the intended decision.

Calibration Versus Validation

Calibration adjusts parameter values so that the model matches selected data.

Validation evaluates model performance against evidence that was not merely used to tune the model.

A model can fit its calibration data extremely well and still perform poorly on new cases.

Overfitting

Overfitting occurs when a model is tuned so closely to one dataset that it captures noise or case-specific details rather than transferable structure.

Overfitting can make calibration error small while validation error remains large.

This is why independent validation cases are important.

Qualitative Validation

Not every useful validation target is a single number.

A model may be expected to reproduce qualitative features such as:

Quantitative Validation

Quantitative validation compares numerical predictions with measured or reference values.

A residual may be written

\[ r_i = y_i^{\text{model}} - y_i^{\text{data}}. \]

Summary statistics can then measure the size and pattern of the residuals.

Mean Absolute Error

One simple validation metric is

\[ \boxed{ \operatorname{MAE} = \frac{1}{N} \sum_{i=1}^{N} \left| y_i^{\text{model}} - y_i^{\text{data}} \right| }. \]

MAE reports the average absolute discrepancy between model and data.

Root Mean Square Error

Another common metric is

\[ \boxed{ \operatorname{RMSE} = \sqrt{ \frac{1}{N} \sum_{i=1}^{N} \left( y_i^{\text{model}} - y_i^{\text{data}} \right)^2 } }. \]

Squaring gives greater weight to large discrepancies.

Error Scale Matters

A model error of \(1\) unit may be negligible in one context and unacceptable in another.

Validation should therefore consider:

Validation Is Purpose-Dependent

A model is not simply valid or invalid in every possible sense.

A model may be adequate for:

while being inadequate for precise long-term forecasting.

Validation Domain

Evidence collected over one range of conditions may not justify model use far outside that range.

If a model has only been validated for

\[ 0\le x\le10, \]

predictions at

\[ x=100 \]

may involve substantial extrapolation risk.

Extrapolation

Extrapolation means applying a model beyond the region supported by calibration or validation evidence.

Extrapolation may be necessary, but uncertainty should increase when model assumptions have not been tested under the new conditions.

Uncertainty in Validation

Validation data are rarely exact.

Measurements may contain uncertainty:

\[ y_{\text{obs}} = y_{\text{true}} + \varepsilon. \]

Therefore, disagreement between model and observation must be interpreted relative to both model uncertainty and measurement uncertainty.

Parameter Uncertainty

Model parameters may not be known precisely.

If

\[ \theta\in[\theta_{\min},\theta_{\max}], \]

validation should consider whether model-data agreement remains acceptable across plausible parameter values.

Structural Uncertainty

Structural uncertainty comes from uncertainty about the form of the model itself.

Examples include uncertainty about:

Validation Across Multiple Outputs

A complex model should not be judged by one convenient output alone.

Useful validation may compare:

Cross-Validation Idea

When data are limited, one strategy is to divide available observations into separate subsets.

One subset supports calibration, while another supports validation.

Rotating or repeating such splits can provide a more robust assessment of model performance.

Historical Validation

A model can sometimes be tested against historical cases not used during model development.

The model is initialized with earlier conditions and asked to reproduce later outcomes.

This does not guarantee future performance, but it provides evidence about whether the model can reproduce known system behavior.

Blind Prediction

Stronger validation evidence can come from predictions made before the comparison data are known to the modeler.

Such tests reduce the possibility of unconsciously tuning the model to match the validation case.

Validation of Stochastic Models

Stochastic models should generally be validated using distributions or repeated-run statistics rather than one simulated trajectory.

Appropriate comparisons may include:

Verification of Stochastic Models

Verification of stochastic models also requires checking the random mechanisms.

Questions include:

Benchmark Problems

A benchmark problem is a standard case used to compare numerical methods or implementations.

Good benchmarks often have:

Regression Testing

Once a model has passed verification tests, those results can be stored and checked after later software changes.

This is called regression testing.

It helps ensure that adding new features does not unintentionally break previously correct behavior.

Reproducibility

Verification and validation are much stronger when another person can reproduce the computation.

Reproducibility requires documenting:

Traceability

A robust modeling workflow should make it possible to trace a reported result back to:

Traceability supports auditing, debugging, and scientific review.

Verification Evidence Is Cumulative

No single test establishes correctness.

Confidence grows when multiple independent checks agree:

Validation Evidence Is Also Cumulative

Likewise, one successful data comparison does not establish universal model validity.

Confidence grows when the model performs adequately across:

Common Error: Validating an Unverified Code

If the implementation contains errors, apparent disagreement with data cannot be attributed confidently to the model.

Verification should therefore precede serious validation claims.

Common Error: Treating Calibration as Validation

A model that has been tuned to match one dataset has demonstrated calibration, not independent validation.

Validation should use evidence that provides a meaningful test beyond the tuning process.

Common Error: Reporting Agreement Without Uncertainty

Model and data values should not be compared as if both were exact.

Measurement uncertainty, parameter uncertainty, numerical error, and stochastic variability may all matter.

Common Error: Using One Error Metric

One summary statistic can hide important failures.

A model can have a small average error while still:

Common Error: Extrapolating Beyond Evidence

Good validation inside one operating range does not guarantee reliable behavior far outside that range.

Extrapolation should be identified explicitly and accompanied by increased uncertainty.

Common Error: Assuming Agreement Proves the Mechanism

Different models can sometimes reproduce the same observed output.

Agreement with data supports the model's usefulness but does not necessarily prove that every internal mechanism is uniquely correct.

A Reliable Verification Workflow

  1. Write the mathematical specification clearly.
  2. Test individual functions and modules.
  3. Check units, signs, and domain restrictions.
  4. Compare with hand calculations.
  5. Use analytical or manufactured solutions when available.
  6. Check initial and boundary conditions.
  7. Test limiting and symmetric cases.
  8. Perform spatial and temporal refinement studies.
  9. Check expected convergence order.
  10. Check conservation or invariant quantities.
  11. Compare independent implementations or benchmark solutions.
  12. preserve the tests for regression testing.

A Reliable Validation Workflow

  1. State the intended use of the model.
  2. Identify outputs relevant to that use.
  3. Choose credible validation data.
  4. Separate calibration data from validation evidence when possible.
  5. Compare both qualitative and quantitative behavior.
  6. Use multiple error measures when appropriate.
  7. Account for measurement and parameter uncertainty.
  8. Test across several operating conditions.
  9. Identify the validated domain.
  10. Document remaining discrepancies and limitations.
  11. Avoid claiming validity beyond the evidence.

Practice 1: Verification or Validation?

A numerical solver is tested against a differential equation with a known exact solution.

Is this primarily verification or validation?

Practice 2: Verification or Validation?

A traffic model is compared with measured traffic counts from a roadway not used during calibration.

Is this primarily verification or validation?

Practice 3: Observed Order

A numerical error is

\[ E_h=0.08 \]

on one grid and

\[ E_{h/2}=0.02 \]

after halving the spacing.

Estimate the observed order \(p\).

Practice 4: Calibration Versus Validation

Parameters are adjusted until a model closely matches one dataset.

Has the model necessarily been independently validated?

Practice 5: Mean Absolute Error

Model values are

\[ 5,\ 8,\ 11 \]

and observed values are

\[ 6,\ 7,\ 10. \]

Compute the MAE.

Practice 6: Extrapolation

A model has been validated for temperatures between

\[ 10^\circ\text{C} \]

and

\[ 40^\circ\text{C}. \]

A prediction is requested at \(120^\circ\text{C}\). What issue should be recognized?

Practice 7: Stochastic Validation

A stochastic model produces different trajectories on different runs.

Should validation normally compare one trajectory only, or distributions and repeated-run statistics?

Practice 8: Reproducibility

Name two pieces of computational information that should be recorded so another researcher can reproduce a stochastic simulation.

Practice Answers

Practice 1

This is primarily verification because the implementation is being checked against a known mathematical solution.

Practice 2

This is primarily validation because model output is being compared with independent observations of the real system.

Practice 3

\[ p \approx \log_2 \left( \frac{0.08}{0.02} \right) = \log_2(4) = \boxed{2}. \]

Practice 4

No. Matching data used to tune the parameters demonstrates calibration, not independent validation.

Practice 5

The absolute errors are

\[ 1,\ 1,\ 1. \]

Therefore,

\[ \operatorname{MAE} = \frac{1+1+1}{3} = \boxed{1}. \]

Practice 6

The prediction is a substantial extrapolation beyond the validated temperature range, so model uncertainty may be much larger.

Practice 7

Validation should normally compare distributions or repeated-run statistics rather than relying on one random trajectory.

Practice 8

Examples include the random seed, software version, parameter values, input data, model version, numerical settings, and number of simulation trials.

What to Remember

Verification and validation answer different but complementary questions.

The central principle is: first establish that the computation faithfully solves the intended model, then establish that the model is adequate for the question being asked.

Where We Go Next

Once a computational model has been implemented and subjected to verification and validation, the next question is often how its behavior changes as parameters are varied.

The next lesson, Parameter Sweeps, develops systematic methods for running many model configurations, organizing multi-parameter experiments, and identifying regions of sensitivity, transition, and robust behavior.