Identifiability

The Central Question

Suppose a model depends on an unknown parameter vector \(\boldsymbol{\theta}\), and the model predicts observations through a map

An inverse problem starts with observed data \(\mathbf{y}\) and asks us to recover \(\boldsymbol{\theta}\).

The central identifiability question is:

If two different parameter values produce exactly the same observable output, then the data cannot distinguish those parameter values.

What Identifiable Means

A parameter is identifiable when the available observations, together with the model structure, determine that parameter uniquely within the admissible parameter space.

In the simplest one-parameter case, if

implies

then \(F\) is one-to-one with respect to the parameter and the parameter is identifiable.

A Simple Identifiable Example

Suppose

Solving for \(\theta\) gives

Every possible output \(y\) corresponds to exactly one parameter value. Therefore, \(\theta\) is identifiable from a single exact observation of \(y\).

A Simple Nonidentifiable Example

Now suppose

If the admissible parameter domain contains both positive and negative values, then

For example,

An observation \(y=9\) cannot determine whether \(\theta=3\) or \(\theta=-3\). The parameter is not globally identifiable on \(\mathbb{R}\).

Parameter Restrictions Can Change Identifiability

If the same model

is restricted to

then

is unique whenever \(y\ge0\).

Identifiability therefore depends not only on the equation but also on the parameter domain, constraints, prior knowledge, and observation design.

Global and Local Identifiability

It is useful to distinguish two levels of uniqueness.

A parameter is globally identifiable when the data determine exactly one admissible parameter value over the entire parameter domain.

A parameter is locally identifiable when the data determine a unique parameter value only within a neighborhood of the true parameter.

A model can be locally identifiable while still possessing several widely separated parameter values that produce the same output.

Structural Identifiability

Structural identifiability asks whether parameters could be uniquely recovered under idealized conditions:

• the model structure is assumed correct;
• observations can be exact;
• the relevant outputs are observed continuously or sufficiently richly;
• there is no measurement noise;
• there is no numerical error.

Structural identifiability is therefore a property of the mathematical model and observation scheme, not of a particular noisy data set.

Practical Identifiability

Practical identifiability asks whether parameters can be estimated with useful precision from the data actually available.

A parameter may be structurally identifiable but practically difficult to estimate because the experiment has:

• too few observations;
• large measurement noise;
• poorly chosen sampling times;
• weak parameter sensitivity;
• highly correlated parameter effects;
• a narrow range of experimental conditions.

Structural Versus Practical Failure

The distinction is important.

If a model is structurally nonidentifiable, collecting more precise data of the same type will not fix the problem. The model or observation design must change.

If a model is structurally identifiable but practically weak, then improved measurements, additional observations, better excitation, or redesigned experiments may substantially improve estimation.

Parameter Confounding

Two parameters are confounded when their effects on the observable output cannot be separated.

Consider

If \(x\) and \(y\) are observed but both \(a\) and \(b\) are unknown, the data identify only the product

Infinitely many pairs \((a,b)\) can produce the same value of \(c\). Therefore, \(a\) and \(b\) are not separately identifiable.

Identifiable Combinations

Even when individual parameters are not identifiable, certain combinations of parameters may be.

In

the individual parameters \(a\) and \(b\) are not identifiable, but their product

is identifiable from the slope \(y/x\), provided \(x\ne0\).

Finding identifiable combinations often reveals which quantities the data genuinely contain information about.

Example: A Ratio Is Identifiable

Suppose a measured quantity satisfies

If both \(k\) and \(m\) are unknown, then the output depends only on the ratio

The experiment may identify \(r\), but not \(k\) and \(m\) individually.

This occurs frequently in physical models where only a dimensionless group or parameter ratio controls the measured behavior.

Observation Design Matters

A parameter may be identifiable under one observation scheme and nonidentifiable under another.

Suppose

If only one measurement \(y(t_1)\) is available and both \(A\) and \(k\) are unknown, then one equation contains two unknowns:

Many pairs \((A,k)\) satisfy the same observation.

Additional measurements at distinct times can separate the amplitude and decay-rate effects.

Two Measurements Can Resolve the Exponential Example

Suppose we observe

with \(t_1\ne t_2\) and positive observations.

Dividing the equations gives

Taking logarithms,

Once \(k\) is known,

The second measurement adds the independent information needed to distinguish the parameters.

Counting Equations Is Only a First Check

Having at least as many observations as unknown parameters is useful, but it does not guarantee identifiability.

Consider

There are two equations and two unknowns, but the second equation is merely twice the first. The two observations contain only one independent piece of information.

The system identifies only

Linear Identifiability and Rank

In a linear parameter-estimation problem,

the parameter vector \(\boldsymbol{\theta}\in\mathbb{R}^p\) is uniquely determined when the design matrix \(X\) has full column rank:

If the columns of \(X\) are linearly dependent, then at least one parameter direction cannot be distinguished from the others.

Null Spaces Reveal Nonidentifiable Directions

Suppose

for some nonzero vector \(\mathbf{v}\).

Then

Therefore, moving the parameter vector in the direction \(\mathbf{v}\) changes the parameters without changing the predicted data.

Such a direction is a nonidentifiable parameter direction.

Sensitivity as Local Information

For a nonlinear model

local parameter effects are described by the sensitivity matrix, or Jacobian,

Its entries are

Each column describes how the model output changes when one parameter is perturbed.

Rank of the Sensitivity Matrix

If the columns of the Jacobian are locally independent, then small changes in different parameters produce distinguishable output changes.

A common local identifiability condition is

where \(p\) is the number of parameters.

Rank deficiency indicates that at least one local parameter direction produces no first-order change distinguishable from other parameter changes.

Nearly Dependent Sensitivities

Practical identifiability can be weak even when the Jacobian has full rank.

If two sensitivity columns are almost proportional, then two parameters affect the output in nearly the same way.

The problem may technically have a unique solution, but small amounts of noise can cause large changes in the estimated parameter values.

This is one route by which identifiability connects directly to conditioning.

Strong and Weak Parameter Sensitivity

If

is large over the observed region, then changes in \(\theta_j\) produce noticeable changes in the output.

If the derivative is near zero, the model output is nearly insensitive to \(\theta_j\), making the parameter difficult to estimate from those observations.

Weak sensitivity does not always imply structural nonidentifiability, but it often leads to poor practical identifiability.

Scaling Sensitivities

Raw sensitivities can be misleading when parameters use very different units or magnitudes.

A dimensionless relative sensitivity may be written as

when the expression is well defined.

Scaling allows parameter effects to be compared on a more consistent basis.

The Geometry of Identifiability

Parameter estimation can be viewed geometrically.

Each parameter vector \(\boldsymbol{\theta}\) maps to a predicted observation vector \(F(\boldsymbol{\theta})\).

If many parameter vectors map to exactly the same observation vector, then the inverse image of the data contains multiple points or even an entire curve or surface.

Identifiability asks whether that inverse image collapses to a single admissible parameter point.

Objective-Function Valleys

Suppose parameter estimates are obtained by minimizing a residual objective

A sharply defined isolated minimum often indicates that the data constrain the parameters well.

A long, flat valley may indicate that many parameter combinations fit the data almost equally well.

Such valleys are common signatures of practical nonidentifiability or strong parameter correlation.

Profile-Based Diagnostics

One practical diagnostic is to fix one parameter at a sequence of values and re-optimize the remaining parameters.

If the optimized objective changes sharply as the fixed parameter moves away from its best-fit value, the parameter is strongly constrained.

If the objective remains nearly unchanged over a wide interval, the data do not strongly determine that parameter.

This idea underlies parameter-profile and profile-likelihood methods.

Correlation Between Estimated Parameters

Strong parameter correlation is another warning sign.

For example, one parameter may increase while another decreases so that their combined effect on the output remains nearly unchanged.

In a two-parameter fit, this can appear as an elongated uncertainty ellipse instead of a compact circular region.

High correlation does not automatically imply exact nonidentifiability, but it often signals weak separation of parameter effects.

Noise Does Not Create Structural Information

Noise can obscure information, but it cannot create information that the observation map does not contain.

If

exactly for two distinct parameter vectors, then arbitrarily precise measurements of that same model output still cannot distinguish them.

Structural nonidentifiability must be addressed through model reformulation, additional observables, parameter constraints, or experimental redesign.

More Data Is Not Always More Information

Repeating the same measurement many times can reduce random measurement uncertainty, but it may not resolve parameter confounding.

If every observation is sensitive only to the combination

then collecting thousands of observations of the same form may estimate \(a+b\) very precisely while still failing to identify \(a\) and \(b\) separately.

Different Experimental Conditions Can Separate Parameters

Consider the model

If every experiment uses the same relationship \(z=2x\), then

so the experiment identifies only \(a+2b\).

Varying \(x\) and \(z\) independently can create linearly independent information and make the parameters separately identifiable.

Identifiability and Experimental Design

Good experimental design aims not merely to collect more observations but to collect observations that distinguish competing parameter effects.

Useful design choices may include:

• measuring additional state variables;
• sampling at more informative times;
• varying input amplitudes or frequencies;
• changing initial conditions;
• extending the observed parameter-response range;
• introducing independent perturbations;
• selecting measurements with complementary sensitivities.

Example: Early and Late Measurements

In a process with fast and slow dynamics, early-time observations may be strongly sensitive to one parameter while late-time observations may be more sensitive to another.

Sampling only at early times can leave the slow parameter weakly identified. Sampling only at late times can miss the fast parameter.

A well-designed schedule can intentionally collect data in regions where the parameter sensitivities differ most strongly.

Reparameterization

When the data identify only a parameter combination, it may be better to reformulate the model in terms of that combination.

If the original model depends only on

then replacing \((a,b)\) with the single parameter \(c\) can remove an artificial nonidentifiability.

Reparameterization aligns the mathematical representation with the information that the data actually contain.

Fixing a Parameter

If one parameter is known from an independent source, fixing it can make the remaining parameters identifiable.

In

if \(b\) is independently measured and nonzero, then

Independent calibration information can therefore break parameter confounding.

Priors and Identifiability

In Bayesian estimation, prior information can constrain parameters that are only weakly informed by the likelihood.

This does not mean the data themselves have become more informative.

It is important to distinguish between:

• information supplied by the observations;
• information supplied by prior assumptions;
• information supplied by hard parameter bounds.

A narrow posterior distribution can arise from strong prior information even when the data alone provide weak identification.

Identifiability and Regularization

Regularization can select a stable or preferred solution when an inverse problem is poorly constrained.

For example, one may minimize

where \(R\) penalizes undesirable parameter values.

Regularization can make the computational problem well behaved, but the selected estimate partly reflects the regularization rule rather than data information alone.

Identifiability Is Not the Same as Accuracy

A parameter estimate can be identifiable and still inaccurate.

Accuracy can be degraded by:

• measurement bias;
• model misspecification;
• incorrect boundary or initial conditions;
• numerical approximation error;
• unmodeled disturbances.

Identifiability addresses uniqueness and information content. Accuracy asks whether the resulting estimate is close to the physical or target value.

Identifiability Is Not the Same as Precision

Precision concerns the spread or repeatability of an estimate.

A parameter can be structurally identifiable yet estimated with very poor precision because the data are noisy.

Conversely, a strongly regularized estimator may produce a very narrow numerical result even though the data alone do not strongly identify the parameter.

Identifiability and Model Complexity

Adding parameters increases model flexibility, but it can also reduce identifiability.

Two parameters may be introduced to represent two physical effects, yet the available measurements may respond only to their sum or product.

A more complex model is not automatically a more informative model.

Parameterization should be matched to the resolution supported by the data.

A Hierarchy of Questions

Before performing parameter estimation, it is useful to ask:

1. Which parameters are unknown?
2. Which outputs are actually observed?
3. Which parameter combinations affect those outputs?
4. Can different parameter values produce the same observations?
5. Are the relevant sensitivities linearly independent?
6. Are the sensitivities large enough relative to measurement noise?
7. Would different experiments separate the parameter effects more clearly?

A Practical Identifiability Workflow

1. Define the parameter domain and constraints.
2. Write the forward observation map explicitly.
3. Look for algebraic parameter combinations such as sums, products, and ratios.
4. Check for symmetries that leave the output unchanged.
5. Compute or approximate parameter sensitivities.
6. Examine the rank and conditioning of the sensitivity matrix.
7. Fit the model and inspect the objective-function geometry.
8. Examine parameter profiles, correlations, or uncertainty regions.
9. Test whether alternate parameter values fit nearly equally well.
10. Redesign the model or experiment if necessary.

Common Error: Assuming a Successful Fit Means Identifiability

An optimizer may return one parameter vector even when infinitely many parameter vectors fit the data equally well.

The fact that an algorithm produced a numerical answer does not prove that the answer is uniquely determined by the data.

Identifiability must be assessed separately from optimization convergence.

Common Error: Treating Bounds as Data

Suppose the likelihood is nearly flat over

but the fitting software imposes

The optimizer may return a finite interval or a boundary solution. That does not mean the data identify \(\theta\) well.

Bounds restrict the admissible solution space; they do not create new observational information.

Common Error: Looking Only at One Parameter at a Time

A parameter can appear sensitive when varied alone but become poorly identifiable when several parameters are varied together.

This happens because another parameter may compensate for its effect.

Multi-parameter identifiability therefore requires studying joint parameter directions, not only one-at-a-time perturbations.

Common Error: Ignoring Units and Scaling

Large numerical differences among sensitivity columns can reflect different units rather than fundamentally different information content.

Appropriate parameter and residual scaling is often necessary before interpreting singular values, condition numbers, or optimization geometry.

Common Error: Confusing Nonidentifiability with Numerical Failure

A solver that struggles to converge may indicate a poor algorithm, poor initialization, bad scaling, or true nonidentifiability.

Conversely, a solver that converges quickly may still be solving a nonidentifiable problem.

Numerical behavior is evidence to investigate, not a substitute for identifiability analysis.

Practice 1: One-to-One Observation Map

Determine whether \(\theta\) is identifiable from an exact observation of

Practice 2: Symmetry

Suppose

Is \(\theta\) globally identifiable on all of \(\mathbb{R}\)?

Practice 3: Parameter Combination

Consider

If \(a\) and \(b\) are both unknown, identify the parameter combination that the data can determine.

Practice 4: Dependent Equations

Determine whether \(a\) and \(b\) are separately identifiable from

Practice 5: Full-Rank Linear Design

Consider

Are \(\theta_1\) and \(\theta_2\) identifiable?

Practice 6: Rank-Deficient Design

Consider

Explain why two independent parameters cannot be identified from \(\mathbf{y}=X\boldsymbol{\theta}\).

Practice 7: Exponential Measurements

For

explain why one exact measurement at one time is insufficient to determine both \(A\) and \(k\).

Practice 8: Structural or Practical?

A model has a unique parameter solution in exact arithmetic, but the measured output changes only by about \(0.001\%\) when the parameter changes by \(10\%\).

Is the main concern structural identifiability or practical identifiability?

Practice 9: Experimental Design

Suppose every experiment obeys \(z=3x\) in the model

What parameter combination is identified, and what experimental change could help identify \(a\) and \(b\) separately?

Practice 10: Regularization

A nonidentifiable inverse problem is solved by adding a strong penalty term that selects one parameter vector.

Does the existence of this unique regularized solution prove that the data uniquely identify the parameters? Explain.

Practice Answers

Practice 1

Yes. Solve

for \(\theta\):

Each exact observation \(y\) corresponds to one parameter value.

Practice 2

No. The outputs are symmetric about \(\theta=2\). If

and

then

Therefore, the observation generally corresponds to two parameter values.

Practice 3

The data determine the sum

They do not separately determine \(a\) and \(b\) unless additional information is supplied.

Practice 4

The second equation is twice the first:

The two equations contain only one independent linear constraint, so \(a\) and \(b\) are not separately identifiable.

Practice 5

Yes. The design matrix is the identity matrix and has rank \(2\):

Therefore,

Practice 6

The second column is exactly twice the first:

Therefore,

The two parameter effects are linearly dependent.

Practice 7

One measurement gives only one equation:

Many pairs \((A,k)\) satisfy this equation. At least one additional independent observation or source of information is needed.

Practice 8

The main concern is practical identifiability. The parameter is theoretically unique, but the observable response is extremely weak relative to likely measurement and numerical error.

Practice 9

Because \(z=3x\),

The experiment identifies

To identify \(a\) and \(b\) separately, vary \(x\) and \(z\) independently so that the resulting design matrix has two independent columns.

Practice 10

No. The regularization term adds information or preference beyond the data. A unique regularized solution shows that the modified optimization problem has selected one estimate, not that the observations alone uniquely determine the parameters.

What to Remember

Identifiability asks whether unknown parameters can be distinguished from the available observations.

• Global identifiability means a unique admissible parameter value overall.
• Local identifiability means uniqueness only near a parameter value.
• Structural identifiability concerns ideal mathematical uniqueness.
• Practical identifiability concerns useful estimation from finite noisy data.
• Parameter combinations may be identifiable even when individual parameters are not.
• Rank deficiency reveals nonidentifiable directions in linearized problems.
• Nearly dependent sensitivities can produce severe practical uncertainty.
• More data help only when the additional observations add independent information.
• Experimental design can be used deliberately to separate parameter effects.
• Regularization or priors can stabilize estimation without creating new data information.

The key principle is:

Where We Go Next

Identifiability tells us whether the inverse problem contains enough information to distinguish unknown parameters. Once that question has been addressed, the next task is to determine parameter values that make the model agree with observed data.

The next lesson, Parameter Fitting, develops the general fitting framework: choosing a residual, defining an objective function, searching parameter space, comparing model predictions with observations, and evaluating whether the fitted result is meaningful.