Floating-Point Representation | Malone Math

The Central Idea

A floating-point number is modeled after scientific notation.

In base \(10\), we might write

\[ 6.274\times 10^5. \]

A general floating-point system uses the form

\[ \boxed{x=\pm m\,\beta^e}, \]

where \(\beta\) is the base, \(m\) is the significand, and \(e\) is the exponent.

The word floating refers to the fact that the radix point can effectively move as the exponent changes.

Why Floating-Point Numbers Are Necessary

The real number line contains infinitely many numbers, and between any two distinct real numbers there are infinitely many more.

A computer has finite memory. It can therefore store only finitely many distinct bit patterns and only finitely many numerical values in a fixed format.

Floating-point arithmetic is a compromise:

represent a very wide range of magnitudes;
retain a useful number of significant digits;
use a fixed amount of storage;
perform arithmetic efficiently.

Scientific Notation as a Model

Consider the decimal number

\[ 43820. \]

In normalized scientific notation,

\[ 43820=4.382\times 10^4. \]

The same pattern works for small numbers:

\[ 0.000731=7.31\times 10^{-4}. \]

Floating-point systems use the same idea, usually with base \(2\) rather than base \(10\).

Binary Floating-Point

Most modern general-purpose computers use binary floating-point.

A normalized nonzero binary floating-point number has the conceptual form

\[ \boxed{x=(-1)^s(1.f)_2\,2^e}, \]

where:

\(s\) determines the sign;
\((1.f)_2\) is the binary significand;
\(e\) determines the scale through a power of \(2\).

The Three Main Pieces

A floating-point representation can be understood as three coordinated pieces.

Sign: determines whether the number is positive or negative.
Significand: carries the meaningful digits of the number.
Exponent: determines the overall scale.

For example, in

\[ -6.25\times 10^3, \]

the sign is negative, the significand is \(6.25\), and the exponent is \(3\).

Significand and Mantissa

The term significand refers to the digits that determine the precision of a floating-point value.

The older term mantissa is still encountered in computing literature, although significand is the more precise term for floating-point representation.

More significand digits generally mean greater numerical precision.

The Exponent Controls Range

The exponent allows a fixed number of significant digits to represent numbers over a very large range.

Compare

\[ 1.25\times 10^{-20} \]

and

\[ 1.25\times 10^{20}. \]

The significand is the same, but the exponent changes the magnitude by forty powers of ten.

Normalization

A floating-point number is usually stored in a normalized form so that its representation is essentially unique.

In decimal scientific notation,

\[ 5200=5.2\times 10^3, \]

rather than

\[ 52\times 10^2. \]

Normalization keeps the leading significant digit in a standard position.

Normalized Binary Form

For a nonzero binary number, normalization typically places a leading \(1\) before the binary point.

For example,

\[ (10110.01)_2 \]

can be written as

\[ \boxed{(1.011001)_2\times 2^4}. \]

This is the binary analogue of normalized scientific notation.

Converting a Binary Integer

Binary place values are powers of \(2\).

Consider

\[ (10110)_2. \]

Expanding by place value gives

\[ 1(2^4)+0(2^3)+1(2^2)+1(2^1)+0(2^0). \]

Therefore,

\[ \boxed{(10110)_2=22_{10}}. \]

Binary Fractions

Positions to the right of the binary point use negative powers of \(2\).

For example,

\[ (0.101)_2 \]

means

\[ 1(2^{-1})+0(2^{-2})+1(2^{-3}). \]

Thus,

\[ \frac12+\frac18=\frac58=0.625. \]

\[ \boxed{(0.101)_2=0.625_{10}}. \]

Example: A Binary Floating-Point Value

Consider

\[ (1.101)_2\times 2^3. \]

First convert the significand:

\[ (1.101)_2 = 1+\frac12+\frac18 = 1.625. \]

Then multiply by \(2^3=8\):

\[ 1.625(8)=13. \]

Therefore,

\[ \boxed{(1.101)_2\times 2^3=13}. \]

Finite Precision

A floating-point format allocates only a fixed number of bits to the significand.

If a calculation produces more significant binary digits than can be stored, the result must be rounded.

This creates a fundamental distinction:

\[ \boxed{\text{stored floating-point value}\neq\text{exact real value}} \]

in many ordinary computations.

Not Every Decimal Fraction Is Exact in Binary

A decimal fraction may terminate in base \(10\) but repeat forever in base \(2\).

One important example is

\[ 0.1_{10}. \]

In binary, its expansion is repeating:

\[ 0.1_{10}=0.00011001100110011\ldots{}_2. \]

A finite binary floating-point format must therefore store a nearby rounded value rather than exact decimal \(0.1\).

Why Some Fractions Terminate in Binary

A rational number has a terminating binary expansion when, after reduction, its denominator is a power of \(2\).

For example,

\[ \frac38 \]

terminates because

\[ 8=2^3. \]

Indeed,

\[ \frac38=(0.011)_2. \]

Why \(0.1\) Repeats in Binary

The fraction

\[ 0.1=\frac{1}{10} \]

has denominator

\[ 10=2\cdot5. \]

The factor \(5\) cannot be absorbed into a power of \(2\), so the binary expansion does not terminate.

A Simple Model Floating-Point System

Before studying machine standards, it is useful to imagine a small decimal floating-point system with three significant digits.

In this model,

\[ 3.14159 \]

might be stored as

\[ \boxed{3.14}. \]

Similarly,

\[ 98765 \]

might be represented as

\[ \boxed{9.88\times10^4} \]

under ordinary rounding.

Rounding to the Nearest Representable Number

If an exact value lies between two representable floating-point numbers, the machine must choose one according to a rounding rule.

Conceptually,

\[ x\longrightarrow \operatorname{fl}(x), \]

where \(\operatorname{fl}(x)\) denotes the floating-point number used to represent \(x\).

Representation Error

The difference between an exact real number and its stored approximation is a form of representation error.

If the exact number is \(x\) and the stored value is \(\operatorname{fl}(x)\), the absolute representation error is

\[ \boxed{|x-\operatorname{fl}(x)|}. \]

A relative form is

\[ \boxed{\frac{|x-\operatorname{fl}(x)|}{|x|}}, \]

provided \(x\neq0\).

Precision and Range Are Different

Precision describes how finely values can be distinguished.

Range describes how small and how large the representable magnitudes can be.

In floating-point design:

more significand bits primarily increase precision;
more exponent bits primarily increase range.

Floating-Point Numbers Are Not Uniformly Spaced

On the ordinary real number line, we often imagine equal spacing between successive values.

Floating-point numbers behave differently.

Near a fixed exponent, representable values are evenly spaced. When the exponent increases, the spacing increases as well.

Thus, large-magnitude numbers are generally farther apart than small-magnitude numbers.

A Spacing Example

Suppose a simplified binary system stores numbers of the form

\[ (1.b_1b_2)_2\times2^e. \]

With \(e=0\), adjacent normalized significands differ by

\[ 2^{-2}=\frac14. \]

With \(e=3\), that same significand step is scaled by \(2^3\):

\[ \frac14\cdot 8=2. \]

The representable numbers are therefore much farther apart at the larger magnitude.

Machine Epsilon

A common measure of floating-point resolution is machine epsilon.

Informally, machine epsilon describes a small separation near \(1\) associated with the format's precision.

In a binary precision-\(p\) model, a characteristic spacing near \(1\) is

\[ 2^{1-p} \]

when the significand contains \(p\) effective binary digits.

Exact conventions for the symbol \(\varepsilon\) can vary, so numerical work should state the convention being used.

The Unit Roundoff View

Numerical analysis often models rounding with

\[ \operatorname{fl}(x)=x(1+\delta), \]

where the relative perturbation \(\delta\) is small.

Under a standard round-to-nearest model,

\[ |\delta|\le u, \]

where \(u\) is called the unit roundoff.

This compact model will be useful later when we study error propagation and stability.

IEEE 754 Floating-Point

Modern systems commonly follow the IEEE 754 family of floating-point standards.

The familiar binary formats organize bits into fields representing:

a sign;
an encoded exponent;
a fraction field contributing to the significand.

The precise interpretation depends on the format and on whether the bit pattern represents a normal number, subnormal number, infinity, or NaN.

Single and Double Precision

Two widely encountered binary floating-point formats are commonly called single precision and double precision.

Conceptually:

single precision uses fewer bits and therefore less storage;
double precision uses more bits and therefore offers greater precision and range.

The choice between formats affects memory use, bandwidth, speed on some systems, and the accuracy available to a numerical computation.

The Hidden Leading Bit

A normalized nonzero binary significand always begins with \(1\):

\[ 1.f. \]

Because that leading \(1\) is predictable, a binary format can often omit it from the stored fraction field and recover it during interpretation.

This gives one additional effective significand bit for normal numbers.

Biased Exponents

Floating-point formats need to represent both positive and negative exponents.

A common strategy stores a biased exponent.

Instead of storing \(e\) directly, the format stores

\[ E=e+\text{bias}. \]

The bias shifts the exponent range into a nonnegative encoded field.

Zero

Zero cannot be represented by the ordinary normalized pattern

\[ 1.f\times2^e, \]

because that significand is nonzero.

Floating-point formats therefore reserve special bit patterns for zero.

Many standard binary systems distinguish positive zero and negative zero at the representation level, even though they compare as equal in ordinary numerical equality.

Subnormal Numbers

Near zero, ordinary normalized numbers eventually reach a smallest positive normal magnitude.

Subnormal numbers extend the representable range closer to zero by relaxing the usual normalized leading-digit rule.

This produces a more gradual transition toward zero rather than an immediate jump from the smallest normal number to zero.

Underflow

Underflow occurs when a computed magnitude becomes too small for ordinary normalized representation.

Depending on the system and the exact magnitude, the result may enter the subnormal range or eventually round to zero.

Underflow is therefore associated with the lower end of floating-point range.

Overflow

Overflow occurs when the magnitude of a computed result exceeds the largest finite representable floating-point value.

For example, repeatedly multiplying very large positive numbers may cause

\[ |x|>\text{largest finite representable magnitude}. \]

Standard floating-point systems can then produce an infinity value rather than an ordinary finite number.

Infinity

Floating-point standards commonly provide representations for positive and negative infinity:

\[ +\infty,\qquad -\infty. \]

These values are useful for representing certain overflow results and extended arithmetic outcomes.

They are not ordinary real numbers, but they provide structured behavior inside the floating-point system.

NaN

NaN means Not a Number.

NaN values represent numerical results that do not correspond to an ordinary real or infinite floating-point result.

Examples can arise from operations such as

\[ \frac{0}{0} \]

or other invalid floating-point operations.

Exact Arithmetic and Floating-Point Arithmetic Differ

In exact real arithmetic,

\[ (a+b)+c=a+(b+c). \]

In floating-point arithmetic, intermediate rounding may cause the two evaluation orders to produce different stored results.

Thus, familiar algebraic identities remain mathematically true, but a finite numerical implementation may not reproduce them exactly after rounding.

Example: Loss of a Small Addend

Suppose a simplified decimal floating-point system retains only four significant digits.

Let

\[ a=10000,\qquad b=1. \]

The exact sum is

\[ 10001. \]

But with only four significant digits, the stored result may round back to

\[ \boxed{10000}. \]

The smaller contribution disappears because the spacing at that magnitude is too coarse.

Cancellation

Subtracting nearly equal floating-point numbers can expose previously hidden rounding error.

For example, if two nearby quantities are each known only approximately,

\[ x\approx y, \]

then the difference

\[ x-y \]

may contain far fewer reliable significant digits than either input.

This phenomenon is called cancellation and will become important in the study of numerical stability.

Floating-Point Equality Requires Care

When decimal quantities are converted to binary floating point, apparently simple arithmetic may not land on the exact mathematical value expected.

Therefore, numerical programs often compare whether two values are sufficiently close rather than assuming exact bit-for-bit equality after a chain of computations.

A typical mathematical idea is to ask whether

\[ |x-y|\le \tau, \]

where \(\tau\) is an application-appropriate tolerance.

Absolute and Relative Tolerances

An absolute tolerance controls raw distance:

\[ |x-y|\le \tau_{\text{abs}}. \]

A relative tolerance scales the comparison to the magnitude involved:

\[ |x-y| \le \tau_{\text{rel}}\max(|x|,|y|). \]

Robust numerical comparisons often consider both ideas because behavior near zero differs from behavior at large magnitudes.

Floating-Point Representation Is a Discrete Model of a Continuous Line

The real numbers form a continuum, but floating-point values form a finite, discrete set.

We can think of floating-point storage as mapping an exact real number to a nearby grid point:

\[ \mathbb{R}\longrightarrow\mathbb{F}, \]

where \(\mathbb{F}\) denotes the finite set of floating-point values available in the chosen format.

This viewpoint is central to numerical analysis.

Representable Numbers Depend on the Base

Exact representation depends on the radix.

For example,

\[ \frac{1}{10} \]

terminates in base \(10\), but not in base \(2\).

Conversely,

\[ \frac18 \]

terminates in both:

\[ 0.125_{10}=0.001_2. \]

Representation error is therefore not merely about the size of a number; it is also about how that number relates to the base.

Significant Digits and Significant Bits

In decimal work, precision is often discussed in significant digits.

In binary floating-point, precision is naturally measured in significant bits.

More significant bits allow the format to distinguish more values within each exponent interval.

This increases the number of reliable binary digits carried through ordinary arithmetic.

A Geometric View of Floating-Point Spacing

Floating-point numbers can be pictured as clusters of evenly spaced points within exponent bands.

As the exponent increases by \(1\), the magnitude doubles in binary, and the absolute spacing between adjacent numbers also doubles.

The system therefore maintains roughly comparable relative precision across a wide range of magnitudes even though absolute spacing changes.

Why Relative Error Fits Floating-Point Arithmetic

Because spacing scales with magnitude, floating-point error is often better understood relatively than absolutely.

An absolute error of

\[ 10^{-6} \]

may be tiny for a quantity near \(10^3\) but enormous for a quantity near \(10^{-9}\).

Relative error places the error in context.

Storage Is Not the Same as Display

A calculator, spreadsheet, programming language, or numerical library may display fewer decimal digits than are actually stored.

Conversely, a displayed decimal may be a rounded rendering of an underlying binary value.

Therefore:

\[ \boxed{\text{displayed digits}\neq\text{stored representation}} \]

in general.

Input Conversion and Output Conversion

Numerical software often performs two conversions around a computation.

A decimal input string is converted to a binary floating-point value.
The binary floating-point result is converted back to decimal text for display.

Both stages are representation processes and should be distinguished from the mathematical operation itself.

A Reliable Procedure for Reading Floating-Point Form

Identify the base \(\beta\).
Identify the sign.
Interpret the significand.
Interpret the exponent.
Apply the scale factor \(\beta^e\).
Convert to the desired numerical form.
Check whether the representation is normalized.

A Reliable Procedure for Normalizing a Number

Write the value in the required base.
Move the radix point until the leading significant digit is in the normalized position.
Count how far the radix point moved.
Use that count to determine the exponent.
Preserve the sign.
Verify by multiplying the significand by the base raised to the exponent.

Example: Normalize a Decimal Number

Normalize

\[ 0.00482. \]

Move the decimal point three places to the right:

\[ 4.82. \]

Compensate with \(10^{-3}\):

\[ \boxed{0.00482=4.82\times10^{-3}}. \]

Example: Normalize a Binary Number

Normalize

\[ (0.001101)_2. \]

Move the binary point three places to the right:

\[ (1.101)_2. \]

Therefore,

\[ \boxed{(0.001101)_2=(1.101)_2\times2^{-3}}. \]

Example: Binary Fraction to Decimal

Convert

\[ (10.101)_2. \]

Expand by powers of \(2\):

\[ 1(2^1)+0(2^0)+1(2^{-1})+0(2^{-2})+1(2^{-3}). \]

Thus,

\[ 2+\frac12+\frac18 = 2.625. \]

Therefore,

\[ \boxed{(10.101)_2=2.625_{10}}. \]

Example: A Number That Cannot Be Stored Exactly

Suppose a toy decimal system keeps only four significant digits.

The exact value

\[ \sqrt{2}=1.41421356\ldots \]

cannot be stored exactly.

Under rounding to four significant digits, it becomes

\[ \boxed{\operatorname{fl}(\sqrt2)=1.414}. \]

The stored number is close to \(\sqrt2\), but it is not identical to it.

Example: Relative Representation Error

Using

\[ \operatorname{fl}(\sqrt2)=1.414, \]

the absolute error is approximately

\[ |1.41421356-1.414| \approx 2.1356\times10^{-4}. \]

The relative error is approximately

\[ \frac{2.1356\times10^{-4}}{1.41421356} \approx 1.51\times10^{-4}. \]

Relative error shows the approximation error compared with the size of the quantity itself.

Common Error: Assuming Every Decimal Is Exact

A value may look simple in decimal and still be nonterminating in binary.

For example,

\[ 0.1 \]

cannot be represented exactly by a finite binary fraction.

The appropriate conclusion is not that the computer is malfunctioning. The issue comes from finite representation in a different base.

Common Error: Confusing Precision with Accuracy

A format may store many digits precisely while the modeled quantity itself is inaccurate.

Conversely, a mathematically accurate model can still suffer from insufficient floating-point precision during computation.

Precision describes numerical resolution; accuracy describes closeness to the desired or true quantity.

Common Error: Confusing Precision with Range

Increasing the number of significand bits and increasing the number of exponent bits solve different problems.

More significand bits provide finer distinctions among nearby values.
More exponent bits extend the magnitudes that can be represented.

Common Error: Treating Floating-Point Arithmetic as Exact Algebra

Real-number identities provide the mathematical target, but an implementation can round after each arithmetic operation.

Therefore, changing the evaluation order can change the final floating-point result even when the two expressions are algebraically equivalent.

Common Error: Using Exact Equality for Every Numerical Test

Exact equality is appropriate when exact representation and exact computation are guaranteed.

But after general floating-point computations, a tolerance-based test may be more meaningful.

The tolerance must be chosen from the scale, purpose, and error requirements of the problem rather than selected arbitrarily.

Common Error: Ignoring Scale

The spacing between floating-point values changes with magnitude.

A difference that is negligible near one scale may be significant at another.

Numerical reasoning should therefore keep magnitude, units, and relative error in view.

Practical Checking Strategy

When a floating-point result looks surprising, inspect the computation in layers.

Check the exact mathematical formula.
Check the scale of all quantities.
Identify decimal values that may not be exact in binary.
Inspect subtraction of nearly equal values.
Check for overflow or underflow.
Compare with a higher-precision calculation when available.
Estimate the size of acceptable absolute and relative error.

Floating-Point Representation and Numerical Analysis

Floating-point representation is not an isolated implementation detail.

It affects nearly every later topic in numerical analysis:

error propagation;
conditioning;
stability;
root-finding;
linear algebra;
interpolation;
differentiation and integration;
differential equations;
convergence testing;
adaptive error control.

Practice 1: Identify the Components

For

\[ -7.25\times10^6, \]

identify the sign, significand, base, and exponent.

Practice 2: Normalize a Decimal Number

Write

\[ 0.0000917 \]

in normalized decimal scientific notation.

Practice 3: Convert Binary to Decimal

Convert

\[ (101.011)_2 \]

to decimal.

Practice 4: Normalize a Binary Number

Normalize

\[ (11010.1)_2. \]

Practice 5: Determine Whether a Fraction Terminates in Binary

Decide whether

\[ \frac{5}{16} \]

has a terminating binary representation.

Practice 6: Another Binary Termination Test

Decide whether

\[ \frac{1}{5} \]

has a terminating binary representation.

Practice 7: Rounding in a Toy System

A decimal floating-point system keeps four significant digits. Round

\[ 82.3764 \]

to the nearest representable four-significant-digit value.

Practice 8: Absolute Error

An exact quantity is

\[ x=3.14159265, \]

while the stored approximation is

\[ \operatorname{fl}(x)=3.142. \]

Find the absolute representation error.

Practice 9: Precision or Range?

For each goal, identify whether additional significand bits or additional exponent bits are primarily needed.

Distinguish more closely spaced values near \(1\).
Represent much larger finite magnitudes.

Practice 10: Explain the Numerical Issue

A program computes two mathematically equivalent expressions in different orders and obtains slightly different last digits.

Explain why this can happen in floating-point arithmetic.

Practice Answers

Practice 1

For

\[ -7.25\times10^6, \]

sign: negative;
significand: \(7.25\);
base: \(10\);
exponent: \(6\).

Practice 2

Move the decimal point five places to the right:

\[ \boxed{0.0000917=9.17\times10^{-5}}. \]

Practice 3

Expand by powers of \(2\):

\[ (101.011)_2 = 1(2^2)+0(2^1)+1(2^0)+0(2^{-1})+1(2^{-2})+1(2^{-3}). \]

Therefore,

\[ 4+1+\frac14+\frac18=5.375. \]

\[ \boxed{(101.011)_2=5.375_{10}}. \]

Practice 4

Move the binary point four places to the left:

\[ \boxed{(11010.1)_2=(1.10101)_2\times2^4}. \]

Practice 5

Since

\[ 16=2^4, \]

the fraction terminates in binary.

In fact,

\[ \boxed{\frac{5}{16}=(0.0101)_2}. \]

Practice 6

The denominator \(5\) is not a power of \(2\), so

\[ \boxed{\frac15} \]

does not have a terminating binary representation.

Practice 7

Four significant digits retain

\[ 82.37 \]

before rounding. The next digit is \(6\), so round upward:

\[ \boxed{82.38}. \]

Practice 8

The absolute error is

\[ |3.14159265-3.142| = \boxed{0.00040735}. \]

Practice 9

More closely spaced values near \(1\): additional significand bits.
Much larger finite magnitudes: additional exponent bits.

Practice 10

Floating-point arithmetic rounds intermediate results to a finite representable set. Different evaluation orders can therefore round at different stages and produce slightly different final stored values even when the exact algebraic expressions are equivalent.

What to Remember

Floating-point representation models a real number using a sign, a significand, a base, and an exponent:

\[ \boxed{x\approx\pm m\beta^e}. \]

the significand controls much of the available precision;
the exponent controls much of the available range;
normalization places the significand in a standard form;
many decimal fractions are not exactly representable in binary;
rounding maps exact values to nearby representable values;
floating-point values are discrete and nonuniformly spaced;
overflow, underflow, infinity, subnormals, and NaN belong to the representation system;
floating-point arithmetic can differ from exact real arithmetic because of intermediate rounding.

The essential habit is to treat a stored floating-point value as a numerical approximation whose reliability depends on precision, scale, representation, and the operations applied to it.

Where We Go Next

Floating-point representation explains why numerical computation begins with approximation even before an algorithm introduces additional error.

The next lesson, Sources and Propagation of Error, develops a systematic language for measuring numerical error, distinguishing its sources, and understanding how errors move through a sequence of calculations.