Numbers feel authoritative. When a headline announces that a new policy reduced crime by 40 percent, or that a supplement increases energy levels by 300 percent, or that nine out of ten dentists recommend a particular product, most readers slow down their skepticism. Numbers suggest measurement. Measurement suggests science. Science suggests truth.

But numbers are not self-interpreting. Every statistic is the product of decisions: decisions about what to count, how to count it, which population to study, which time period to examine, which comparisons to make, and how to present the results. Those decisions can be made carefully and transparently, or they can be made carelessly, misleadingly, or dishonestly. The number itself does not tell you which category you are in.

This chapter teaches students how to read, interpret, and evaluate numerical evidence. By the end of it, you should be able to approach any statistical claim with informed skepticism, ask the right questions about numerical evidence, and find the original data source rather than accepting secondhand reports at face value. The goal is simple and non-negotiable: do not merely find data. Earn trust in data.

This chapter teaches students how to read, interpret, and evaluate numerical evidence. Students will learn why data always requires interpretation, how percentages, averages, and rates can mislead, how sample size affects what conclusions data can support, how to distinguish correlation from causation, how charts and graphs can be constructed to distort reality, how cherry-picked data creates false impressions, and how to check data claims before trusting or repeating them.

Data does not speak for itself. This is true even when data is accurate, honestly collected, and transparently reported. Data is always a representation of the world — a partial, constructed, and purposive representation shaped by the choices of the people who gathered it.

Every dataset begins with someone deciding what to measure and what not to measure. A study of workplace productivity might count the number of tasks completed per hour, the revenue generated per employee, or the score on a standardized output measure. Each choice captures something real and misses something real. The dataset you are looking at reflects those choices, and understanding what was left out is as important as understanding what was included.

This is not a criticism of data collection. Perfect measurement is impossible. The point is that when you encounter data, your first question should be: what did this dataset actually measure? Not "what does this tell me about the world?" but "what specific phenomenon did these researchers actually count, under what conditions, and with what definitions?"

A number without context is not information. The statement "the unemployment rate is 4.2 percent" conveys almost nothing until you know: 4.2 percent of whom? Measured how? Compared to when? What counts as "unemployed" in this definition? Does it include people who have stopped looking for work?

Every piece of data exists within a framework that gives it meaning. When a data claim is presented without that framework, the framework has been hidden rather than removed. Your job as a careful reader is to surface it.

Even when data is accurate and honestly presented, the gap between what the data shows and what conclusion is drawn from it can be enormous. A study might show that students who eat breakfast perform better on standardized tests. The data may be accurate. But "students who eat breakfast perform better" does not prove that eating breakfast causes better performance. Students who eat breakfast may come from households with more stable morning routines, more resources, and more consistent sleep schedules. Any of those factors could explain the difference.

The distance between data and conclusion — the inferential gap — is where most statistical errors, exaggerations, and distortions live. Understanding that gap is the central task of evaluating numerical evidence.

Responsible data analysis almost always produces conclusions that are probabilistic, qualified, and uncertain. When you encounter data presented with complete certainty — "this proves," "the evidence is clear" — that confidence should increase your skepticism. Real data almost never justifies certainty of that kind. Uncertainty is a feature of honest research, not a bug.

Percentages, raw counts, and rates are the most common forms in which numerical evidence appears in everyday communication. Each tells a different story. Choosing among them — or mixing them without explanation — is one of the most common ways numerical claims mislead.

A raw count is simply the number of instances of something. Raw counts are useful when the total population is already known and roughly equivalent across comparisons. They become misleading when used to compare groups of very different sizes.

If one city reports 200 car thefts and a neighboring city reports 150, it sounds like the first city has a worse problem. But if the first city has 2 million residents and the second has 50,000, the second city's rate is dramatically higher per person. Comparing raw counts between unequal populations produces misleading impressions almost by design. Always ask whether the populations being compared are roughly equivalent in size.

"A 50 percent increase" sounds large. But 50 percent of a very small number is still a very small number. If a disease affects 2 people per million and a new study finds it now affects 3 people per million, that is a 50 percent increase. It is also an increase of one person per million. Whether you describe this as a dramatic rise or a negligible change depends entirely on which number you emphasize.

Always ask: what is the baseline? What does this percentage represent in absolute terms? The absolute and relative descriptions of the same change can both be technically accurate while creating very different impressions.

These two terms are not interchangeable, but they are routinely confused. If an approval rating rises from 40 percent to 50 percent, it has risen by 10 percentage points. It has also risen by 25 percent (because 10 is 25 percent of 40). Both statements are technically accurate. Which you choose shapes how dramatic the change appears. Reputable sources distinguish between them. When a source does not, ask which one they mean.

When people say "the average," they usually mean a number that represents the center or typical value of a dataset. But "average" can mean several different things, and each one tells a different story about the same data.

Even when the correct type of average is used, the average alone often misleads because it hides the variation in the data. Two datasets can have identical means while looking completely different. If the average test score in two classes is 75 out of 100, one class might have scores ranging from 70 to 80 — a tightly clustered group — while the other has scores ranging from 30 to 100. The mean tells you nothing about this difference.

When a source reports an average without any indication of how widely individual values differ, ask: how varied are the individual values that produced this average? Would knowing the range change my interpretation? Variation carries essential information that the average alone cannot provide.

Most data about large populations comes from samples — subsets of the population studied, whose results are used to draw inferences about the whole. Understanding how samples work is essential for evaluating any study that generalizes from a group of participants to a broader population.

The conclusions drawn from samples are always estimates with some degree of uncertainty. That uncertainty is not a flaw. It is an honest acknowledgment of the gap between the sample and the population. When that uncertainty is communicated honestly — through confidence intervals and margins of error — a sample-based finding can be highly informative. When it is hidden, the finding is presented as more certain than the evidence warrants.

A larger sample generally produces more precise estimates. A poll of 1,000 randomly selected voters produces a more reliable estimate of national opinion than a poll of 30 voters, because larger samples are less susceptible to the influence of unusual individuals or random chance. But sample size alone does not determine quality. A large, poorly constructed sample can produce worse estimates than a smaller but well-constructed one.

Be cautious about claims based on very small samples — fewer than 30 participants is generally too small to support broad generalizations, and even samples of 100 to 200 may be inadequate for detecting subtle effects. Watch especially for claims that cite "a study of participants" without specifying how many. Findings from pilot studies of 20 or 30 people are preliminary at best.

Sample size matters less than how the sample was selected. A randomly selected sample of 1,000 people is far more representative of a national population than a self-selected sample of 10,000 who chose to respond to an online survey. The key concept is representativeness: does the sample reflect the diversity and characteristics of the population you are trying to understand?

Most reputable polls report a margin of error — the range within which the true population value is estimated to fall with a given level of confidence. A poll reporting 52 percent support with a margin of error of ±3 percentage points at 95 percent confidence means: if this poll were conducted repeatedly, 95 percent of the time the true value would fall between 49 and 55 percent.

Two critical points: First, margins of error apply only to random sampling error. They do not account for biased question wording or sampling undercoverage. Second, a margin of error means results that look different may not actually be different. If Candidate A polls at 51 percent and Candidate B at 49 percent with a ±3 margin, those candidates are in a statistical tie. Headlines that declare a "lead" without acknowledging the margin of error mislead readers.

One of the most consequential errors in everyday reasoning about data is confusing correlation with causation. It is consequential because the policy implications of "A causes B" are dramatically different from "A and B happen to occur together."

Two variables are correlated when changes in one tend to coincide with changes in the other. Correlations are measured by a coefficient ranging from -1 to +1: near +1 indicates a strong positive relationship; near -1 indicates a strong negative one; near 0 indicates little relationship. But correlation says nothing about why two variables move together.

Many things are correlated without any causal relationship. Ice cream sales and drowning rates both rise in summer — not because ice cream causes drowning, but because both are driven by a third factor: warm weather. This is called a spurious correlation, a statistical relationship produced by a common cause rather than any direct connection between the correlated variables.

Establishing causation is much harder than demonstrating correlation. Three conditions must generally be met:

The strongest evidence for causation comes from randomized controlled trials (RCTs), in which participants are randomly assigned to treatment and control groups. Because assignment is random, the two groups should be equivalent in all ways except the treatment. Any subsequent difference can therefore be attributed to the treatment rather than to pre-existing differences.

Most data we encounter in everyday life comes from observational studies, where researchers observe what happens without randomly assigning anyone to a condition. In observational studies, confounders are always a concern. The relationship between two variables may be real, but it may be produced by third variables rather than a direct causal connection. Treat any observational study that claims to have "proven" causation with proportional skepticism.

Visual representations of data — charts, graphs, and infographics — are powerful communication tools. They are also powerful tools for distortion. The same dataset can look dramatically different depending on how a chart is drawn. Most readers absorb the visual impression a chart creates without closely examining its structure. Understanding how charts mislead helps you read them critically.

The most common chart manipulation is a y-axis that does not start at zero. When the y-axis starts at a value other than zero, small differences between data points appear much larger than they actually are.

Imagine quarterly sales figures where Q1 is $9.8 million and Q2 is $10.1 million — a 3 percent increase. On a chart with a y-axis running from $0 to $15 million, the two bars look nearly identical. On a chart with a y-axis running from $9.5 million to $10.5 million, the Q2 bar visually dominates, even though the underlying change is small. This manipulation is especially common in political and advocacy communication, where the goal is to make small improvements look dramatic or small declines look catastrophic.

How to check: Always look at the y-axis before drawing conclusions from a chart's visual impression. Note where the axis starts. Ask: if the axis started at zero, would the visual story change significantly?

Some charts use two different y-axes — one on the left and one on the right — to display two variables on the same graph. This can be a legitimate technique, but it is frequently used to imply a relationship between variables by visually aligning their trend lines.

If you independently scale two unrelated variables and overlay their trends on the same chart, they will often appear to move together simply because of how the axes are scaled relative to each other. The visual impression of correlation is a product of axis scaling, not of any actual relationship between the variables. Be especially cautious about dual-axis charts that appear to show two variables moving dramatically together or in dramatic opposition.

Pie charts show proportions of a whole. They become misleading when there are too many categories — small slices become visually indistinguishable — and especially when presented in three-dimensional perspective, which distorts apparent slice sizes based on position in the view. Slices at the front of a 3D pie chart appear larger than equal slices at the back. Always check the actual percentages labeled on pie charts rather than relying on visual impression.

When data is represented by the size of shapes, readers often judge size by a linear dimension rather than area. A circle intended to be twice the area of another only needs a radius about 1.4 times as large. A designer who doubles the radius creates a circle that appears four times as large to most readers. Check whether shapes represent area or a linear dimension.

Cherry-picking is the practice of selecting data points, time periods, or comparisons that support a desired conclusion while ignoring data that contradicts it. Cherry-picked data is often technically accurate — the specific numbers may be correct — but the selection creates an impression the full dataset would not support.

One of the most common forms of cherry-picking is choosing a time period that starts or ends at a particularly favorable point. If you want to show that an economy has grown, start your graph at the lowest point of a recession. If you want to show it has declined, start just before a market peak. The actual trend over a longer and more neutral period might tell a very different story.

Climate data is frequently cherry-picked by time period. Because temperatures vary year to year, it is possible to select a starting year that was unusually warm and an ending year that was slightly cooler and claim this shows "cooling," even while the long-term trend is strongly upward.

How to check: Look for whether the time period shown is the full available record or a selected portion. Ask whether the result would change if the period were extended backward or forward.

Another common form is selecting comparison groups that make one's preferred data point look favorable. If you want to show that your city's schools perform well, compare them to the lowest-performing cities rather than to the national average. If you want to show that a country's health outcomes are poor, compare them to the best-performing countries and omit the middle. The comparison group determines whether a number looks good or bad — and that choice can be made strategically.

How to check: Ask who is being compared to whom, and whether this group was chosen because it is the most meaningful comparison or because it makes the desired conclusion look strongest.

A third form involves choosing which metric to report. Most complex phenomena can be measured in multiple ways, and different metrics can tell different stories. A hospital reporting excellent outcomes might highlight survival rates while omitting complication rates. A company might emphasize revenue growth while omitting profit margins.

Ask whether there are other metrics that would measure the same underlying phenomenon. Ask whether the chosen metric is the most meaningful one for the question being asked, or whether it was chosen because it supports a particular conclusion.

Encountering a statistical claim is the beginning of an evaluation process, not the end of it. This section provides a practical workflow for checking data claims before trusting or repeating them in your research.

What exactly is being claimed? Numerical claims are often expressed vaguely. "Crime is up" could mean any of dozens of things depending on what kind of crime, which geography, which time period, and which measurement method. Before you can evaluate a claim, you need to know what it specifically asserts. Restate the claim in your own words as precisely as you can: what quantity is measured, what comparison is made, and what conclusion is drawn.

Most data claims you encounter in news articles, social media, and informal writing are secondhand. Each step away from the original source introduces opportunities for misrepresentation, oversimplification, or error. Your goal is to find the primary source: for research findings, the original published study; for government statistics, the official agency report or database; for survey data, the methodology documentation.

When you cannot access the original source, treat the claim with proportional skepticism based on how many steps removed your source is from the original data.

What organization produced this data? What is their methodology? What is their funding source? What is their institutional mission? Be especially cautious about data produced by organizations with a direct financial or political interest in the findings. Industry-funded studies on the health effects of their own products, and advocacy organizations' analyses of their own programs, represent situations where the data producer has interests that might influence how data is collected and reported.

If the claim comes from a study, how was it designed? What was the sample size? How was the sample selected? Was the study experimental or observational? How were key variables measured? These questions are usually answered in the methods section of a published paper or in methodology documents published by reputable polling organizations. If the methodology is not described and cannot be found, the claim cannot be fully evaluated.

Does the data cover the full available time period, or a selected portion? Is the comparison group the most meaningful available? Is the metric reported the standard one for this type of measurement? Checking for cherry-picking requires knowing something about the broader context. If you are not already familiar with it, look for other sources covering the same phenomenon — they may use different time periods or metrics, and the differences can reveal whether any single source is presenting a selective picture.

Even if the data is accurately reported and the methodology is sound, does the conclusion drawn actually follow from it? Is causal language being used for a correlation study? Is a finding from a narrow population being generalized to a broader one? Is a statistically significant finding being treated as practically significant?

Statistically significant does not mean practically significant. A study with a large enough sample can find a statistically significant result for a relationship that is far too small to matter. A drug that improves an outcome by 0.2 percent in a study of 100,000 people may have a statistically significant effect that is nonetheless clinically trivial.

No single study or dataset should be the sole basis for a strong conclusion. Look for whether other researchers have studied the same question. Triangulation — finding multiple independent sources that converge on the same conclusion — is the gold standard for confidence in empirical claims. When studies disagree, that disagreement is itself important information: it suggests the phenomenon is complex, context-dependent, or not well understood.