The Central Idea

A causal question asks what would happen to an outcome if a condition were changed. An experiment approaches that question by comparing groups exposed to different assigned conditions.

\[ \boxed{ \text{assign treatments} \;\longrightarrow\; \text{measure outcomes} \;\longrightarrow\; \text{compare groups} } \]

Consider a study of whether a new review method improves quiz performance. One group uses the new method, while another uses the usual method. Both groups take the same quiz under the same conditions. The difference between their results estimates the effect of changing the review method.

The quality of that estimate depends on how the treatments are defined, how the comparison is controlled, and how the outcome is measured.

Experimental Units and Participants

An experimental unit is the smallest unit independently assigned to a treatment. Experimental units may be people, animals, plants, classrooms, plots of land, machines, stores, or digital accounts.

When the units are people, they are often called participants or subjects. Participant is generally preferred because it recognizes the person's active role in the research.

The unit of assignment matters. If entire classrooms are assigned to teaching methods, then the classrooms—not the individual students—are the independently assigned experimental units. Student outcomes are measured within those units. Treating all students as independently assigned would overstate the amount of independent information in the experiment.

Treatments

A treatment is a specific experimental condition imposed on an experimental unit. Treatment does not necessarily mean medical care. In experimental design, the word applies to any assigned condition.

Examples include:

A treatment must be described precisely enough to be implemented consistently. “Receive tutoring” is vague. “Receive two 30-minute individual tutoring sessions per week for eight weeks using the same lesson sequence” is operationally clearer.

Factors, Levels, and Treatment Combinations

A factor is an explanatory variable deliberately varied in an experiment. A level is one of the specific values or settings of that factor.

Suppose a plant-growth experiment varies fertilizer amount at three levels:

\[ 0\text{ g},\qquad 5\text{ g},\qquad 10\text{ g}. \]

Fertilizer amount is the factor, and the three quantities are its levels. If the experiment also varies watering frequency at two levels, then each treatment is a combination of one fertilizer level and one watering level. The total number of treatment combinations is

\[ 3\times 2=6. \]

This distinction is useful: the factor is what varies, a level is a setting, and a treatment is the complete assigned condition.

Control and Comparison

Control means creating a fair comparison by holding other relevant conditions as constant as practical and by including an informative comparison condition. Control does not mean that every possible influence can be eliminated.

A control group receives the comparison condition against which another treatment is evaluated. Depending on the question, the control condition might be:

The control group is not automatically a group that receives nothing. If an effective standard treatment already exists, withholding it may be unethical or scientifically unhelpful. The appropriate control must match the research question and the ethical setting.

Choosing an Appropriate Control

Control Condition What It Helps Measure Example
No-treatment control What happens without the intervention Plants receiving no fertilizer
Placebo control The effect beyond expectations and treatment experience An inactive pill resembling the study medication
Active control Performance relative to an established alternative A new therapy compared with standard care
Usual-practice control Improvement over the current process A new lesson sequence compared with the existing sequence
Baseline setting Change from a standard operating level A machine tested at its normal temperature

A useful control isolates the contrast that matters. If the research question asks whether a new process is better than the current process, a no-process control may answer the wrong question.

Placebos and the Experience of Treatment

A placebo is an inactive or comparison treatment designed to resemble the treatment being studied. A placebo helps separate the specific effect of an intervention from effects caused by expectations, attention, routine, or the experience of receiving treatment.

Placebos are most familiar in medical research, but the underlying principle is broader. If one group receives extensive staff attention while another receives none, the groups differ in more than the intended treatment. A good comparison makes the surrounding experience as similar as practical.

A placebo is not always possible. Participants can usually see whether they are using a standing desk, attending a tutoring session, or following a new exercise routine. Other design tools, including consistent procedures and blinded outcome assessment, may then become especially important.

Outcomes

An outcome is the result measured after treatment assignment. It is also called a response variable or dependent variable, depending on the field.

Outcomes may be quantitative, such as reaction time, blood pressure, crop yield, profit, or test score. They may also be categorical, such as whether a machine failed, whether a customer purchased, or whether a patient recovered.

A useful outcome is defined before results are examined. The definition should specify:

“Improvement” is not yet an operational outcome. “Change in quiz score from the pretest to the assessment administered after four weeks” is much clearer.

Primary and Secondary Outcomes

The primary outcome is the main result the experiment is designed to evaluate. Secondary outcomes provide additional information but are not the central basis of the study.

For a reading intervention, the primary outcome might be reading-comprehension score after twelve weeks. Secondary outcomes might include reading speed, vocabulary retention, attendance, and student confidence.

Naming the primary outcome in advance prevents researchers from measuring many variables and later presenting whichever one happened to look most favorable as though it had always been the central question.

Direct and Surrogate Outcomes

A direct outcome measures the result that ultimately matters. A surrogate outcome is an intermediate measurement used as a substitute because it is faster, less expensive, or easier to observe.

For example, a short-term laboratory marker may be used as a surrogate for a long-term health outcome. A click may be used as a surrogate for customer satisfaction. Surrogates can be useful, but an effect on the surrogate does not automatically imply the same effect on the final outcome of interest.

A study should state exactly what was measured and avoid silently replacing the measured outcome with a broader claim.

Comparing Outcomes

In a simple two-group experiment, the estimated treatment effect is often a difference between group outcomes. For a quantitative outcome, one common comparison is

\[ \widehat{\text{effect}} = \overline{y}_{\text{treatment}} - \overline{y}_{\text{control}}. \]

Suppose the treatment group has an average score of (84) and the control group has an average score of (79). The estimated difference is

\[ 84-79=5\text{ points}. \]

The arithmetic is simple, but the interpretation depends on the design. Was assignment fair? Were the groups treated alike apart from the intended contrast? Was the outcome measured consistently? Is the difference large enough to matter, and could it plausibly be explained by chance variation?

The Comparison We Cannot Observe Directly

For one experimental unit, the ideal causal comparison would measure its outcome under both treatment and control at the same time. If (Y_i(1)) is unit (i)'s outcome under treatment and (Y_i(0)) is its outcome under control, then its individual treatment effect would be

\[ Y_i(1)-Y_i(0). \]

Only one of those outcomes can actually be observed for the unit. A plant cannot both receive and not receive the fertilizer in the same growing period. Experiments address this problem by comparing groups designed to be similar before treatment. The control group provides an estimate of what would have happened to the treatment group without the treatment.

This is why the comparison group is not an optional extra. It supplies the missing reference needed for causal reasoning.

Example: A Tutoring Experiment

A school wants to determine whether a new tutoring program improves algebra performance. Eighty students are assigned either to the new program or to the school's usual academic-support program. After eight weeks, every student completes the same algebra assessment.

Experimental Units

The 80 students are the experimental units because treatment is assigned to individual students.

Treatments

The two treatment conditions are the new tutoring program and the usual academic-support program.

Control

The usual-support group is an active control. It answers whether the new program improves on current practice, not merely whether receiving any help is better than receiving none.

Outcome

The outcome is each student's score on the common algebra assessment after eight weeks. A stronger design would also define the scoring procedure, testing conditions, and treatment of missing assessments in advance.

Comparison

The central comparison is the difference in assessment results between the new-program group and the usual-support group.

Example: Treatments with Two Factors

A bakery tests two oven temperatures, (175^\circ\text{C}) and (190^\circ\text{C}), and three baking times, (18), (21), and (24) minutes. The factors are oven temperature and baking time. Their levels are

\[ \begin{aligned} \text{temperature: }&175^\circ\text{C},\ 190^\circ\text{C},\\ \text{time: }&18,\ 21,\ 24\text{ minutes}. \end{aligned} \]

The six treatment combinations are formed by pairing every temperature level with every time level. If product quality is rated on a fixed 100-point scale, that rating is the outcome.

A multifactor experiment can estimate the effect of each factor and examine whether the effect of one factor changes across levels of the other. For example, additional baking time may help at one temperature but harm quality at the other.

What Must Be Held Consistent?

A fair comparison keeps procedures similar except for the intended treatment contrast. Depending on the experiment, researchers may standardize:

Standardization does not require making every unit identical. It prevents the experiment itself from introducing avoidable differences between groups.

Threats to a Treatment Comparison

Unequal Additional Care

If the treatment group receives more attention, equipment, or support than the control group, the comparison mixes the intended treatment with those additional differences.

Noncompliance

Participants may not follow the assigned condition. Some may switch groups, skip sessions, or use another intervention. Assignment and actual exposure must therefore be distinguished.

Attrition

If units leave the study, especially at different rates or for different reasons across groups, the remaining groups may no longer be comparable.

Outcome Measurement Bias

If outcomes are measured differently across groups—or if a subjective rater is influenced by knowing the treatment—the measured difference may not reflect the treatment effect.

Selective Outcome Reporting

Reporting only favorable outcomes after examining the data can create a misleading impression. Primary outcomes and analysis rules should be specified before the results are known whenever possible.

Ethical Control and Treatment

A scientifically convenient control is not automatically ethical. Human participants should not be denied effective care merely to create a clean comparison. Risks should be proportionate to the value of the knowledge sought, informed consent should be obtained when required, and vulnerable groups need appropriate protection.

Ethical design also affects interpretation. If only low-risk participants can ethically enter a study, the findings may not generalize to people facing greater risks. Ethical limits are part of the study's scope, not an obstacle to be ignored.

A Design Checklist

Before an experiment begins, answer these questions explicitly:

  1. Units: Who or what will receive the assigned condition?
  2. Factors: Which explanatory variables will be deliberately varied?
  3. Levels: Which values or versions of each factor will be used?
  4. Treatments: What complete conditions will be assigned?
  5. Control: What comparison best answers the causal question?
  6. Outcome: What result will be measured, how, and when?
  7. Consistency: Which surrounding procedures must be standardized?
  8. Interpretation: What group contrast will estimate the effect?

If any answer is vague, the study is not yet operationally complete.

Common Errors

Using Treatment to Mean Only Medicine

In experimental design, treatment means any assigned condition. A price, interface, temperature, or instructional method can be a treatment.

Assuming a Control Group Receives Nothing

A control group may receive a placebo, current standard, baseline setting, or usual process. The right control depends on the comparison of interest.

Leaving the Outcome Vague

Words such as success, improvement, quality, and engagement require a stated measurement rule. Without operational definitions, the study cannot be reproduced or interpreted precisely.

Changing More Than One Intended Condition

If treatment groups differ in several uncontrolled ways, the experiment may not reveal which difference produced the outcome.

Measuring Only After Treatment Without a Comparison

An outcome observed after treatment is not automatically a treatment effect. A causal effect is a comparison with a meaningful alternative condition.

Guided Practice

Problem 1

A company assigns customer-support teams to use either the current ticket system or a redesigned system. It records the median time required to resolve a ticket during the following month. Identify the units, treatments, control, and outcome.

Show the solution

The support teams are the experimental units because the system is assigned by team. The treatments are the current and redesigned ticket systems. The current system is the usual-practice control. The outcome is the median ticket-resolution time for each team during the following month.

Problem 2

A plant experiment uses two light levels and three watering levels. How many treatment combinations are there? Name the factors.

Show the solution

The factors are light level and watering level. There are (2\times 3=6) treatment combinations.

Problem 3

A study tests a new pain medication against an inactive pill. The treatment group receives weekly phone calls from a nurse, while the placebo group receives no calls. What is wrong with the comparison?

Show the solution

The groups differ in both medication and staff attention. Any outcome difference could reflect the medication, the weekly contact, or both. The surrounding experience should be made comparable, such as by providing the same scheduled contact to both groups.

Problem 4

An educational experiment lists “student engagement” as its outcome. Explain what additional information is needed.

Show the solution

Engagement needs an operational definition: for example, attendance rate, completed activities, time on task measured by a stated method, or a score on a validated survey. The design should also state when the measurement is made and how multiple measurements or missing data are handled.

Problem 5

A treatment group averages (42) units on an outcome, and the control group averages (38). Compute the observed difference and state one question that must be answered before interpreting it causally.

Show the solution

The observed difference is (42-38=4) units. Before interpreting it causally, ask whether treatment was assigned fairly, whether other procedures were held consistent, whether attrition or noncompliance occurred, whether the outcome was measured consistently, or whether chance variation could plausibly explain the difference.

A Final Experimental Map

Reduce an experiment to four connected questions:

\[ \boxed{ \text{Who receives what condition, compared with what, measured how?} } \]

The answers identify the experimental units, treatments, control, and outcome. If those pieces are clear, the causal comparison becomes visible. If one is missing, the interpretation is incomplete.

Summary

Next Step

Treatments and outcomes define the comparison, but the groups must also be formed fairly. The next lesson develops random assignment—the chance mechanism used to prevent systematic differences from determining who receives each treatment.