The Central Idea
A dependency diagram is a graphical representation of a probability model.
A typical directed connection
\[ A\longrightarrow B \]means that the model treats the distribution of \(B\) as depending directly on \(A\).
The arrow organizes conditional structure:
\[ \boxed{ P(B\mid A) } \]rather than implying that \(A\) and \(B\) must be independent or that the arrow alone proves a causal relationship.
Nodes
Each quantity in a dependency diagram is represented by a node.
A node may represent:
- an event;
- a categorical variable;
- a binary state;
- a numerical random variable;
- an observed measurement;
- a hidden or unobserved state.
For example, the three nodes
\[ W,\qquad T,\qquad D \]might represent weather, traffic, and delivery time.
Directed Edges
A directed edge is an arrow between two nodes:
\[ A\longrightarrow B. \]It indicates that \(B\) is modeled conditionally on \(A\).
The node \(A\) is called a parent of \(B\), and \(B\) is a child of \(A\).
Parents and Children
If
\[ A\longrightarrow B, \]then:
- \(A\) is a parent of \(B\);
- \(B\) is a child of \(A\).
A node can have several parents and several children.
A Simple Chain
Consider
\[ A\longrightarrow B\longrightarrow C. \]This is a chain.
The model says that \(B\) depends directly on \(A\), and \(C\) depends directly on \(B\).
The relationship between \(A\) and \(C\) is indirect through \(B\).
Direct and Indirect Dependence
In the chain
\[ A\longrightarrow B\longrightarrow C, \]there is no direct arrow from \(A\) to \(C\).
Yet \(A\) and \(C\) may still be probabilistically associated because information can pass through \(B\).
Dependency diagrams therefore distinguish:
- direct modeled dependence;
- indirect dependence through intermediate nodes.
Example: Weather, Traffic, and Travel Time
Consider
\[ W\longrightarrow T\longrightarrow R, \]where:
- \(W\) = weather;
- \(T\) = traffic level;
- \(R\) = travel time.
Weather may affect traffic, and traffic may affect travel time.
Weather can therefore be associated with travel time even if the diagram contains no direct arrow
\[ W\longrightarrow R. \]Conditional Independence in a Chain
In a simple chain,
\[ A\longrightarrow B\longrightarrow C, \]the model may assert that once \(B\) is known, \(A\) provides no additional information about \(C\).
This conditional-independence statement is written
\[ \boxed{ C\perp A\mid B }. \]Equivalently,
\[ P(C\mid A,B)=P(C\mid B) \]when the relevant conditional probabilities are defined.
Conditional Independence Does Not Mean Ordinary Independence
In the chain
\[ A\longrightarrow B\longrightarrow C, \]\(A\) and \(C\) may be dependent marginally.
But after conditioning on \(B\), they may become independent.
Thus,
\[ A\not\!\perp C \]can coexist with
\[ A\perp C\mid B. \]A Common Cause
Consider
\[ A\longleftarrow B\longrightarrow C. \]Here \(B\) is a common parent of \(A\) and \(C\).
This pattern is sometimes called a fork or common-cause structure.
Common Causes Can Create Association
In
\[ A\longleftarrow B\longrightarrow C, \]observing \(A\) can provide information about \(B\), which in turn can provide information about \(C\).
Therefore, \(A\) and \(C\) may be associated even though neither directly points to the other.
Conditioning on the Common Cause
In a simple common-cause model,
\[ A\longleftarrow B\longrightarrow C, \]conditioning on \(B\) may separate \(A\) and \(C\):
\[ \boxed{ A\perp C\mid B }. \]Once the common source of variation is known, one child may provide no additional information about the other.
Example: Weather as a Common Cause
Consider
\[ \text{Weather} \longrightarrow \text{Umbrella Use} \]and
\[ \text{Weather} \longrightarrow \text{Traffic Delay}. \]Umbrella use and traffic delay may be associated because both are influenced by weather.
If weather is already known, umbrella use may add little or no information about traffic delay in the simplified model.
A Common Effect
Consider
\[ A\longrightarrow C\longleftarrow B. \]Here \(C\) has two parents.
This structure is often called a collider or common-effect structure.
Common-Effect Structure Behaves Differently
In
\[ A\longrightarrow C\longleftarrow B, \]\(A\) and \(B\) can be independent before information about \(C\) is known.
But conditioning on the common effect \(C\) can create dependence between \(A\) and \(B\).
Example: Two Causes of an Alarm
Suppose an alarm \(A\) can be triggered by either smoke \(S\) or a sensor fault \(F\):
\[ S\longrightarrow A\longleftarrow F. \]Before observing the alarm, smoke and sensor fault may be modeled as independent.
If the alarm sounds and then we learn that there is no smoke, the sensor fault becomes more plausible.
Conditioning on the common effect can connect the two possible causes.
Explaining Away
In a common-effect structure,
\[ A\longrightarrow C\longleftarrow B, \]one cause can reduce the need for the other cause after the common effect is observed.
This phenomenon is often called explaining away.
A Branching Structure
A node may have several children:
\[ A\longrightarrow B, \qquad A\longrightarrow C, \qquad A\longrightarrow D. \]The variable \(A\) provides direct conditioning information for all three child nodes.
A Converging Structure
A node may also have several parents:
\[ A\longrightarrow D, \qquad B\longrightarrow D, \qquad C\longrightarrow D. \]Then the model may require a conditional distribution of the form
\[ \boxed{ P(D\mid A,B,C) }. \]Parent Sets
The collection of parents of a node \(X\) is often written
\[ \operatorname{Pa}(X). \]For example, if
\[ A\longrightarrow D, \qquad B\longrightarrow D, \]then
\[ \operatorname{Pa}(D)=\{A,B\}. \]Root Nodes
A node with no parents is often called a root node.
In
\[ A\longrightarrow B\longrightarrow C, \]\(A\) is a root node.
Its probability distribution is specified without conditioning on another node in the diagram.
Leaf Nodes
A node with no children is often called a leaf node.
In
\[ A\longrightarrow B\longrightarrow C, \]\(C\) is a leaf node.
Ancestors and Descendants
If there is a directed path
\[ A\longrightarrow\cdots\longrightarrow D, \]then \(A\) is an ancestor of \(D\), and \(D\) is a descendant of \(A\).
Parent and child relationships are immediate; ancestor and descendant relationships can span several arrows.
Directed Paths
A directed path follows arrows consistently in their indicated direction.
For example,
\[ A\longrightarrow B\longrightarrow C\longrightarrow D \]contains a directed path from \(A\) to \(D\).
Dependency Diagrams Organize Conditional Models
If a node \(X\) has parents
\[ A,B, \]the diagram indicates that its local behavior is described using a conditional model such as
\[ P(X\mid A,B). \]A full Bayesian network will formalize how all such local pieces combine.
Arrows Do Not Automatically Mean Deterministic Dependence
An arrow
\[ A\longrightarrow B \]does not mean that \(A\) completely determines \(B\).
It may simply mean that the probability distribution of \(B\) changes with \(A\).
For example,
\[ P(B=1\mid A=1) \]may differ from
\[ P(B=1\mid A=0) \]without either probability being \(0\) or \(1\).
Example: Binary Parent and Child
Suppose
\[ A\longrightarrow B \]with
\[ P(B=1\mid A=1)=0.80 \]and
\[ P(B=1\mid A=0)=0.30. \]The distribution of \(B\) changes with \(A\), so the diagram represents dependence.
No Arrow Does Not Always Mean Marginal Independence
In a structured dependency model, the absence of a direct arrow means that the model does not specify a direct parent-child dependence.
But two nodes without a direct arrow can still be associated through other paths.
For example,
\[ A\longrightarrow B\longrightarrow C \]has no direct arrow from \(A\) to \(C\), yet \(A\) and \(C\) may be dependent.
Separate Graph Structure from Numerical Probabilities
A dependency diagram describes which conditional relationships are modeled.
It does not by itself provide the numerical probabilities.
A complete probability model also needs quantities such as
\[ P(A), \qquad P(B\mid A), \qquad P(C\mid B). \]Example: Diagram Plus Numbers
Consider
\[ R\longrightarrow M\longrightarrow S, \]where:
- \(R\) = rain;
- \(M\) = muddy field;
- \(S\) = slipping event.
A numerical model might specify
\[ P(R)=0.20, \] \[ P(M\mid R), \] \[ P(M\mid R^c), \]and probabilities of slipping conditional on whether the field is muddy.
Information Flow Is Not the Same as Arrow Reversal
Even if the diagram contains
\[ A\longrightarrow B, \]observing \(B\) can still provide information about \(A\).
Bayes' theorem allows us to calculate
\[ P(A\mid B) \]from information such as
\[ P(B\mid A). \]The graphical arrow describes model structure, not the only direction in which inference can occur.
Example: Cause-to-Observation Structure
Suppose
\[ D\longrightarrow T, \]where \(D\) is an underlying condition and \(T\) a test result.
The forward model may specify
\[ P(T\mid D). \]After observing \(T\), Bayesian inference can still calculate
\[ P(D\mid T). \]Inference can move opposite the arrow even though the model factorization follows the arrow.
Observed and Hidden Nodes
Some diagrams distinguish:
- observed nodes, whose values are known;
- hidden nodes, whose values are unknown or latent.
Conditioning on observed nodes changes probabilities assigned to hidden nodes.
Example: Hidden State and Measurement
Consider
\[ H\longrightarrow M, \]where \(H\) is a hidden system state and \(M\) is an observed measurement.
The model specifies how measurements behave under each hidden state.
Once \(M\) is observed, Bayes' theorem updates the probability distribution over \(H\).
Multiple Observations from One Hidden State
Consider
\[ H\longrightarrow M_1, \qquad H\longrightarrow M_2. \]The hidden state \(H\) is a common parent of two measurements.
The measurements may be associated because both depend on the same hidden state.
Conditioning on the Hidden State Can Separate Measurements
In a simplified common-parent model,
\[ M_1\longleftarrow H\longrightarrow M_2, \]the model may assert
\[ M_1\perp M_2\mid H. \]Once \(H\) is known, one measurement may add no information about the other.
Conditional Independence Is a Modeling Claim
A dependency diagram encodes assumptions about which variables become independent after certain others are known.
These assumptions are not automatic facts of nature.
They must be justified by the intended model, data-generating process, or domain structure.
Dependency Diagrams Reduce Complexity
Without structure, a probability model involving many variables may require a very large joint distribution.
A dependency diagram can express the model using smaller local conditional relationships.
This reduction is one of the major reasons graphical probability models are useful.
Example: Four Variables
Consider
\[ A\longrightarrow B, \] \[ A\longrightarrow C, \] \[ B\longrightarrow D, \qquad C\longrightarrow D. \]Here:
- \(A\) is a parent of \(B\) and \(C\);
- \(B\) and \(C\) are parents of \(D\);
- \(A\) is an ancestor of \(D\);
- \(D\) is a descendant of \(A\).
Reading the Local Conditional Structure
The previous diagram suggests local probability components of the form
\[ P(A), \] \[ P(B\mid A), \] \[ P(C\mid A), \] \[ P(D\mid B,C). \]The next lesson on Bayesian networks will show how these pieces can combine into a joint probability model.
Acyclic Structure
Many dependency diagrams used for Bayesian networks are directed and acyclic.
A directed cycle would look like
\[ A\longrightarrow B\longrightarrow C\longrightarrow A. \]A directed acyclic graph does not contain such a cycle.
Why Cycles Matter
Acyclic structure allows the variables to be ordered so that each node is conditioned on earlier parent nodes.
This makes a clean probability factorization possible.
The full factorization rule belongs to the next lesson.
Dependency Diagrams Are Not Ordinary Flowcharts
A process flowchart often shows temporal or procedural steps.
A dependency diagram represents probabilistic conditioning structure.
The two may sometimes look similar, but they answer different questions.
Time Order Can Help, but Is Not the Whole Story
In some models, arrows follow time:
\[ X_1\longrightarrow X_2\longrightarrow X_3. \]But a dependency diagram is fundamentally about conditional structure, not merely chronology.
Two variables can occur at different times without requiring an arrow between them in the model.
Arrows and Causality
In some carefully designed graphical models, arrows are intended to carry a causal interpretation.
In other models, arrows merely encode probabilistic dependence or a useful factorization.
Therefore, an arrow should not be interpreted as proof that one variable causes another unless causal meaning is explicitly justified.
Example: Correlation Without Direct Arrow
Suppose
\[ Z\longrightarrow X, \qquad Z\longrightarrow Y. \]Then \(X\) and \(Y\) can be associated through their common parent \(Z\) even though the diagram has no arrow
\[ X\longrightarrow Y \]or
\[ Y\longrightarrow X. \]Example: Conditioning Can Remove an Association
In
\[ X\longleftarrow Z\longrightarrow Y, \]\(X\) and \(Y\) may be associated because both depend on \(Z\).
After conditioning on \(Z\), that association may disappear:
\[ X\perp Y\mid Z. \]Example: Conditioning Can Create an Association
In
\[ X\longrightarrow Z\longleftarrow Y, \]\(X\) and \(Y\) may begin independent.
Conditioning on \(Z\) can create dependence:
\[ X\not\!\perp Y\mid Z. \]This is the key contrast between a common cause and a common effect.
Three Basic Patterns to Recognize
The three elementary directed structures are:
\[ \boxed{ A\longrightarrow B\longrightarrow C } \]chain,
\[ \boxed{ A\longleftarrow B\longrightarrow C } \]common cause, and
\[ \boxed{ A\longrightarrow B\longleftarrow C } \]common effect.
Chain and Common Cause Have Similar Conditioning Behavior
In the simple patterns
\[ A\longrightarrow B\longrightarrow C \]and
\[ A\longleftarrow B\longrightarrow C, \]conditioning on the middle node \(B\) commonly blocks the association between the endpoints in the modeled structure.
Common Effect Has the Opposite Pattern
In
\[ A\longrightarrow B\longleftarrow C, \]the endpoints may be independent before conditioning on \(B\).
Conditioning on \(B\) can open an association between them.
This reversal is one of the most important ideas in graphical probability.
Dependency Diagrams and Evidence
When a node is observed, the information can change probabilities elsewhere in the graph.
Evidence may affect:
- parents;
- children;
- ancestors;
- descendants;
- other connected nodes.
The exact update depends on the graph structure and the numerical probability model.
Example: Evidence Propagating Backward
Suppose
\[ A\longrightarrow B\longrightarrow C. \]Observing \(C\) can change beliefs about \(B\), and therefore about \(A\).
The arrows do not prevent inference from moving backward through Bayes' theorem.
Example: Evidence Propagating Forward
In the same chain,
\[ A\longrightarrow B\longrightarrow C, \]observing \(A\) can change the distribution of \(B\), which can then change the distribution of \(C\).
Graph structure helps organize how these updates are related.
Dependency Diagrams and Conditional Expectation
A diagram can also organize conditional expectations.
If
\[ A\longrightarrow X, \]where \(X\) is numerical, we may compare
\[ E[X\mid A=a_1] \]with
\[ E[X\mid A=a_2]. \]The diagram indicates that the distribution, and therefore possibly the mean, of \(X\) changes with \(A\).
Example: Risk Class and Loss
Suppose
\[ R\longrightarrow L, \]where \(R\) is risk class and \(L\) is annual loss.
The model may specify
\[ E[L\mid R=\text{low}] \]and
\[ E[L\mid R=\text{high}]. \]The dependency diagram organizes which conditioning variable determines the local loss distribution.
Common Error: Reading Every Arrow as Proof of Causation
An arrow can encode a conditional dependence assumption without proving a causal mechanism.
Causal interpretation requires additional assumptions beyond the diagram itself.
Common Error: Assuming No Arrow Means Independence
Two nodes may lack a direct arrow and still be dependent through another path.
For example,
\[ A\longrightarrow B\longrightarrow C \]contains no direct \(A\)-to-\(C\) edge, yet \(A\) and \(C\) may be associated.
Common Error: Ignoring Indirect Paths
Dependency can travel through intermediate nodes.
When interpreting a diagram, inspect full paths rather than only adjacent nodes.
Common Error: Treating Conditional Independence as Marginal Independence
The statement
\[ A\perp C\mid B \]does not imply
\[ A\perp C. \]Conditioning can fundamentally change dependence relationships.
Common Error: Treating Common Causes and Common Effects the Same Way
In a common-cause structure,
\[ A\longleftarrow B\longrightarrow C, \]conditioning on \(B\) can remove dependence between \(A\) and \(C\).
In a common-effect structure,
\[ A\longrightarrow B\longleftarrow C, \]conditioning on \(B\) can create dependence between \(A\) and \(C\).
Common Error: Assuming an Arrow Means Determinism
A directed edge represents conditional probability structure.
The child can remain uncertain even when all parent values are known.
Common Error: Treating the Diagram as a Complete Numerical Model
The diagram specifies structure.
Numerical probability tables or distributions are still needed to perform calculations.
Common Error: Reversing Arrows to Perform Inference
Bayesian inference can move opposite an arrow without changing the graph.
The graph represents the model structure; Bayes' theorem handles reversed conditional questions.
A Reliable Procedure for Reading a Dependency Diagram
- Identify every node and what it represents.
- Identify the parent set of each node.
- Trace direct and indirect paths.
- Locate chains, common causes, and common effects.
- Determine which nodes are observed and which are hidden.
- Ask what conditional-independence claims the model is making.
- Separate structural assumptions from numerical probabilities.
- Avoid assigning causal meaning unless the model explicitly supports it.
A Reliable Procedure for Translating a Diagram into Conditional Terms
- Start with a root node and write its unconditional distribution.
- For each non-root node, list its parents.
- Write a conditional distribution given those parents.
- Check whether two nonadjacent nodes are connected indirectly.
- Use conditional-independence statements only when supported by the diagram's modeled structure.
- Reserve full joint factorization for the Bayesian-network formulation.
Example: Full Diagram Reading
Consider
\[ W\longrightarrow T, \] \[ W\longrightarrow R, \] \[ T\longrightarrow D, \qquad R\longrightarrow D. \]Interpret the nodes as:
- \(W\) = weather;
- \(T\) = traffic;
- \(R\) = road condition;
- \(D\) = delivery delay.
Then:
- \(W\) is a root node;
- \(T\) and \(R\) are children of \(W\);
- \(D\) has parents \(T\) and \(R\);
- \(W\) is an ancestor of \(D\);
- \(D\) is a descendant of all three upstream nodes.
Local Probability Components of the Example
The diagram suggests quantities such as
\[ P(W), \] \[ P(T\mid W), \] \[ P(R\mid W), \] \[ P(D\mid T,R). \]These local components will be combined systematically in the next lesson.
Practice 1: Identify Parent and Child
In
\[ A\longrightarrow B, \]identify the parent and child.
Practice 2: Identify the Pattern
Classify
\[ A\longrightarrow B\longrightarrow C \]as a chain, common cause, or common effect.
Practice 3: Common Cause
In
\[ A\longleftarrow B\longrightarrow C, \]which node is the common parent?
Practice 4: Common Effect
In
\[ A\longrightarrow C\longleftarrow B, \]which node is the common effect?
Practice 5: Conditional Independence in a Chain
In the simplified model
\[ A\longrightarrow B\longrightarrow C, \]write the conditional-independence relationship commonly encoded between \(A\) and \(C\) once \(B\) is known.
Practice 6: Parent Set
Suppose
\[ A\longrightarrow D, \qquad B\longrightarrow D, \qquad C\longrightarrow D. \]Write \(\operatorname{Pa}(D)\).
Practice 7: Root and Leaf
In
\[ A\longrightarrow B\longrightarrow C, \]identify the root node and the leaf node.
Practice 8: Indirect Dependence
In
\[ A\longrightarrow B\longrightarrow C, \]does the absence of a direct arrow from \(A\) to \(C\) guarantee that \(A\) and \(C\) are independent? Explain.
Practice 9: Collider Conditioning
In
\[ A\longrightarrow C\longleftarrow B, \]what can happen to the dependence between \(A\) and \(B\) after conditioning on \(C\)?
Practice 10: Identify the Error
A student sees
\[ X\longrightarrow Y \]and says, “This proves that \(X\) causes \(Y\).”
Explain why that conclusion is too strong from the diagram alone.
Practice Answers
Practice 1
\(A\) is the parent and \(B\) is the child.
Practice 2
The structure
\[ A\longrightarrow B\longrightarrow C \]is a
\[ \boxed{\text{chain}}. \]Practice 3
The common parent is
\[ \boxed{B}. \]Practice 4
The common effect is
\[ \boxed{C}. \]Practice 5
The common conditional-independence statement is
\[ \boxed{ A\perp C\mid B }. \]Practice 6
\[ \boxed{ \operatorname{Pa}(D)=\{A,B,C\} }. \]Practice 7
The root node is
\[ \boxed{A}, \]and the leaf node is
\[ \boxed{C}. \]Practice 8
No. \(A\) and \(C\) can be dependent through the indirect path
\[ A\longrightarrow B\longrightarrow C. \]No direct edge means no direct parent-child relationship, not necessarily marginal independence.
Practice 9
Conditioning on the common effect \(C\) can create dependence between \(A\) and \(B\), even if they were independent before conditioning.
Practice 10
A directed edge can encode a conditional-dependence or factorization assumption without proving a causal mechanism.
A causal interpretation requires additional assumptions and justification.
What to Remember
A dependency diagram uses nodes and directed edges to organize conditional probability structure.
The three basic patterns are
\[ \boxed{ A\longrightarrow B\longrightarrow C } \]chain,
\[ \boxed{ A\longleftarrow B\longrightarrow C } \]common cause, and
\[ \boxed{ A\longrightarrow B\longleftarrow C } \]common effect.
- Nodes represent random quantities or events.
- Directed edges identify modeled parent-child relationships.
- No direct edge does not automatically imply marginal independence.
- Chains and common causes can become conditionally independent when the middle node is known.
- Conditioning on a common effect can create dependence between its parents.
- Dependency diagrams describe structure but do not supply numerical probabilities by themselves.
- Inference can move opposite an arrow through Bayes' theorem.
- Arrows should not automatically be interpreted as proof of causation.
- Directed acyclic structure prepares the way for Bayesian-network factorization.
The central habit is: read the graph as a map of conditional structure—trace parents, children, paths, and what changes when information is conditioned upon.
Where We Go Next
Dependency diagrams organize which variables directly condition which others. The next step is to turn that graphical structure into a complete joint probability model.
The next lesson, Introductory Bayesian Networks, develops directed acyclic graphs, local conditional distributions, joint factorization, and basic probabilistic inference across connected variables.