Content
Summary
Directed Graphical Models encode conditional independence via d-separation, where a graph G is an I-map for distribution p if all graph-encoded independencies hold in p. The d-separation algorithm determines independence by checking if all paths between variable sets are blocked by observed evidence, following specific rules for chains, forks, and colliders.
Intuition
Think of a Directed Graphical Model as a map that tells you which variables are independent of each other given certain observations. This map is valid (an I-map) if it doesn't claim any independencies that are actually false in the real data. To navigate this map, we use the 'Bayes Ball' algorithm: imagine a ball rolling along paths between variables. If the ball can reach from A to B, they are dependent; if the path is blocked, they are independent. The rules for blocking depend on the structure: in a chain or fork, observing the middle node blocks the ball. In a collider (v-structure), the path is blocked unless the collider itself or one of its descendants is observed, which creates a phenomenon called 'explaining away' where observing the effect makes the causes dependent.
Key equations
A is independent of B given C in graph G
G is an I-map for p if all graph independencies hold in p
Definition of CI in a DAG via d-separation
Independence of x and z given y in a chain or fork
Marginal independence of x and z in a v-structure
Joint distribution showing dependence of x and z given y in a v-structure
Derivation
- A graph G is defined as an I-map for distribution p if I(G) is a subset of I(p), meaning the graph's asserted independencies are true for p.
- Conditional independence in a DAG is determined by d-separation, which checks if all paths between sets A and B are blocked by evidence E.
- A path is blocked if it contains a chain or fork where the middle node is in E, or a fork where the root is in E.
- A path is blocked if it contains a collider where the collider is not in E and no descendant of the collider is in E.
- In a chain X->Y->Z or fork X<-Y->Z, observing Y blocks the path, making X and Z independent given Y.
- In a v-structure X->Y<-Z, X and Z are marginally independent but become dependent when Y is observed (explaining away).
- Boundary conditions apply: if a child of a node is observed, the node is effectively observed, causing the ball to bounce back.
Worked example
Consider a v-structure X -> Y <- Z. X and Z are independent marginally. If we observe Y, are X and Z independent? Explain using the concept of 'explaining away'.
- Identify the structure: X -> Y <- Z is a collider (v-structure).
- Recall the rule: Colliders block paths unless the collider or its descendant is observed.
- Since Y is observed, the path between X and Z is unblocked.
- This leads to dependence between X and Z given Y, known as 'explaining away' or Berkson's paradox.
- For example, if X and Z are independent causes of Y, observing Y makes X and Z dependent because knowing one cause explains the effect, reducing the need for the other.
Answer No, X and Z are not independent given Y. Observing the collider Y opens the path, creating dependence between X and Z through 'explaining away'.
Common mistakes
- Assuming colliders block paths when unobserved; they actually open paths when observed.
- Thinking that observing a child of a node is equivalent to observing the node itself without considering the 'bounce back' rule.
- Confusing marginal independence with conditional independence in v-structures; parents are independent marginally but dependent given the child.