7  m-separation and graphical models

These notes were automatically generated by AI and lightly reviewed by the lecturer. They may still contain mistakes and factual errors.

We can now finish the graphical rule for conditional independence in Gaussian linear systems. The proof turns on a useful idea: marginalization compresses walks but preserves the relations they express. We will then replace the infinite collection of walks by a path criterion, define conditional independence without Gaussian assumptions, and introduce three ways for a graph to define a statistical model.

7.1 Finishing the Gaussian separation theorem

7.1.1 Invariance of graphical relations

Recall the proof strategy from the last lecture (Section 6.4.3). To study two variables \(V_j,V_k\) given \(L\), retain only

\[\tilde V=\{V_j,V_k\}\cup L, \qquad \tilde G=\operatorname{margin}_{\tilde V}(G).\]

In this smaller graph, conditioning on \(L\) means conditioning on every other retained variable. The collider-connection lemma (Lemma 6.6) from Lecture 6 can therefore finish the proof—provided that the original m-separation condition becomes the absence of a collider connection in \(\tilde G\).

The earlier calculation (Equation (6.1)) expanded an off-diagonal entry of the inverse covariance matrix into four terms. If \(V_j\) and \(V_k\) are not collider-connected, every one of those terms vanishes. Our task here is to link that collider criterion in the small graph to m-separation in the original graph.

Think of marginalization as a map from walks to walks, generated by

\[ \operatorname{marg}_{\tilde V}:\quad W\left[\tilde V \left\{\begin{matrix}\rdpath\\\ldpath\\\trek\end{matrix}\right\} \tilde V\mid\tilde V\text{ in }G\right] \longmapsto W\left[\tilde V \left\{\begin{matrix}\to\\\leftarrow\\\leftrightarrow\end{matrix}\right\} \tilde V\text{ in }\tilde G\right]. \]

Here \(W[A\rdpath B\mid C\text{ in }G]\) denotes the directed walks from \(A\) to \(B\) that are not blocked by \(C\). Each directed walk whose internal vertices lie outside \(\tilde V\) maps to one directed edge; each trek with this property maps to one bidirected edge. Concatenations of these pieces become walks in the marginal graph.

Lemma 7.1.

Let \(G\in\mathbb{G}(V)\), let \(L\subseteq\tilde V\subseteq V\), and let \(\tilde G=\operatorname{margin}_{\tilde V}(G)\). Marginalization preserves unblocked directed walks and treks:

\[ \operatorname{marg}_{\tilde V}\!\left( W\left[\tilde V \left\{\begin{matrix}\rdpath\\\ldpath\\\trek\end{matrix}\right\} \tilde V\mid L\text{ in }G\right]\right) = W\left[\tilde V \left\{\begin{matrix}\rdpath\\\ldpath\\\trek\end{matrix}\right\} \tilde V\mid L\text{ in }\tilde G\right]. \]

Proof.

Put \(M=\tilde V\setminus L\). Working in \(G\), decompose a directed walk at its successive visits to \(\tilde V\):

\[ \begin{aligned} &W[\tilde V\rdpath\tilde V\mid L]\\ &\quad=W[\tilde V\rdpath\tilde V\mid\tilde V]\\ &\qquad+W[\tilde V\rdpath M\mid\tilde V] \left(\sum_{r\geq0}W[M\rdpath M\mid\tilde V]^r\right) W[M\rdpath\tilde V\mid\tilde V]. \end{aligned} \]

Here multiplication concatenates walk pieces and the sum collects alternatives. The first term crosses no retained non-endpoint. In the second, the retained non-endpoints lie in \(M\), not in \(L\). Marginalization turns each piece between consecutive retained vertices into one directed edge. In \(\tilde G\), the same decomposition becomes

\[ \begin{aligned} &\operatorname{marg}_{\tilde V} \bigl(W[\tilde V\rdpath\tilde V\mid L\text{ in }G]\bigr)\\ &\quad=W[\tilde V\to\tilde V]\\ &\qquad+W[\tilde V\to M] \left(\sum_{r\geq0}W[M\to M]^r\right) W[M\to\tilde V]\\ &\quad=W[\tilde V\rdpath\tilde V\mid L]. \end{aligned} \]

The trek argument has the same structure. Start at the top of a trek and move down each side until first reaching \(\tilde V\). The middle part is a trek whose non-endpoints are outside \(\tilde V\); the two remaining sides are directed walks. In \(G\),

\[ \begin{aligned} &W[\tilde V\trek\tilde V\mid L]\\ &\quad=\bigl(I+W[\tilde V\ldpath\tilde V\mid L]\bigr)\\ &\qquad{}\cdot W[\tilde V\trek\tilde V\mid\tilde V] \bigl(I+W[\tilde V\rdpath\tilde V\mid L]\bigr). \end{aligned} \]

Here \(I\) collects zero-length walks, allowing either directed side to be empty. The middle term becomes one bidirected edge under marginalization, while the two side terms map to the corresponding directed-walk factors in \(\tilde G\). Thus, in \(\tilde G\),

\[ \begin{aligned} &\operatorname{marg}_{\tilde V} \bigl(W[\tilde V\trek\tilde V\mid L\text{ in }G]\bigr)\\ &\quad=\bigl(I+W[\tilde V\ldpath\tilde V\mid L]\bigr)\\ &\qquad{}\cdot W[\tilde V\leftrightarrow\tilde V] \bigl(I+W[\tilde V\rdpath\tilde V\mid L]\bigr)\\ &\quad=W[\tilde V\trek\tilde V\mid L]. \end{aligned} \]

\(\square\)

These formulas are more than walk algebra. Under a causal reading, an unblocked directed walk says that some causal route avoids \(L\), while an unblocked trek records shared randomness not blocked by \(L\). Whether either relation holds depends only on the retained variables and \(L\), not on latent variables that have been marginalized out.

7.1.2 From treks to m-connection

Definition 7.2.

Vertices are t-connected given \(L\) if they can be joined by a concatenation of one or more unblocked treks, meeting at colliders in \(L\).

Lemma 7.3.

For all disjoint \(J,K,L\subseteq\tilde V\subseteq V\),

\[ J\trek *\trek K\mid L\text{ in }G \quad\Longleftrightarrow\quad J\trek *\trek K\mid L\text{ in }\tilde G. \]

Proof.

Each constituent trek is preserved by marginalization (Lemma 7.1), and the vertices where they meet remain in \(L\). \(\square\)

Definition 7.4.

Now restrict attention to canonical directed mixed graphs: graphs containing a bidirected loop at every vertex. Write their class as \(\mathbb{G}^{*}(V)\).

Lemma 7.5.

Let \(G\in\mathbb G^*(V)\). For disjoint \(\{V_j\},\{V_k\},L\subseteq V\), t-connection and m-connection are equivalent:

\[ V_j\trek *\trek V_k\mid L \quad\Longleftrightarrow\quad V_j\mconn V_k\mid L. \]

Proof.

The \(\Rightarrow\) direction is immediate because a concatenation of treks is a walk. For the \(\Leftarrow\) direction, split an m-connecting walk into its collider-free arcs. An arc that is not already a trek consists of one or two directed walks pointing away from a top vertex. Add the bidirected loop at that vertex; the arc becomes a trek without changing whether it is blocked. Doing this to every arc produces the required sequence of treks. \(\square\)

7.1.3 The conditional-independence theorem

We can now prove the result stated at the end of Lecture 6 (Theorem 6.9).

Theorem 7.6.

Suppose \(V\) follows a Gaussian linear system on \(G\in\mathbb{G}_A(V)\) with noise covariance \(\Lambda\succ0\). For all disjoint \(J,K,L\subseteq V\),

\[ \newcommand{\independent}{\mathrel{\rlap{\perp}\mkern2mu\perp}} \text{not }J\mconn K\mid L\text{ in }G \quad\Longrightarrow\quad J\independent K\mid L. \]

Here \(J,K,L\) also denote the corresponding random subvectors. Positive definiteness forces every noise variance to be positive, so every bidirected loop is present and \(G\) is canonical.

Proof.

For a jointly Gaussian vector it is enough to establish the result for each pair \(V_j\in J\) and \(V_k\in K\). Put \(\tilde V=\{V_j,V_k\}\cup L\) and \(\tilde G=\operatorname{margin}_{\tilde V}(G)\). The graphical part of the argument, using Lemma 7.5 and Lemma 7.3, is

\[ \begin{aligned} \text{not }V_j\mconn V_k\mid L\text{ in }G &\Longleftrightarrow \text{not }V_j\trek *\trek V_k\mid L\text{ in }G\\ &\Longleftrightarrow \text{not }V_j\trek *\trek V_k\mid L\text{ in }\tilde G\\ &\Longleftrightarrow \text{not }V_j\mconn V_k\mid L\text{ in }\tilde G\\ &\Longleftrightarrow \text{not }V_j\colliderconn V_k\text{ in }\tilde G. \end{aligned} \]

For the last equivalence, trim any repeated endpoint visits. The remaining non-endpoints lie in \(L\), so the walk is unblocked exactly when they are all colliders. Marginalization preserves the Gaussian linear system (Theorem 6.3), so the collider-connection lemma from Lecture 6 (Lemma 6.6) gives \(V_j\independent V_k\mid L\). \(\square\)

The useful heuristic is to project onto the smallest graph containing the variables of interest. There, a general m-separation question reduces to the collider criterion already read from the inverse covariance matrix.

That proof was a little longer than expected. It is mathematically clean, but it also exposes a practical problem: walks may go back and forth and loop around arbitrarily many times.

7.2 Checking m-separation with paths

7.2.1 The path criterion

The walk formulation is convenient for proofs, but it looks impractical: walks may repeat vertices and there are infinitely many of them. A path is a walk with no repeated vertices, so a finite graph has only finitely many paths.

Recall from Lecture 6 (Definition 6.7) the standard, slightly weaker notion of blocking: a path is ancestrally blocked by \(L\) if either

1. it contains a non-colliding non-endpoint in \(L\); or
2. it contains a collider that is neither in \(L\) nor an ancestor of any vertex in \(L\).

Equivalently, a collider is open when it or one of its descendants is in \(L\).

Proposition 7.7.

Let \(G\in\mathbb{G}^{*}(V)\) and let \(J,K,L\subseteq V\) be disjoint. Then

\[ \begin{aligned} J\mconn K\mid L \quad\Longleftrightarrow\quad &\text{some path from $V_j\in J$ to $V_k\in K$}\\ &\text{is not ancestrally blocked by $L$.} \end{aligned} \]

This turns m-separation into a finite search problem.

7.2.2 Example: academics, sport and enrolment

Suppose academic and sport rankings both affect admission, and admission is required for enrolment.

Academic and sport rankings both point to admission, which points to enrolment. Directed edges are blue.

On a small screen, scroll the figure horizontally.

Condition on enrolment, \(L=\{V_4\}\). On the direct path \(V_1\to V_3\leftarrow V_2\), the admission vertex \(V_3\) is a collider. It is not itself in \(L\), so this path is blocked under our walk-based rule. It is not ancestrally blocked, however, because its descendant \(V_4\) is in \(L\). This matches the familiar selection effect: among enrolled pupils, academic and sport rankings can become associated.

The walk criterion reaches the same conclusion by allowing the repeated-vertex walk

\[V_1\to V_3\to V_4\leftarrow V_3\leftarrow V_2.\]

Its only collider is \(V_4\), which is conditioned on. The two occurrences of \(V_3\) are non-colliders and are not conditioned on, so this walk is not blocked by \(V_4\).

Most graphical-models texts start from paths rather than walks. Why keep both formulations?

• Paths are computationally efficient. No vertex repeats, so path-based criteria lead to fast algorithms for checking separation.
• Walks are convenient for proofs. Whether a walk is blocked depends only on that walk and \(L\), so the relation behaves cleanly under marginalization. By contrast, checking whether a path is ancestrally blocked requires searching the ambient graph for descendants of its colliders, which is awkward in structural arguments.

7.3 Conditional independence beyond Gaussian models

7.3.1 A distributional definition

We now strip away the linear algebra.

Definition 7.8.

Let \(X,Y,Z\) be random variables, vectors or, more generally, random elements. We write \(X\independent Y\mid Z\) if, for all measurable sets \(B_X,B_Y\),

\[ \mathsf P(X\in B_X,Y\in B_Y\mid Z) =\mathsf P(X\in B_X\mid Z)\, \mathsf P(Y\in B_Y\mid Z) \quad\text{almost surely}. \]

Equivalently, conditional on \(Z\), learning \(Y\) is irrelevant to the conditional law of \(X\):

\[ \mathsf P(X\in B_X\mid Y,Z) =\mathsf P(X\in B_X\mid Z) \quad\text{almost surely}. \]

When conditional densities exist, the definition becomes the familiar factorization

\[p(x,y\mid z)=p(x\mid z)p(y\mid z).\]

We can also view conditional independence as a relation between variables, with two basic properties.

Proposition 7.9.

For disjoint random subvectors \(J,K,L,M\),

\[ J\independent K\mid L \quad\Longleftrightarrow\quad K\independent J\mid L \qquad\text{(symmetry)}, \]

and

\[ J\independent (K,M)\mid L \quad\Longleftrightarrow\quad \bigl\{J\independent M\mid L \text{ and }J\independent K\mid(L,M)\bigr\} \qquad\text{(chain rule)}. \]

The equivalence contains three directions: each right-hand statement follows from the left, while the two right-hand statements together give the converse. Its proof, extensions and interpretation are left for Example Sheet 2.

7.3.2 Why move beyond linear models?

Recall the sequential, multiple assignment, randomized trial (SMART) from Lecture 4 (Section 4.2.2). Let \(A_1\) be the autumn message assignment, \(Y_1\) autumn absence, \(A_2\) the spring message assignment, and \(Y_2\) spring absence.

The SMART graph. Blue arrows follow the sequence from autumn assignment through autumn absence and spring assignment to spring absence, with additional arrows from the first assignment to the later assignment and outcome. A red bidirected edge joins the two absence outcomes.

On a small screen, scroll the figure horizontally.

The assignments \(A_1\) and \(A_2\) are categorical, and the second assignment depends on earlier information. A Gaussian linear equation is not a natural description. The example raises two questions:

1. How can a graph define a model without linearity or Gaussian noise?
2. How should sequential randomization be defined formally?

We address the first question now. The second motivates the next lecture.

7.4 Equation, Markov and factorization models

Let \(G\in\mathbb G_A^*(V)\) be a canonical acyclic directed mixed graph on the finite vertex set \(V=\{V_1,\ldots,V_d\}\). The star records that every vertex has its bidirected variance loop. Write

\[ \operatorname{pa}_G(j)=\{k:V_k\to V_j\text{ in }G\}, \qquad \operatorname{ch}_G(j)=\{k:V_j\to V_k\text{ in }G\} \]

for the parent and child indices of \(V_j\). We will compare three collections of probability distributions.

7.4.1 The nonparametric equation model

Definition 7.10.

The nonparametric equation model (E-model) replaces each linear equation by an unrestricted function:

\[ V_j=f_j\bigl(V_{\operatorname{pa}_G(j)},E_j\bigr), \qquad j=1,\ldots,d. \]

The graph also restricts the joint distribution of the noise variables: for all disjoint index sets \(J,K\subseteq\{1,\ldots,d\}\),

\[ \text{not }V_J\leftrightarrow V_K\text{ in }G \quad\Longrightarrow\quad E_J\independent E_K. \]

Thus \(E_J\) and \(E_K\) are independent whenever no bidirected edge joins a vertex in \(V_J\) to a vertex in \(V_K\). This is the nonparametric counterpart of requiring zeros in the noise covariance matrix of a Gaussian linear system. The functions \(f_j\) may be nonlinear, and the noises need not be Gaussian.

The model is a collection of distributions of \(V\): a distribution belongs to the E-model when some functions and noise variables satisfying these conditions generate it.

7.4.2 The global Markov model

Definition 7.11.

The global Markov model (M-model) is defined directly through the graphical separation rule. A distribution belongs to it when, for every three disjoint sets \(J,K,L\subseteq V\),

\[ \text{not }J\mconn K\mid L\text{ in }G \quad\Longrightarrow\quad J\independent K\mid L. \]

Thus the M-model collects all distributions for which every m-separation in the graph gives a valid conditional independence.

7.4.3 The factorization model

Definition 7.12.

For a canonical directed acyclic graph \(G\in\mathbb G_{DA}^*(V)\)—so no bidirected edge joins distinct vertices—order the variables topologically so that parents come before children. The factorization model (F-model) consists of the laws that can be written as an iterated product of conditional kernels:

\[ \mathsf P(dv_1,\ldots,dv_d) =\prod_{i=1}^{d} \mathsf P\bigl(dv_i\mid v_{\operatorname{pa}_G(i)}\bigr). \]

This definition does not require a density. When a joint density exists, it specializes to

\[p(v)=\prod_{i=1}^{d}p\bigl(v_i\mid v_{\operatorname{pa}_G(i)}\bigr).\]

For example, consider the DAG below; bidirected variance loops are suppressed.

A four-variable DAG used to illustrate factorization. All directed edges are blue.

On a small screen, scroll the figure horizontally.

Its law factorizes as

\[ \mathsf P(dv_1,dv_2,dv_3,dv_4) =\mathsf P(dv_1) \mathsf P(dv_2\mid v_1) \mathsf P(dv_3\mid v_1) \mathsf P(dv_4\mid v_2,v_3). \]

7.4.4 How the models are related

Theorem 7.13.

1. For a canonical acyclic directed mixed graph \(G\in\mathbb G_A^*(V)\),

   \[E\Longrightarrow M.\]

2. For a canonical DAG \(G\in\mathbb G_{DA}^*(V)\),

   \[M\Longrightarrow F\Longrightarrow E.\]

The first statement generalizes the Gaussian result proved earlier (Theorem 7.6): neither linearity nor Gaussian noise is needed. What matters is that each variable depends only on its parents and its own noise, while the noise distribution respects the bidirected part of the graph. Its proof is deferred.

For DAGs, \(M\Longrightarrow F\) follows by ordering the variables topologically, applying the Markov property to each variable given its parents—which m-separate it from the other preceding variables—and using the probability chain rule. For \(F\Longrightarrow E\), conditional CDFs and the probability integral transform provide independent noise variables and equations that generate the required conditional kernels. These two arguments are left as exercises.

Corollary 7.14.

For DAGs, the three models coincide:

\[E\Longleftrightarrow M\Longleftrightarrow F.\]

© 2026 Qingyuan Zhao. Except where otherwise noted, these lecture notes are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). Third-party material is excluded from this licence; see the individual credits and rights notices. Public-domain material remains in the public domain.