5  Path Analysis

These notes were automatically generated by AI and lightly reviewed by the lecturer. They may still contain mistakes and factual errors.

Let’s use the walk algebra from Lecture 4 (Section 4.4) to understand how a graph determines covariances. Linear models make the calculations manageable. They also help us discover graphical relations that we can later study without assuming linearity. We will move from matrix algebra to the trek rule, simplify it using Wright’s path analysis, and then ask what observed correlations can tell us about an unobserved common factor.

5.1 Linear systems and directed walks

5.1.1 The covariance we want to understand

Recall the linear structural equation model (Definition 4.4) on a directed mixed graph \(G\in\mathbb{G}(V)\):

\[V=\beta^T V+E, \qquad V_j=\sum_k\beta_{kj}V_k+E_j, \qquad E\sim\mathcal{N}(0,\Lambda).\]

Here \(V=(V_1,\ldots,V_{|V|})^T\) is the vector of variables. The matrix \(\beta\) respects the directed edges: \(\beta_{kj}=0\) if \(V_k\to V_j\) is absent. Similarly, \(\Lambda\) respects the bidirected edges: \(\Lambda_{jk}=0\) if \(V_j\leftrightarrow V_k\) is absent. A bidirected self-loop represents the noise variance \(\Lambda_{jj}\).

Assuming \(I-\beta\) is invertible, we can solve the system:

\[V=(I-\beta)^{-T}E\sim\mathcal{N}(0,\Sigma), \qquad \Sigma=(I-\beta)^{-T}\Lambda(I-\beta)^{-1}.\tag{5.1}\]

Our aim is to read this expression from the graph. Consider the running example below. We will first trace directed walks from \(V_2\) to \(V_5\), then calculate \(\operatorname{Cov}(V_3,V_4)\).

Teaching redraw of the running directed mixed graph. Directed edges are blue; bidirected edges, including the variance loops, are red.

On a small screen, scroll the figure horizontally to read the labels.

5.1.2 Acyclicity and the Neumann series

Definition 5.1.

A graph is acyclic if we cannot follow directed edges in their forward direction and return to the vertex where we started. Such a graph has a topological ordering: choose a vertex with no incoming directed edges, then a vertex whose parents have already been chosen, and continue. In that ordering, \(\beta\) is strictly triangular.

Write \(\mathbb{G}_A(V)\) for the class of acyclic directed mixed graphs on \(V\).

Lemma 5.2 (finite Neumann series).

If \(G\in\mathbb{G}_A(V)\), then

\[(I-\beta)^{-1}=I+\beta+\beta^2+\cdots+\beta^{|V|-1}.\]

Proof.

To see why, think of \(\beta\) as a weighted adjacency matrix. Its \(r\)th power collects products along directed walks of length \(r\) (the weighted version of Proposition 4.6). In an acyclic graph a directed walk cannot repeat a vertex, so its length is at most \(|V|-1\). Consequently \(\beta^{|V|}=0\), and

\[(I-\beta)(I+\beta+\cdots+\beta^{|V|-1})=I-\beta^{|V|}=I.\]

More generally, the infinite Neumann series converges when the spectral radius of \(\beta\)—the largest absolute value of its eigenvalues—is less than 1. For the graphical calculations here, we will assume acyclicity, so the series is finite.

5.1.3 Directed walks and their weights

Recall that \(W[V\to V]\) is a matrix whose entries are sets of directed edges. In the walk algebra, addition is set union, multiplication concatenates walks at a common vertex, and transposition reverses their traversal. Matrix multiplication uses these operations in place of ordinary addition and multiplication of numbers.

Definition 5.3.

Define the matrix of nontrivial directed walks by

\[W[V\rdpath V]=\sum_{r\geq1}W[V\to V]^r.\]

The sum is an entry-wise union: the \((j,k)\) entry contains all positive-length directed walks from \(V_j\) to \(V_k\). The wavy arrow describes a sequence of edges, rather than a single edge. This gives a precise statement of acyclicity:

\[G\in\mathbb{G}_A(V) \quad\Longleftrightarrow\quad W[V_j\rdpath V_j]=\emptyset\quad\text{for every }j.\]

Equivalently, \(G\) is acyclic if and only if every entry of \(W[V\to V]^{|V|}\) is empty. The definition of a directed walk and the restriction that the graph be acyclic are separate: acyclicity ensures that only finitely many powers contribute.

Definition 5.4.

To connect walks to the linear model, define a weight function \(\sigma\):

\[\sigma(V_j\to V_k)=\beta_{jk}, \qquad \sigma(V_j\leftrightarrow V_k)=\Lambda_{jk}.\]

Multiply edge weights along a walk; sum over distinct walks in a set. Reversing the traversal of a directed edge does not change its weight: the edge \(V_j\leftarrow V_k\) still has weight \(\beta_{kj}\). The empty set has weight zero.

For example, the graph has two directed walks from \(V_2\) to \(V_5\): \(V_2\to V_5\) and \(V_2\to V_3\to V_5\). Hence

\[\sigma\bigl(W[V_2\rdpath V_5]\bigr) =\beta_{25}+\beta_{23}\beta_{35}.\]

When \(\sigma\) acts on a matrix of sets of walks, apply it entry by entry. It gives a number for each set, and therefore a numerical matrix for the whole walk matrix. The Neumann-series lemma (Lemma 5.2) can now be written as

\[(I-\beta)^{-1}=I+\sigma\bigl(W[V\rdpath V]\bigr).\]

This correspondence is the reason for introducing the walk algebra: the numerical matrices lead us to relations between vertices in the graph.

5.2 The trek rule

5.2.1 From the covariance product to treks

Definition 5.5.

Write \(W[V\ldpath V]=W[V\rdpath V]^T\) for directed walks traversed backwards. We also need the length-zero walk \(\mathrm{id}_j\) at each vertex. It leaves any walk unchanged when concatenated with it, and has weight 1. Let \(\mathrm{Id}\) be the walk matrix with \(\{\mathrm{id}_j\}\) on its diagonal and empty sets elsewhere. Then \(\sigma(\mathrm{Id})=I\).

Substitute the walk expression for each inverse in the covariance formula (Equation (5.1)):

\[ \begin{aligned} \Sigma &=(I-\beta)^{-T}\Lambda(I-\beta)^{-1}\\ &=\sigma\bigl(\mathrm{Id}+W[V\ldpath V]\bigr) \sigma\bigl(W[V\leftrightarrow V]\bigr) \sigma\bigl(\mathrm{Id}+W[V\rdpath V]\bigr). \end{aligned} \]

Definition 5.6.

We can concatenate the walks first and then take their weights. A resulting walk consists of a directed part traversed backwards, one bidirected edge, and a directed part traversed forwards. Either directed part may have length zero. Expanding the product gives four possibilities:

\[ \begin{aligned} W[V\trek V] ={}&W[V\leftrightarrow V]\\ &+W[V\ldpath V\leftrightarrow V]\\ &+W[V\leftrightarrow V\rdpath V]\\ &+W[V\ldpath V\leftrightarrow V\rdpath V]. \end{aligned} \]

These walks are called treks. The identity terms matter: without them we would omit treks having no directed leg on one side or on either side. The bidirected edge can be a self-loop, so a trek can revisit vertices.

Theorem 5.7 (trek rule).

For a linear system on \(G\in\mathbb{G}_A(V)\),

\[\operatorname{Cov}(V)=\sigma\bigl(W[V\trek V]\bigr).\]

Thus to calculate \(\operatorname{Cov}(V_j,V_k)\), list the treks from \(V_j\) to \(V_k\), multiply edge weights along each trek, and add the resulting monomials. The same rule with \(V_j=V_k\) gives the variance of \(V_j\).

5.2.2 Example

Return to the running graph. There is no direct bidirected edge between \(V_3\) and \(V_4\), but there are four treks connecting them:

Trek	Weight
\(V_3\leftarrow V_1\leftrightarrow V_4\)	\(\Lambda_{14}\beta_{13}\)
\(V_3\leftrightarrow V_3\to V_4\)	\(\Lambda_{33}\beta_{34}\)
\(V_3\leftarrow V_1\leftrightarrow V_1\to V_3\to V_4\)	\(\Lambda_{11}\beta_{13}^2\beta_{34}\)
\(V_3\leftarrow V_2\leftrightarrow V_2\to V_3\to V_4\)	\(\Lambda_{22}\beta_{23}^2\beta_{34}\)

Every term contains exactly one entry of \(\Lambda\), corresponding to the bidirected edge. Summing gives

\[ \begin{aligned} \operatorname{Cov}(V_3,V_4) ={}&\Lambda_{14}\beta_{13}+\Lambda_{33}\beta_{34}\\ &+\Lambda_{11}\beta_{13}^2\beta_{34} +\Lambda_{22}\beta_{23}^2\beta_{34}. \end{aligned} \]

The last three terms all end with \(V_3\to V_4\). Before that final edge, they trace the three treks from \(V_3\) back to itself. Their combined weight is therefore \(\operatorname{Var}(V_3)\):

\[\operatorname{Var}(V_3) =\Lambda_{33}+\Lambda_{11}\beta_{13}^2+\Lambda_{22}\beta_{23}^2.\]

Consequently,

\[\operatorname{Cov}(V_3,V_4) =\Lambda_{14}\beta_{13}+\operatorname{Var}(V_3)\beta_{34}.\tag{5.2}\]

This regrouping suggests a simpler calculation. Rather than tracking all the loops that contribute to a variance, can we work with paths that do not repeat vertices?

5.3 Arcs and Wright’s path analysis

Sewall Wright developed path analysis more than a century ago, motivated by the problem of genetic heritability illustrated by the guinea-pig example in Lecture 4 (Section 4.2.3). To explain his method, let’s first define a couple of things.

5.3.1 Walks without colliders

Definition 5.8.

A collider is an interior occurrence of a vertex where the two adjoining edges both have arrowheads pointing into it, as in \(\to V_j\leftarrow\). The far ends of those edges may be heads or tails. A walk with no collider is called an arc, or an m-connected walk; the “m” stands for mixed.

We denote an arc by \(V_j\mconnarc V_k\). The half-arrowheads leave the endpoint marks unrestricted: either a head or a tail is allowed. There are two kinds of nontrivial arcs:

• Treks have exactly one bidirected edge. Their possible forms are the four terms in the trek decomposition (Definition 5.6).
• d-connected walks have no bidirected edges. The “d” stands for directed, meaning that only directed edges occur; their traversal need not follow the arrows throughout.

A d-connected walk can go forwards, go backwards, or first go backwards and then forwards. In walk-matrix notation,

\[ \begin{aligned} W[V\dconnarc V] ={}&W[V\rdpath V]+W[V\ldpath V]\\ &+W[V\ldpath V\rdpath V]. \end{aligned} \]

Going forwards and then backwards would instead produce a collider at the turning point. This explains the forms of walk that appear in the decomposition.

5.3.2 Replacing walks by paths

Definition 5.9.

A path is a walk with no repeated vertex. Use \(P[\cdots]\subseteq W[\cdots]\) to select the paths from a specified collection of walks. For example, \(V_3\leftrightarrow V_3\to V_4\) is a trek but not a path: it visits \(V_3\) twice.

Wright’s path analysis lets us calculate covariances without enumerating all such repeated-vertex walks.

Theorem 5.10 (Wright's path analysis).

Suppose \(G\in\mathbb{G}_A(V)\) and every variable is normalized so that \(\operatorname{Var}(V_j)=1\). For distinct vertices \(V_j\) and \(V_k\),

\[\operatorname{Cov}(V_j,V_k) =\sigma\bigl(P[V_j\mconnarc V_k]\bigr).\]

In words: sum the weights of all collider-free paths between the two vertices. These include both trek paths and d-connected paths. The unit-variance assumption concerns the variables \(V_j\), not their individual noise terms \(E_j\).

In our example, the collider-free paths between \(V_3\) and \(V_4\) are

\[V_3\leftarrow V_1\leftrightarrow V_4, \qquad V_3\to V_4.\]

Their weights immediately give

\[\operatorname{Cov}(V_3,V_4)=\Lambda_{14}\beta_{13}+\beta_{34}.\]

Compare this with our earlier calculation (Equation (5.2)). The first trek was already a path and remains unchanged. The other three, all involving repeated vertices, combine into \(\operatorname{Var}(V_3)\beta_{34}\). Normalizing \(\operatorname{Var}(V_3)\) to 1 leaves the weight of the single edge \(V_3\to V_4\).

Paths are often easier to use for a hand calculation; drawings for path analysis commonly leave the variance loops implicit. Walks with explicit loops remain useful mathematically because they give the direct correspondence with matrix products and the covariance formula.

5.4 Factor analysis

Next, let’s do something more fun and see how path analysis can be useful.

5.4.1 Spearman’s correlations

The idea of explaining observed covariances using a common factor predates Wright’s path analysis. Charles Spearman’s 1904 paper, “General Intelligence,” Objectively Determined and Measured, helped establish factor analysis in psychology.¹ The question is whether a small number of latent factors can explain variation in a larger number of observed variables. Factor analysis is related to principal component analysis, but here we will study an explicit measurement model.

One part of Spearman’s study examined school-study rankings for 22 music pupils at a preparatory school. The table shows six of the reported correlations, for four school subjects:

	Classics	French	English	Mathematics
Classics	—	0.83	0.78	0.70
French		—	0.67	0.67
English			—	0.64
Mathematics				—

Only correlations between different subjects are shown. Spearman proposed that one “general intelligence” factor could explain their common variation. Let’s take that interpretation as a working hypothesis and ask what mathematical restrictions it imposes.

5.4.2 Three measurements of one latent variable

Begin with an unobserved variable \(U\) and three measurements \(X_1,X_2,X_3\).

Teaching diagram of a one-factor measurement model. The dashed node is the latent factor U. Blue arrows carry the loadings; red bidirected loops represent variances.

The corresponding linear system is

\[X_j=\beta_jU+E_j,\qquad j=1,2,3.\]

Assume centred Gaussian variables, with the noises independent of \(U\) and \(E=(E_1,E_2,E_3)^T\sim\mathcal{N}(0,\Lambda)\), where \(\Lambda\) is diagonal. The missing bidirected edges express these independence assumptions. Thus the measurements are independent conditional on \(U\).

We can set \(\operatorname{Var}(U)=1\) to fix the scale of the latent factor: rescaling \(U\) can be offset by rescaling the loadings \(\beta_j\). Here we normalize \(U\); we do not require each observed \(X_j\) to have unit variance.

The observed covariance matrix is

\[\Sigma=\operatorname{Cov}(X)=\beta\beta^T+\Lambda, \qquad \beta=(\beta_1,\beta_2,\beta_3)^T,\]

or, explicitly,

\[\Sigma= \begin{pmatrix} \beta_1^2+\Lambda_{11}&\beta_1\beta_2&\beta_1\beta_3\\ \beta_1\beta_2&\beta_2^2+\Lambda_{22}&\beta_2\beta_3\\ \beta_1\beta_3&\beta_2\beta_3&\beta_3^2+\Lambda_{33} \end{pmatrix}.\]

Identifiability asks whether we can invert the map \((\beta,\Lambda)\mapsto\Sigma\): can we recover the model parameters from the distribution of what we observe? With a latent variable this is not automatic. There are six unknown parameters here—three loadings and three noise variances—and six distinct entries of the observed covariance matrix. The count suggests identifiability might be possible, but we still need an argument.

Proposition 5.11.

If \(\beta_1,\beta_2,\beta_3\ne0\), the parameters are identifiable up to a simultaneous sign change \(\beta\mapsto-\beta\).

Proof.

Multiply the three off-diagonal entries:

\[\Sigma_{12}\Sigma_{13}\Sigma_{23} =(\beta_1\beta_2\beta_3)^2.\]

Taking a square root identifies \(|\beta_1\beta_2\beta_3|\). Choose either sign for the product \(\beta_1\beta_2\beta_3\); dividing it by \(\Sigma_{23}\), \(\Sigma_{13}\) and \(\Sigma_{12}\) recovers \(\beta_1\), \(\beta_2\) and \(\beta_3\), respectively. The other sign choice changes all three signs together. Finally, recover each noise variance from

\[\Lambda_{jj}=\Sigma_{jj}-\beta_j^2.\]

The sign ambiguity reflects the fact that \(U\) itself is unobserved. In practice we usually choose its orientation by interpretation: a factor called “intelligence” should have the intended positive association with performance; with the opposite orientation we might call it “non-intelligence”. The overall sign does not change the observed covariance matrix.

5.4.3 Four measurements and tetrads

Now allow \(p\) measurements of the same latent factor. With its variance fixed at 1, there are \(2p\) parameters: \(p\) loadings and \(p\) noise variances. The observed covariance matrix supplies \(p(p+1)/2\) distinct entries.

Measurements	Parameters	Variances and covariances	Identification
2	4	3	Under-identified in general
3	6	6	Just-identified, up to sign, with nonzero loadings
4	8	10	Over-identified: additional covariance restrictions

Two measurements do not generally identify the model. Three suffice under the condition in Proposition 5.11. With four measurements, we have more observable covariance entries than free model parameters. What are the extra restrictions?

Teaching diagram with four indicators of one latent factor. The additional measurement makes the model impose restrictions on the observable covariances. Blue edges are directed; red bidirected loops represent variances.

The off-diagonal entries still satisfy \(\Sigma_{jk}=\beta_j\beta_k\). Suppressing the diagonal entries, which also contain noise variances, we have

\[\Sigma= \begin{pmatrix} *&\beta_1\beta_2&\beta_1\beta_3&\beta_1\beta_4\\ \beta_1\beta_2&*&\beta_2\beta_3&\beta_2\beta_4\\ \beta_1\beta_3&\beta_2\beta_3&*&\beta_3\beta_4\\ \beta_1\beta_4&\beta_2\beta_4&\beta_3\beta_4&* \end{pmatrix}.\]

Pair the four distinct indices in three different ways. Each product gives the same monomial:

\[\Sigma_{12}\Sigma_{34} =\Sigma_{13}\Sigma_{24} =\Sigma_{14}\Sigma_{23} =\beta_1\beta_2\beta_3\beta_4.\]

Differences of these products are called tetrads, and under the one-factor model they vanish. The three pairwise differences yield two independent equalities. Another way to see the restriction is that the off-diagonal \(2\times2\) blocks lose rank. For example, take rows \(\{1,2\}\) and columns \(\{3,4\}\):

\[\det\begin{pmatrix} \Sigma_{13}&\Sigma_{14}\\ \Sigma_{23}&\Sigma_{24} \end{pmatrix} =\Sigma_{13}\Sigma_{24}-\Sigma_{14}\Sigma_{23}=0.\]

These are constraints on observed covariances even though the common cause \(U\) is hidden. This was the attraction of Spearman’s argument: a hypothesis about one unobserved factor has implications that can be examined in the data.

For the four subjects in the table, labelled in their displayed order, one sample tetrad is

\[0.78\times0.67-0.70\times0.67=0.0536\approx0.054.\]

It is not exactly zero. The equality is a population restriction, whereas the reported correlations are estimated from a sample. Spearman used the approximate agreement of such relations across subjects as evidence for his general-factor hypothesis—the sense in which he described the factor as “objectively determined and measured”.

I hope that was fun. Next we will return to more general graphical relations.

© 2026 Qingyuan Zhao. Except where otherwise noted, these lecture notes are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). Third-party material is excluded from this licence; see the individual credits and rights notices. Public-domain material remains in the public domain.

---

1. C. Spearman (1904), “General Intelligence,” Objectively Determined and Measured, especially p. 275; accessible transcription, Chapter V, §§4–5.