Does causal mediation analysis need to have an interventionist interpretation?
At the end of Vanessa Didelez’s talk in the Foundations of causal inference workshop at the Isaac Newton Institute today, I asked her whether she would be okay with a mediation publication that only discusses potential interventionist interpretation in the Discussion section of the paper. The immediate reaction from Vanessa was that she does not like this but will not police it. In the discussion afterward, Thomas Richardson showed more disagreement with me. When I raised the potential that requiring scientists to specify separable components of the treatment will make us not popular, Thomas responded that “when I got my academic training we were not asked to be popular” (or something like this). Provoked by this comment, I decided to write a post to articulate my point.
I guess there are two kinds of popularity: winning people by exploiting their emotions or winning people using well-articulated reasoning. (If you are following the news this week, these are like the speeches given at Davos by Trump and Carney.) I was talking about statistics/causal inference gaining popularity and respect among applied scientists in the second way. I suppose that’s what we all want to do, right?
A related observation: after my history talk on randomization this Monday, I was asked by a member of the audience about whether causal inference only includes observational studies and not randomized experiment. It was very surprising to me it seemed obvious that causal inference as a field should consider any argument (even qualitative) that tries to infer a causal relationship; why would we on earth not include people doing randomized clinical trials in our field? I very much hope this was just an isolated instance of unfortunate misunderstanding, but it still makes me worry about how we as methodologists present recommendations to practitioners. On this matter, could the Society for Causal Inference organize an effort to publish guidelines on the use of certain observational designs (like a medical association would do for some diseases)?
Coming back to mediation. My point was that I think people often use mediation analysis to get an understanding of the relative importance “tagged” by different pathways, so they have an understanding about which mechanism is more “important” than other. I agree with the point in Vanessa’s talk that unfortunately people often just stop there with such an “understanding” and don’t do anything else. I’m of course frustrated with that, too, but I am not too offended because any such “understanding” is going inconsequential and almost harmless. Perhaps it will become a part of some qualitative argument for something else (and as a field we haven’t provided very much guidance on such qualitative reasoning), but I can’t see how the results of a mediation analysis about pure direct/indirect effects will become a strong piece of evidence if interpreted correctly (which is another thing we need to provide more guidance).
In our short discussion, Thomas suggested that if Jamie and he had taken my attitude above, they would not have developed the enterprise of interventionist/separable effects. Of course I would be in a worse position to infer that counterfactual compared to Thomas. Nonetheless here is slightly different account of why the interventionist approach arise and prevailed. I think what was really problematic in this scientific practice is how the direct and indirect effects are abused. And really the larger problem is that theoretical statisticians, at least in how many of them publish their papers, treat “causal identification” as done when we prove an identification formula under some assumptions. That’s entirely wrong—“causal identification” is only done when those assumptions can be reasonably justified in the practical problem. And I think this is why the interventionist/separable approach is useful: it gives a story that can be further validated.
Putting it differently, I believe the fundamental problem that the interventionist approach addresses is the attitude that we just prove causal identification in a theoretical paper and pretend that the assumptions behind are not a problem in any applications. Several people pointed out that this is deeply problematic: Paul Rosenbaum emphasized that “Observational studies of the effects caused by treatments are always subject to the concern that an ostensible treatment effect may reflect a bias in treatment assignment” in his talk, and both George Davey Smith and I quoted the following paragraph from Wright (1923)
It was of course realized that the “concrete, phenomenal actuality” of the results was not proved by the analysis by path coefficients. This rests on the validity of the premises, i.e., on the evidence for Mendelian heredity.
George further pointed out that Wright indeed included the variable “Chance” (due to Mendelian randomization) in the lovely Guinea pig diagram in his original paper.
Coming back to the mediation problem again, let me give a more concrete (but still hypothetical) example to demonstrate my point. Suppose we would like to cure a Mendelian disease, which we know is due to a single mutation \(A\). It is also known that this mutation \(A\) has a causal effect on protein \(M_1\). A scientist ran a mediation analysis to estimate the direct and indirect effects, and found that the indirect effect through \(M_1\) appears to be small. Having done this analysis, the scientist decided to do more experiments and found protein \(M_2\). In a second mediation analysis, the scientist found a significant indirect effect through \(M_2\). They then designed a drug targeting the biological pathway from \(A\) to \(M2\) and ran a randomized clinical trial to test the effectiveness of that drug.
Although I can’t give any drug that came up exactly like this, I am confident that the story above is not far-fetched and George can give an example. The steps above all look very reasonable, and the way mediation analysis is used is how I imagine most (good and honest) scientists would do. I don’t think they would be bothered in the mediation analyses about whether they can separate \(A\) into parts that only change \(M_1\), \(M_2\), or \(Y\). In fact, \(A\) is a single mutation so not physically separable so I don’t think there can be an interventionist explanation of this apparent success of mediation analysis!
I gave this example to Lin Liu afterward and he suggested that we can instead test the controlled (indirect?) effect \(Y(1, m_1) - Y(1, m_1’)\). In the usual simple mediation graph this is identified by \(E(Y \mid A = 1, M_1 = m_1) - E(Y \mid A = a, M_1 = m_1’)\). In contrast, the usual pure/natural indirect effect identification formula (with binary \(A\) and \(M\)) is \[E\{Y(1,M(1)) - Y(1, M(0))\} = \underbrace{(E[Y \mid A=1, M=1] - E[Y \mid A=1, M=0])}_{\text{Effect of M on Y}} \times \underbrace{(P(M=1 \mid A=1) - P(M=1 \mid A=0))}_{\text{Effect of A on M}}.\] So Lin’s suggestion appears to be doing basically the same test. Personally I would argue that the scientist will find the indirect effect estimand more interpretable because they are indeed looking for proteins that mediate the effect of the genetic mutation instead of the protein that has a causal effect on the disease. The former interpretation gives them confidence in trying to find drug target that disrupts the \(A \rightarrow M_2\) pathway in the example above, which is nicely summarized by George as the principle of gene-environment equivalence in Mendelian randomization in his talk.
My explanation of the apparent inability of the interventionist account for this problem is that the scientists are thinking about intervening the mechanism as represented by the edge \(A \rightarrow M_2\) instead of the variable \(A\) or \(M\). I had this thought for a while and it is reinforced by the so-called “edge intervention” in the paper by Ilya Shpitser and Eric Tchetgen Tchetgen. That paper defines “edge intervention” using intervention on the mediator: “We call a forced assignment of variables corresponding to source vertices of edges from \(\boldsymbol{\alpha}\) to an element of \(\mathfrak{X}_{\boldsymbol{\alpha}}\) an edge intervention.” In my opinion this more accurately describes how the scientist thinks in the above example and is still not quite able to describe the concept of gene-environment equivalence.
More broadly and beyond mediation analysis, the point I wanted to make is that strict adherence to the “causal inference = causal identification and then statistical inference” doctrine can turn our allies (people interested in using causal inference methods) to enemies (people turned away by strict requirements of the interventionist account). I will save this topic for another post, but for now, my position is that I am happy for practitioners to publish their results about pure/natural direct and indirect effects (at least in the simple mediation graph) as long as they discuss practical implications of the results in the context of their problem (such as how a strong direct effect may be mediated by a different biomarker).