[2026 update] Heterogeneity as information rather than as a nuisance

Heterogeneity as information rather than as a nuisance Writing it up because I had to work it out twice and would rather nobody else did.

I have seen SURPASS-4 (Lancet, 2021) cited in support of a claim I do not think it supports, twice this month, so I would like to work through what it actually shows.

My reading is that the trial is sound for its own question and is being stretched to answer a different one. I might be wrong about that, which is why this is a topic rather than a correction.

What I would like from this discussion: someone who disagrees with me to say why, with the section of the paper they are relying on.

Quality assessment of included trials should change the analysis rather than sit beside it. A sensitivity analysis excluding the weakest studies is the minimum.

I have said this before in a thread nobody could find, so it is worth repeating.

The opening post describes the usual case. This is about the unusual one.

Individual participant data pooling is a much stronger design than aggregate pooling and is rare because it requires cooperation rather than a search.

Adding the measurement that the opening post says would settle it.

Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.

I would put this at better than even and not much better.

The I-squared statistic describes the proportion of variability not attributable to chance and is frequently read as a threshold. It is a description rather than a test.

I have no interest in any supplier named above.