This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.
Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.
I read the earlier replies on pooling trials with different estimands twice before writing this, because I had assumed the opposite and wanted to be sure I was disagreeing with what was said rather than what I expected.
Coming back to post #29, because the follow-up matters more than the original answer.
Worth stating the null on pooling trials with different estimands before we explain it: the observation may be nothing. That possibility deserves a sentence and usually does not get one.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
Take the reasoning and check the arithmetic; I do not always get it right.
Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis.
That holds under the stated conditions and I have stated them.
Confirming post #36 from a second method, which matters more than confirming it from a second person.
Checked the pooling trials with different estimands claim against the primary source this morning. It survives, with a narrower scope than the version quoted here. Posting the narrower scope.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.
That is one dataset and I would not build a rule on it.
This follows post #38 rather than contradicting it.
Before the thread moves on from pooling trials with different estimands — what is the sample size behind the claim? I am not being difficult; I have seen the same figure quoted from an n of four and from an n of four hundred.
Worth separating two things that post #40 runs together.
Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.
For what it is worth, the same held on the two occasions I checked.
The confident answers on pooling trials with different estimands and the well-sourced answers are not the same answers, which is the most useful thing I have learned reading this category.
Taking post #42 at face value and following it one step further.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
A guess, clearly labelled as one.
Small correction to my own earlier position on pooling trials with different estimands. I had the units the wrong way round, which changes the conclusion by an order of magnitude and therefore changes it entirely.
Thank you for the correction. I would rather find out here than later.
Overlapping populations across included trials inflate the apparent sample size. It happens more than people expect where the same programme reports multiple papers.
Quality assessment of included trials should change the analysis rather than sit beside it. A sensitivity analysis excluding the weakest studies is the minimum.
Two sources, same conclusion, and I could not rule out that one copied the other.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
I would keep pooling trials with different estimands and the decision it usually gets used for separate in this thread. They are related and they are not the same question, and merging them is why the last one went badly.
Fixed-effect and random-effects models answer different questions. The first assumes one true effect; the second assumes a distribution of them. Choosing between them is an assumption, not a technicality.
Narrowing post #53, because the general version has more than one answer.
Individual participant data pooling is a much stronger design than aggregate pooling and is rare because it requires cooperation rather than a search.
I have left out the parts I could not verify.
Where I part company with post #53, and it is a narrow parting.
When trials differ in population, duration and comparator, the pooled estimate answers a question no individual trial asked. That is worth saying before the number is quoted.
Happy to expand any of that if it is the useful part.
Where I would push back on the pooling trials with different estimands consensus is the confidence, not the direction. The direction looks right. The confidence is borrowed.
Small methodological point on pooling trials with different estimands: repeating a measurement is cheap and resolves most of what is being argued about here at no cost to anyone.
I had written a reply contradicting post #55 and deleted it. Here is what survived.
Pooling trials with different estimands sits at the boundary between what this community can usefully discuss and what it cannot, and I think it falls on the discussable side, narrowly.
Confirming post #59 from a second method, which matters more than confirming it from a second person.
Where the included trials share a sponsor and a protocol template, their errors correlate and pooling does not average them out.
It is worth stating the boring hypothesis before the interesting one.