Evidence · Meta-analyses · continued

Pooling trials with different estimands posts 121–133

Evidence › Meta-analyses publication bias risk of bias effect size meta-analysis Solved

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.

v.kirchnerTL223 Jul 2026#121

n.broberg, post #112: When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate. Go to post

Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.

I would be glad to be shown a cleaner way of putting this.

0 likes in reply to #112 5d

v.szaboTL3Analytical chemist23 Jul 2026#122

Where the included trials share a sponsor and a protocol template, their errors correlate and pooling does not average them out.

The confident version of this sentence would be wrong, so here is the hedged one.

24 likes 5d

c.vasquezTL223 Jul 2026#123

Post #121 and I disagree about the size of the effect, not about the direction.

Fixed-effect and random-effects models answer different questions. The first assumes one true effect; the second assumes a distribution of them. Choosing between them is an assumption, not a technicality.

11 likes 4d

a.finnegan_rdTL2Dietitian24 Jul 2026 · edited#124

Taking post #121 at face value and following it one step further.

A request rather than an answer: could whoever has the primary source for pooling trials with different estimands post it? I have seen the claim three times this month and each version had lost a qualifier.

3 likes 4d

p.ostergaardTL224 Jul 2026#125

m.eriksen, post #5: This follows post #2 rather than contradicting it. Nobody has said the unglamorous part of pooling trials with different estimands yet, so: most of the variation is explained by things that are boring to write about and easy to check. Go to post

I read post #121 twice before replying, because I had assumed the opposite.

Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.

0 likes in reply to #5 4d

customs_ledgerTL3Regular24 Jul 2026#126

Understood, and I withdraw the assumption I opened with.

17 likes 3d

s.girardTL225 Jul 2026#127

What would change my mind on pooling trials with different estimands is a second dataset collected by someone with no stake in the first. Until then I hold it loosely and I would rather say so than pretend to more.

7 likes 3d

logbook_erinTL3Regular25 Jul 2026#128

This follows post #125 rather than contradicting it.

I changed my mind about pooling trials with different estimands after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.

1 like 3d

k.asanteTL226 Jul 2026#129

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

That holds for the case as described. Change the assumptions and it may not.

1 like 2d

c.okaforTL3Regular26 Jul 2026#130

Adding the measurement that post #129 says would settle it.

Worth separating pooling trials with different estimands as a question about the compound from pooling trials with different estimands as a question about the documentation. They get answered by different people and only one of them is answerable here.

0 likes 2d

LJankowiakTL3Regular26 Jul 2026#131

a.lindqvist, post #96: The arithmetic in post #95 is right; the assumption feeding it is the part to check. The arithmetic on pooling trials with different estimands is the easy part and it is where the errors are, which is an uncomfortable combination. Show your working and someone will catch it. Go to post

Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis.

I am reporting what happened, not recommending it.

0 likes in reply to #96 2d

ne.laurentTL227 Jul 2026#132

Collapsed as off-topic by two members at trust level 3 or above

Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.

ambient_draftTL3Regular27 Jul 2026#133

Appreciated. The plain phrasing does more work here than a longer post would.

12 likes 1d

Suggested topics

Topic	Participants	Replies	Views	Activity
Individual participant data versus aggregate data Evidence › Meta-analyses heterogeneity risk of bias meta-analysis publication bias Posting this under the heading it deserves: Individual participant data versus aggregate data Everything below is what sits behind that. Something about individual participant data versus aggregate does not…	+30	34	64k	14mo
Pooling trials with different estimands — what changed since Evidence › Meta-analyses publication bias risk of bias meta-analysis effect size Pooling trials with different estimands — what changed since — setting out what I have, and where I think it stops being reliable. What changes if the standard account of Pooling trials with different…	+57	64	12k	7mo
Follow-up: Individual participant data versus aggregate data Evidence › Meta-analyses meta-analysis effect size publication bias heterogeneity Individual participant data versus aggregate data — setting out what I have, and where I think it stops being reliable. A follow-up question about Individual participant data versus aggregate that I did not…	+101	111	23k	18mo
Random versus fixed effects: choosing rather than defaulting Evidence › Meta-analyses effect size publication bias risk of bias heterogeneity Closed Random versus fixed effects: choosing rather than defaulting — setting out what I have, and where I think it stops being reliable. Random versus fixed effects, and specifically the version of it that the…	+16	20	22k	13mo
When a network meta-analysis is defensible and when it is not Evidence › Meta-analyses effect size heterogeneity publication bias Solved When a network meta-analysis is defensible and when it is not I have a specific reason for asking rather than idle curiosity, and the context is below. Network meta-analysis, from the point of view of someone…		2	27k	4h

Related topics — sharing the tags publication bias, risk of bias, effect size

Topic	Participants	Replies	Views	Activity
Is there a ceiling dose beyond which semaglutide stops adding benefit? Compounds › Semaglutide effect size type 2 diabetes SELECT trial Is there a ceiling dose beyond which semaglutide stops adding benefit? I have a specific reason for asking rather than idle curiosity, and the context is below. Putting the numbers in the first post, because…		2	14k	2mo
Pooling trials with different estimands — what changed since Evidence › Meta-analyses publication bias risk of bias meta-analysis effect size Pooling trials with different estimands — what changed since — setting out what I have, and where I think it stops being reliable. What changes if the standard account of Pooling trials with different…	+57	64	12k	7mo
Follow-up: What is genuinely unknown about long-term amylin agonism Compounds › Cagrilintide & amylin analogues effect size amylin analogues cagrisema What is genuinely unknown about long-term amylin agonism I have a specific reason for asking rather than idle curiosity, and the context is below. Putting the numbers in the first post, because a question…		2	25k	11mo
Citing a preprint in a maintained document: our rule and its reasoning Evidence › Preprints risk of bias effect size needs a source preprint Pinned On the subject in the title: Citing a preprint in a maintained document: our rule and its reasoning Working notes rather than a conclusion. A preprint question rather than a question about the finding in it.…		2	467	4mo
What TRIUMPH is designed to answer, and why we should not pre-empt it — does this still hold? Compounds › Retatrutide randomised trial effect size TRIUMPH programme retatrutide Solved Asking directly, because I could not find a straight answer: What TRIUMPH is designed to answer, and why we should not pre-empt it — does this still hold? An honest uncertainty about TRIUMPH rather than a…	+43	47	5.5k	just now