Taking pooling trials with different estimands seriously for a moment rather than deflecting: the honest position is that the community has observations and no controlled comparison, and those two things support very different sentences.
On the subject in the title: Pooling trials with different estimands Working notes rather than a conclusion.
Asking about pooling trials with different estimands on behalf of the question I keep seeing asked badly, including by me.
Framed properly it is answerable. Framed the usual way it is not, and that is most of why the previous threads went nowhere.
The reason pooling trials with different estimands keeps being re-asked is that the answer is conditional and people quote it without the condition. It is not that the answer is unknown.
Taking pooling trials with different estimands seriously for a moment rather than deflecting: the honest position is that the community has observations and no controlled comparison, and those two things support very different sentences.
Everything in the opening post holds. The case it does not cover is the one I have.
When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.
Not the whole picture, but the part of it I can speak to.
Overlapping populations across included trials inflate the apparent sample size. It happens more than people expect where the same programme reports multiple papers.
Adding the caveat now so it does not have to be extracted later.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
Someone should write this up properly, and it should probably not be me.
Following, with nothing to contribute beyond having asked the same thing elsewhere.
One caution on pooling trials with different estimands: everything above assumes the underlying documentation is what it claims to be. That assumption is doing real work and is rarely stated.
Picking up post #11: that is the part I would want checked first.
When trials differ in population, duration and comparator, the pooled estimate answers a question no individual trial asked. That is worth saying before the number is quoted.
Worth reading the earlier posts in this thread before acting on mine.
I had written a reply contradicting post #11 and deleted it. Here is what survived.
Whatever the answer on pooling trials with different estimands turns out to be, the method for getting there is the same: state the assumption, do the arithmetic in public, invite the correction.
Post #11 describes the usual case. This is about the unusual one.
A meta-analysis of four small trials is not stronger evidence than one adequately powered trial, whatever the summary statistic looks like.
Marking that as an opinion rather than a finding.
Reading rather than answering, but this is the post I would point somebody at.
Pooling trials with different estimands was covered in the wiki last year and the page has a review date on it, which is a better starting point than my memory of a thread.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
I would put moderate confidence on the mainstream reading of pooling trials with different estimands and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.
Where the included trials share a sponsor and a protocol template, their errors correlate and pooling does not average them out.
The short version is the first sentence; the rest is why.
Confirming post #20 from a second method, which matters more than confirming it from a second person.
Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.
The uncertainty is in the assumption, not in the calculation.
That reframing is the whole thing. The facts I already had.
This follows post #23 rather than contradicting it.
Fixed-effect and random-effects models answer different questions. The first assumes one true effect; the second assumes a distribution of them. Choosing between them is an assumption, not a technicality.
Quality assessment of included trials should change the analysis rather than sit beside it. A sensitivity analysis excluding the weakest studies is the minimum.
Answering the pooling trials with different estimands question as asked, then the question I think is meant. As asked: yes, with the qualification below. As meant: it depends on how the first measurement was taken.
Coming back to post #26, because the follow-up matters more than the original answer.
What I would want before treating pooling trials with different estimands as settled: the method, the sample, and whether anyone tried to find the opposite result. Two of the three are usually missing.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
I have no interest in any supplier named above.
Post #26 and I disagree about the size of the effect, not about the direction.
Marking my uncertainty on pooling trials with different estimands explicitly. I am confident about the direction, much less confident about the size, and not confident at all that it generalises past the case in the first post.