Revisiting: What pooling buys you and what it destroys posts 91–108

This is the answer, and the reason it is the answer is the more useful part.

Worth separating pooling buys as a question about the compound from pooling buys as a question about the documentation. They get answered by different people and only one of them is answerable here.

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

This follows post #92 rather than contradicting it.

Where I have landed on pooling buys, having got it wrong once in public: the direction is clear, the magnitude is not, and anyone quoting a precise magnitude has borrowed it from somewhere that did not measure it.

Adding the boring version of pooling buys, because the interesting version keeps getting posted and the boring one is usually right.

Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the time the second one does.

Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.

On post #95 — agreed on the reasoning, with one qualification.

Small correction to my own earlier position on pooling buys. I had the units the wrong way round, which changes the conclusion by an order of magnitude and therefore changes it entirely.

This follows post #95 rather than contradicting it.

I keep a log for pooling buys specifically because my memory of it turned out to be systematically wrong in one direction. Six weeks of notes cost nothing and settled it.

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

Post #99 is right about the mechanism and I think understates the practical bit.

Pooling buys is one of those subjects where the general answer and the answer for a specific case diverge, and the thread will go in circles until someone says which one is being asked for.

Distinguishing three things in the pooling buys discussion that keep getting used interchangeably: the observation, the proposed mechanism, and the recommendation that gets attached to both.

Building on post #99 rather than restating it.

Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.

Answering the question post #99 raises rather than the one it answers.

Individual participant data pooling is a much stronger design than aggregate pooling and is rare because it requires cooperation rather than a search.

Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.