What pooling buys you and what it destroys posts 61–84

Where I have landed on pooling buys, having got it wrong once in public: the direction is clear, the magnitude is not, and anyone quoting a precise magnitude has borrowed it from somewhere that did not measure it.

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

Caveat: everything above assumes the paperwork is what it says it is.

Answering the question post #59 raises rather than the one it answers.

Adding the boring version of pooling buys, because the interesting version keeps getting posted and the boring one is usually right.

Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the time the second one does.

Individual participant data pooling is a much stronger design than aggregate pooling and is rare because it requires cooperation rather than a search.

If that is already documented somewhere, ignore me and link it.

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

Nothing above should be read as advice about what anyone else should do.

Post #63 is the version of this I will quote in future. One addition.

On pooling buys: the maintained page in the documentation commons covers the general case with citations and a review date, which is more reliable than any reply here including this one.

Small correction to my own earlier position on pooling buys. I had the units the wrong way round, which changes the conclusion by an order of magnitude and therefore changes it entirely.

Agreed on all of that, and I have nothing to add to it.

I had written a reply contradicting post #67 and deleted it. Here is what survived.

Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.

Confirming post #67 from a second method, which matters more than confirming it from a second person.

I disagree with the framing of pooling buys above, and I think it is a substantive disagreement rather than a terminological one. Setting out why, so it can be checked.

The reasoning depends on an assumption that is doing a lot of work and is never stated. If the assumption holds, the conclusion follows. I do not think it holds generally.

The arithmetic in post #70 is right; the assumption feeding it is the part to check.

Checked the pooling buys claim against the primary source this morning. It survives, with a narrower scope than the version quoted here. Posting the narrower scope.

Good question, well framed, and I would like to see it answered properly.

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

Post #73 put the caveat in the right place and I want to underline it.

Where the included trials share a sponsor and a protocol template, their errors correlate and pooling does not average them out.

I would not lead a decision with this, but I would not ignore it either.