Random versus fixed effects: choosing rather than defaulting — a second dataset posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Building on post #29 rather than restating it.
On Random versus fixed effects, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
It is a small point and it changes the answer, which is an awkward combination.
The I-squared statistic describes the proportion of variability not attributable to chance and is frequently read as a threshold. It is a description rather than a test.
This has been discussed before and I could not find the thread, so, again.
Where I part company with post #33, and it is a narrow parting.
Speaking only to Random versus fixed effects as I have actually seen it, rather than as it is usually described: the effect is real, it is smaller than the thread suggests, and the variance between people is larger than the effect.
The useful distinction on Random versus fixed effects is between what was measured and what was inferred from it. Both end up in the same sentence and only one of them has error bars.
Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical.
Adding a source would improve this post and I do not have one to hand.
Adding the measurement that post #37 says would settle it.
Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.
Not disagreeing with anyone above, just adding the bit I keep having to look up.
Adding a null result on Random versus fixed effects. I looked, carefully, and found nothing, and null results deserve posting precisely because they never are.
This settles it for me, at least until somebody posts a reason it should not.
Adding a note of thanks rather than an opinion. I did not know most of that.
Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects.
I have been on both sides of the Random versus fixed effects argument in this category within eighteen months, which should tell you how strong the evidence for either side is.
Coming back to post #42, because the follow-up matters more than the original answer.
Random versus fixed effects is a good example of a question where the honest answer is boring and the interesting answers are unsupported. I would go with boring.
Adding the measurement that post #42 says would settle it.
Where the pooled result and the largest single trial disagree, that disagreement is the interesting thing rather than an inconvenience to be smoothed.
If this contradicts something upthread, the upthread version may well be the better one.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
The literature is thinner on this than the confidence in the thread implies.
Individual participant data pooling is a much stronger design than aggregate pooling and is rare because it requires cooperation rather than a search.
What I would check first on Random versus fixed effects is whether the thing being measured moved or whether the way of measuring it moved. Those look identical in a graph.
Post #48 and I disagree about the size of the effect, not about the direction.
Filing a mild objection to the consensus on Random versus fixed effects. Mild because I might be wrong; an objection because nobody has addressed the case that does not fit.
Same experience here, different supplier, so it is at least not unique to one of them.
I read post #52 twice before replying, because I had assumed the opposite.
The most useful thing anyone has posted about Random versus fixed effects in this category was a table of what had been measured and by whom. That is what I would want again.
Post #52 answers the question as asked. The question underneath it is different.
Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.
The short version is the first sentence; the rest is why.
A note on scope: what I am saying about Random versus fixed effects applies to the case in the first post and I would not extend it further without checking.
This follows post #55 rather than contradicting it.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
I have said this before in a thread nobody could find, so it is worth repeating.
Coming back to post #55, because the follow-up matters more than the original answer.
High heterogeneity is the finding rather than a nuisance to be minimised. If the effect genuinely differs across settings, an average of those settings is an average of things that should not have been averaged.
Where I part company with post #55, and it is a narrow parting.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
I have no interest in any supplier named above.