Random versus fixed effects: choosing rather than defaulting — setting out what I have, and where I think it stops being reliable.
Random versus fixed effects, and specifically the version of it that the documentation does not cover. The maintained page handles the general case well and stops exactly where my question starts.
Setting out the gap in case it is a gap in the page rather than a gap in what is known.
Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.
Taking post #3 at face value and following it one step further.
Where the pooled result and the largest single trial disagree, that disagreement is the interesting thing rather than an inconvenience to be smoothed.
I would want a second opinion before relying on that.
Everything in post #3 holds. The case it does not cover is the one I have.
Practical note on random versus fixed effects: write down what you expect before you look. The number of times I have found what I went looking for is higher than chance would allow.
That is a fair summary of where the discussion has got to.
The I-squared statistic describes the proportion of variability not attributable to chance and is frequently read as a threshold. It is a description rather than a test.
That is the distinction I keep failing to hold on to. Written down now.
Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.
A single observation, in a thread that deserves better than single observations.
The version of random versus fixed effects that circulates here is a simplification of a simplification. It is not wrong, but it has lost the conditions under which it holds, and those conditions are where the interesting cases live.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Marking that as an opinion rather than a finding.
High heterogeneity is the finding rather than a nuisance to be minimised. If the effect genuinely differs across settings, an average of those settings is an average of things that should not have been averaged.
That is what the documentation says. What happens in practice is usually close.
Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects.
The reasoning is more useful than the number, which is why I have shown it.
Double extraction with disagreement resolution is standard and is worth checking for, because single extraction errors are common and non-random.
That is the honest state of it as of this week.
I read post #14 twice before replying, because I had assumed the opposite.
Fixed-effect and random-effects models answer different questions. The first assumes one true effect; the second assumes a distribution of them. Choosing between them is an assumption, not a technicality.
Taking random versus fixed effects seriously for a moment rather than deflecting: the honest position is that the community has observations and no controlled comparison, and those two things support very different sentences.
Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.
Worth separating two things that post #19 runs together.
An observation about random versus fixed effects that I cannot explain and am posting anyway, on the principle that unexplained observations are more useful public than private.
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Random versus fixed effects: choosing rather than defaulting — a second dataset
On the subject in the title: Random versus fixed effects: choosing rather than defaulting — a second dataset Working notes rather than a conclusion. Posting a small dataset on Random versus fixed effects. It…
|
+95 | 106 | 20k | 17mo |
|
Publication bias detection and its low power — does this still hold?
The question in the title: Publication bias detection and its low power — does this still hold? I will give what I have already checked below so nobody repeats it. Collecting what is known about Publication…
|
+24 | 28 | 2.4k | 3mo |
|
A pooled estimate that changed when one trial was added
A pooled estimate that changed when one trial was added — setting out what I have, and where I think it stops being reliable. I was wrong about pooled estimate in a thread last spring and I would like to…
|
+24 | 28 | 3.5k | 2d |
|
Prediction intervals and why they are more honest than confidence intervals
Posting this under the heading it deserves: Prediction intervals and why they are more honest than confidence intervals Everything below is what sits behind that. I have spent a fortnight trying to pin…
|
+21 | 25 | 46k | 4mo |
|
Risk-of-bias tools and the judgement they conceal — does this still hold?
Risk-of-bias tools and the judgement they conceal — does this still hold? — that is the question, and I have not found it answered plainly anywhere I have looked. Asking about Risk-of-bias tools on behalf of…
|
+26 | 30 | 5.8k | 23h |
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Preprint servers and what screening they do — the long version
On the subject in the title: Preprint servers and what screening they do — the long version Working notes rather than a conclusion. A preprint question rather than a question about the finding in it. The…
|
+7 | 11 | 26k | 4mo |
|
[2026 update] Confounding by indication, explained with a concrete example
Confounding by indication, explained with a concrete example Writing it up because I had to work it out twice and would rather nobody else did. I have seen SURPASS-4 ( Lancet , 2021) cited in support of a…
|
+103 | 116 | 22k | 5mo |
|
About the Statistics category
Effect sizes, intervals, multiplicity, and the difference between absent and undetected. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its…
|
+4 | 8 | 2.3k | 7mo |
|
Designing a personal experiment that could change your mind
Posting this under the heading it deserves: Designing a personal experiment that could change your mind Everything below is what sits behind that. I was wrong about designing a personal experiment in a thread…
|
+2 | 6 | 695 | 6mo |
|
Journal club: SUSTAIN 6 and its retinopathy signal — does this still hold?
The question in the title: Journal club: SUSTAIN 6 and its retinopathy signal — does this still hold? I will give what I have already checked below so nobody repeats it. Reading SUSTAIN 6 ( N Engl J Med ,…
|
+60 | 66 | 12k | 16mo |