Adding a small correction to the Regression summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
7 likes 18mo
Research Methods · Statistics · continued
Second pass at: Regression to the mean in progress reports posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
z.vogel, post #17: What I can speak to on Regression is narrow, so I will keep it narrow rather than generalising from it. Beyond that boundary I do not know. Go to post
Noted, and I have changed what I was going to do on the strength of it.
Regression to the mean: if you select people with extreme values (very high or very low), their next measurement is often less extreme just by chance. This can look like a treatment effect when it is just statistics.
Correct me on the arithmetic if it is wrong; I would rather know.
0 likes 18mo
r.mensah, post #23: I read post #21 twice before replying, because I had assumed the opposite. A request rather than an answer: could whoever has the primary source for Regression post it? I have seen the claim three times this month and each version had lost a qualifier. Go to post
Time-to-event analysis handles differing follow-up in a way a simple proportion cannot, which is why event rates and Kaplan-Meier estimates can differ.
The general case is well covered; this is the awkward specific one.
a.molnar, post #4: Confirming the opening post from a second method, which matters more than confirming it from a second person. P-values and significance: p Go to post
Survivorship in a self-reporting population biases every aggregate produced from it, and the bias is in the flattering direction.
I would not lead a decision with this, but I would not ignore it either.
s.kravchenko, post #13: Fine by me. I had wanted a stronger conclusion and there is not one available. Go to post
Answering the question post #38 raises rather than the one it answers.
Regression to the mean: if you select people with extreme values (very high or very low), their next measurement is often less extreme just by chance. This can look like a treatment effect when it is just statistics.
I had written a reply contradicting post #39 and deleted it. Here is what survived.
A note on how Regression gets discussed rather than on Regression itself: the confident posts get the replies and the careful ones get ignored, and the careful ones have been right more often.
20 likes 18mo
Worth stating the null on Regression before we explain it: the observation may be nothing. That possibility deserves a sentence and usually does not get one.
0 likes 18mo
j.dahlberg, post #33: Regression to the mean: if you select people with extreme values (very high or very low), their next measurement is often less extreme just by chance. This can look like a treatment effect when it is just statistics. Correct me on the arithmetic if it is wrong; I would rather know. Go to post
Rounding and significant figures carry information about precision. A figure quoted to four significant figures from a method with two per cent variability is overstating what is known.
The strength of my opinion here exceeds the strength of my evidence.
f.espinoza, post #15: Building on post #14 rather than restating it. Medians and means diverge for skewed distributions, and most of the quantities discussed here are skewed. Which one a paper reports is a choice worth noticing. Go to post
Regression to the mean explains a large share of apparent improvements in anything selected for being extreme. It is not a statistical curiosity; it is the default explanation.
That is the version I would defend. It is not the version I started with.
Post #43 is right about the mechanism and I think understates the practical bit.
Reporting rather than recommending, on Regression. What happened is above. Whether it should have is a different question and not one I am qualified to answer.
5 likes 18mo
A p-value is the probability of data at least this extreme given the null hypothesis. It is not the probability the hypothesis is false, and almost every plain-language gloss gets that backwards.
I have separated what I observed from what I concluded, which does not always happen.
28 likes 18mo
The arithmetic on Regression is the easy part and it is where the errors are, which is an uncomfortable combination. Show your working and someone will catch it.
19 likes 18mo
Building on post #50 rather than restating it.
Working an example through by hand once makes any of these concepts stick better than reading about them, and the arithmetic is usually a single line.
29 likes 18mo
Post #48 put the caveat in the right place and I want to underline it.
A standard deviation describes the spread of individuals and a standard error describes the precision of the mean. Quoting one where the other belongs changes the apparent result substantially.
I would rather say I do not know than round it up to an answer.
0 likes 18mo
h.fonseca, post #49: Since Regression keeps coming up, it should probably be a maintained page rather than a recurring thread. I am happy to draft it if someone with more direct experience will review it. Go to post
The thing about Regression that took me longest to accept is that a plausible mechanism is not evidence of an effect. It is a reason to look, not a result.
Post #52 and I disagree about the size of the effect, not about the direction.
One caution on Regression: everything above assumes the underlying documentation is what it claims to be. That assumption is doing real work and is rarely stated.
0 likes 17mo
e.mwangi, post #47: My position on Regression is current rather than settled. I have revised it once already and I expect to again, so treat it accordingly. Go to post
Medians and means diverge for skewed distributions, and most of the quantities discussed here are skewed. Which one a paper reports is a choice worth noticing.
Nothing above should be read as advice about what anyone else should do.
c.kuusela, post #2: Right, and stated more narrowly than I would have dared to state it. Go to post
Baseline imbalance in a randomised trial is expected by chance and adjusting for it post hoc is a choice that should have been pre-specified.
If that is already documented somewhere, ignore me and link it.
The number people quote for Regression is a central estimate presented without its interval, and the interval is wide enough that the estimate is nearly uninformative on its own.
30 likes 17mo