On the subject in the title: Regression to the mean in progress reports Working notes rather than a conclusion.
Something about regression does not reconcile and I would like a second pair of eyes before I decide which half is wrong.
Two sources, both reputable, giving figures that cannot both be right unless they are measuring different quantities. My suspicion is that they are, and I cannot see how.
Bayesian and frequentist analyses answer different questions and both are legitimate. What matters is that the reader knows which is on offer.
I have deliberately not rounded that, because the rounding is where the argument starts.
The opening post describes the usual case. This is about the unusual one.
Regression: I have looked for the primary source twice and failed twice. Either it does not exist or it is somewhere I do not know to look, and I would like to know which.
Building on post #2 rather than restating it.
P-values and significance: p<0.05 means the data would be surprising if the null hypothesis were true, not that the null hypothesis is false. A non-significant p-value does not mean "no effect".
I have said this before in a thread nobody could find, so it is worth repeating.
Regression is a good example of a question where the honest answer is boring and the interesting answers are unsupported. I would go with boring.
A standard deviation describes the spread of individuals and a standard error describes the precision of the mean. Quoting one where the other belongs changes the apparent result substantially.
Adding this to the thread rather than to the wiki, because I am not confident enough for the wiki.
Reading this regression thread as someone who came in with a fixed view: the third and seventh replies moved me and the confident ones did not.
Post #8 and I disagree about the size of the effect, not about the direction.
Baseline imbalance in a randomised trial is expected by chance and adjusting for it post hoc is a choice that should have been pre-specified.
Posted with less confidence than the sentence structure implies.
Medians and means diverge for skewed distributions, and most of the quantities discussed here are skewed. Which one a paper reports is a choice worth noticing.
Adding the measurement that post #10 says would settle it.
The most common statistical error in this community is not technical: it is treating a self-selected collection of reports as a sample from a population.
This is the first time the answer has come with its own limits attached. Appreciated.
What would change my mind on regression is a second dataset collected by someone with no stake in the first. Until then I hold it loosely and I would rather say so than pretend to more.
Answering the question post #12 raises rather than the one it answers.
I changed my mind about regression after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.
The arithmetic in post #15 is right; the assumption feeding it is the part to check.
Number needed to treat: how many people need to be treated to prevent one bad outcome or achieve one good outcome. More intuitive than relative risk reduction.
Written quickly, so the reasoning may be tighter than the wording.
Confounding: a third variable explains an apparent association. In randomised data, randomisation balances confounders. In observational data, confounders can be adjusted for but unknown ones cannot.
If anyone has run this properly I would rather read that than my own guess.
Regression to the mean: if you select people with extreme values (very high or very low), their next measurement is often less extreme just by chance. This can look like a treatment effect when it is just statistics.
Not a conclusion. A place to stand while looking for one.
Before the thread moves on from regression — what is the sample size behind the claim? I am not being difficult; I have seen the same figure quoted from an n of four and from an n of four hundred.
Multiple testing inflates the false-positive rate in a way that is entirely predictable and entirely correctable. The correction should be declared in advance.
Genuinely open to being wrong about this one.
I read post #18 twice before replying, because I had assumed the opposite.
Posting my regression numbers with the method attached so they can be discounted properly. Uncontrolled, unblinded, and collected by someone who wanted a particular answer.
Seconded. It reads as careful rather than confident, which is the right register.
A p-value is the probability of data at least this extreme given the null hypothesis. It is not the probability the hypothesis is false, and almost every plain-language gloss gets that backwards.
The short answer was in the first line; everything after is the working.
Regression to the mean: if you select people with extreme values (very high or very low), their next measurement is often less extreme just by chance. This can look like a treatment effect when it is just statistics.
Rounding and significant figures carry information about precision. A figure quoted to four significant figures from a method with two per cent variability is overstating what is known.
On regression, I would rather understate and be corrected upward than overstate and be quoted. That is a house style here and it is a good one.