The Peptide CommonsEst. May 2024
Independent. We sell nothing and are affiliated with no manufacturer or pharmacy. Every moderation action is logged in public
Research Methods · Statistics

What a confidence interval means, from scratch — the long version

OV
o.vogelTL219 Sep 2025#1

The question in the title: What a confidence interval means, from scratch — the long version I will give what I have already checked below so nobody repeats it.

Asking about confidence interval directly, because I have read four threads on it and each answered a slightly different question.

The version I want answered is the narrow one: given the method stated below, is the result within what anyone else has seen? I am not asking what it means yet.

Method, numbers and the two assumptions I am aware of making are below. If the assumptions are wrong that is more useful to me than agreement.

3 likes 10mo
TS
t.steenkampTL2Member25 Sep 2025 · edited#2

The opening post answers the question as asked. The question underneath it is different.

A confidence interval is a statement about a procedure rather than about this particular interval, and the practical reading — plausible values consistent with the data — is close enough for most purposes and worth knowing is an approximation.

Adding it because I spent an afternoon working it out and nobody should have to twice.

0 likes 10mo
AW
ai.wikstromTL230 Sep 2025#3

Two things can be true about confidence interval at once: the mechanism is plausible and the evidence for the size of the effect is thin. Most of the argument here is people defending the first against attacks on the second.

25 likes 10mo
CE
crossover_entryTL3Regular4 Oct 2025#4

That is a fair summary of where the discussion has got to.

12 likes 10mo
BW
br.wikstromTL28 Oct 2025#5
t.steenkamp, post #2: The opening post answers the question as asked. The question underneath it is different. A confidence interval is a statement about a procedure rather than about this particular interval, and the practical reading — plausible values consistent with the data — is close enough for most purposes and worth knowing is an approximation.… Go to post

Coming back to post #3, because the follow-up matters more than the original answer.

Medians and means diverge for skewed distributions, and most of the quantities discussed here are skewed. Which one a paper reports is a choice worth noticing.

Correct me on the arithmetic if it is wrong; I would rather know.

4 likes in reply to #2 10mo
GR
gradient_reviewTL2Member12 Oct 2025#6
o.vogel, post #1: The question in the title: What a confidence interval means, from scratch — the long version I will give what I have already checked below so nobody repeats it. Asking about confidence interval directly, because I have read four threads on it and each answered a slightly different question. The version I want answered is the narrow one:… Go to post

Confidence interval is one of those subjects where the general answer and the answer for a specific case diverge, and the thread will go in circles until someone says which one is being asked for.

0 likes in reply to #1 10mo
MM
m.marchettiTL215 Oct 2025#7

Practical note on confidence interval: write down what you expect before you look. The number of times I have found what I went looking for is higher than chance would allow.

0 likes 9mo
MD
methods_draftTL2Member18 Oct 2025#8

Survivorship in a self-reporting population biases every aggregate produced from it, and the bias is in the flattering direction.

If anyone can point at the primary source I would be grateful.

17 likes 9mo
NR
n.ramosTL222 Oct 2025#9

Source for the confidence interval figure, since it was asked for. It is in the discussion rather than the abstract, which is why the version circulating is stronger than the paper is.

Reading the surrounding paragraph is worth the two minutes. The authors are more careful than their summarisers.

0 likes 9mo
S
SHermansenTL2Member25 Oct 2025#10

A request rather than an answer: could whoever has the primary source for confidence interval post it? I have seen the claim three times this month and each version had lost a qualifier.

26 likes 9mo
TI
trough_indexTL3Regular28 Oct 2025#11

Building on post #8 rather than restating it.

Number needed to treat: how many people need to be treated to prevent one bad outcome or achieve one good outcome. More intuitive than relative risk reduction.

0 likes 9mo
EM
e.mwangiTL231 Oct 2025 · edited#12
crossover_entry, post #4: That is a fair summary of where the discussion has got to. Go to post

Post #10 put the caveat in the right place and I want to underline it.

Confidence interval sits at the boundary between what this community can usefully discuss and what it cannot, and I think it falls on the discussable side, narrowly.

1 like in reply to #4 9mo
NB
n.bridgewaterTL2Member3 Nov 2025#13

An honest declaration on confidence interval: I have a prior here and it is strong enough that you should weight what I say downward. Stating it rather than hiding it.

6 likes 9mo
AN
a.norgaardTL26 Nov 2025#14
TN
t.ndiayeTL29 Nov 2025#15

The strongest argument against my own position on confidence interval, stated as well as I can state it, since nobody else has yet.

5 likes 9mo
AF
a.friskTL211 Nov 2025#16
crossover_entry, post #4: That is a fair summary of where the discussion has got to. Go to post

Clear enough that I do not think I have a follow-up, which is unusual.

3 likes in reply to #4 9mo
K
KTurkingtonTL3Regular14 Nov 2025#17

Narrowing post #15, because the general version has more than one answer.

Effect size with an interval is more informative than a significance verdict in essentially every case discussed here.

I have no interest in any supplier named above.

10 likes 8mo
TD
t.demirTL217 Nov 2025#18

Distinguishing three things in the confidence interval discussion that keep getting used interchangeably: the observation, the proposed mechanism, and the recommendation that gets attached to both.

22 likes 8mo
IT
integrator_traceTL2Member20 Nov 2025#19
gradient_review, post #6: Confidence interval is one of those subjects where the general answer and the answer for a specific case diverge, and the thread will go in circles until someone says which one is being asked for. Go to post

The practical version of confidence interval is three sentences long. The rigorous version is three pages and reaches the same conclusion with the conditions attached.

23 likes in reply to #6 8mo
AK
ak.kravchenkoTL222 Nov 2025#20
trough_index, post #11: Building on post #8 rather than restating it. Number needed to treat: how many people need to be treated to prevent one bad outcome or achieve one good outcome. More intuitive than relative risk reduction. Go to post

Bayesian and frequentist analyses answer different questions and both are legitimate. What matters is that the reader knows which is on offer.

0 likes in reply to #11 8mo
GD
glossary_deskTL3Regular25 Nov 2025#21

Rounding and significant figures carry information about precision. A figure quoted to four significant figures from a method with two per cent variability is overstating what is known.

It is the kind of thing that is obvious once and never again.

6 likes 8mo

Suggested topics

TopicParticipantsRepliesViewsActivity
Sample size intuition for a personal experiment — does this still hold?
Sample size intuition for a personal experiment — does this still hold? I have a specific reason for asking rather than idle curiosity, and the context is below. Reading back through what has been written…
AWMSNVBIFA+33 38 26k 23d
About the Statistics category
Effect sizes, intervals, multiplicity, and the difference between absent and undetected. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its…
SLJMBVCAV+4 8 2.3k 7mo
What a confidence interval means, from scratch
What a confidence interval means, from scratch — that is the question, and I have not found it answered plainly anywhere I have looked. Asking about confidence interval directly, because I have read four…
MHSNO 2 2.6k 2mo
Measurement error in home scales, with a worked standard deviation — the long version
On the subject in the title: Measurement error in home scales, with a worked standard deviation — the long version Working notes rather than a conclusion. A narrow question about Measurement error in home…
SKRRAWDYJD+96 105 2k 2mo
[2026 update] Correlation in a self-tracked dataset: what it can support
Posting this under the heading it deserves: Correlation in a self-tracked dataset: what it can support Everything below is what sits behind that. Posting a small dataset on Correlation. It is mine, it is…
GTARLS 2 31k 6h

Related topics — sharing the tags effect size, worked example, confounding

TopicParticipantsRepliesViewsActivity
[2026 update] Journal club: SURPASS-2 and the semaglutide 1 mg comparator
Journal club: SURPASS-2 and the semaglutide 1 mg comparator Writing it up because I had to work it out twice and would rather nobody else did. Session topic: SURPASS-2 ( N Engl J Med , 2021). Please read it…
EDHCITINR+126 138 31k 17mo
Coming back to: Feature request: should a dose-equivalence tool exist at all?
Feature request: should a dose-equivalence tool exist at all? I have a specific reason for asking rather than idle curiosity, and the context is below. Reading back through what has been written here about…
MSERSIKRMN+37 41 725 7mo
Critiquing an observational claim about a class effect
Posting this under the heading it deserves: Critiquing an observational claim about a class effect Everything below is what sits behind that. I was wrong about critiquing an observational claim in a thread…
VRRIJV 2 6.9k 14mo
Journal club: the retatrutide phase 2 obesity paper
Journal club: the retatrutide phase 2 obesity paper — setting out what I have, and where I think it stops being reliable. Collecting what is known about retatrutide phase 2 obesity paper in one place, because…
LFVDAERASV+7 11 33k 7mo
Random versus fixed effects: choosing rather than defaulting
Random versus fixed effects: choosing rather than defaulting — setting out what I have, and where I think it stops being reliable. Random versus fixed effects, and specifically the version of it that the…
NPFSMLAJS+16 20 22k 13mo