Evidence · Study critique · continued

Measurement error in a self-reported exposure posts 121–130

Evidence › Study critique confounding risk of bias effect size disputed Solved Closed

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.

g.ibarraTL227 Feb 2026#121

Two things can be true about measurement error at once: the mechanism is plausible and the evidence for the size of the effect is thin. Most of the argument here is people defending the first against attacks on the second.

0 likes 5mo

j.teixeiraTL227 Feb 2026#122

a.vermeulen, post #66: A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary. I am reporting what happened, not recommending it. Go to post

Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.

The number is defensible. The precision I gave it is not.

0 likes in reply to #66 5mo

GDashwoodTL3Regular27 Feb 2026#123

I would put moderate confidence on the mainstream reading of measurement error and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.

14 likes 5mo

ca.haddadTL227 Feb 2026#124

Picking up post #123: that is the part I would want checked first.

On measurement error: the maintained page in the documentation commons covers the general case with citations and a review date, which is more reliable than any reply here including this one.

5 likes 5mo

v.klausenTL3Regular28 Feb 2026#125

A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.

I am confident about the direction and much less about the magnitude.

0 likes 5mo

e.coelhoTL228 Feb 2026#126

weekly_pin, post #55: Summarising the measurement error thread so far, since it is long and the answer is buried: the first reply has the method, the fourth has the correction to it, and the rest is people agreeing at length. Go to post

That is a fair summary of where the discussion has got to.

28 likes in reply to #55 5mo

integrator_draftTL3Regular28 Feb 2026 · edited#127

Post #123 describes the usual case. This is about the unusual one.

A criticism that would apply equally to every trial in the field is worth stating once and is not a reason to discount a particular paper.

9 likes 5mo

n.szaboTL228 Feb 2026#128

Adding the measurement that post #127 says would settle it.

I disagree with the framing of measurement error above, and I think it is a substantive disagreement rather than a terminological one. Setting out why, so it can be checked.

The reasoning depends on an assumption that is doing a lot of work and is never stated. If the assumption holds, the conclusion follows. I do not think it holds generally.

2 likes 5mo

OkaforTL328 Feb 2026#129

Collapsed as off-topic by two members at trust level 3 or above

Speaking only to measurement error as I have actually seen it, rather than as it is usually described: the effect is real, it is smaller than the thread suggests, and the variance between people is larger than the effect.

f.ibarraTL228 Feb 2026#130

Building on post #127 rather than restating it.

Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence.

0 likes 5mo

This topic was referenced in

Reverse causation in a cohort study of weight and outcomeEvidence › Study critique · 14 replies
Reverse causation in a cohort study of weight and outcome — a second datasetEvidence › Study critique · 131 replies
Coming back to: Measurement error in a self-reported exposureEvidence › Study critique · 27 replies

This topic was closed 60 days after the last reply. Closing is automatic for quiet topics so that a settled answer does not collect new questions underneath it. If you have a follow-up, open a new topic and link back to this one — that keeps both readable and gives your question its own title.

Suggested topics

Topic	Participants	Replies	Views	Activity
Coming back to: Measurement error in a self-reported exposure Evidence › Study critique risk of bias observational data effect size disputed Measurement error in a self-reported exposure — setting out what I have, and where I think it stops being reliable. Reading back through what has been written here about Measurement error, three questions…	+23	27	49k	3mo
A well-designed study with a badly written abstract Evidence › Study critique disputed effect size risk of bias observational data Posting this under the heading it deserves: A well-designed study with a badly written abstract Everything below is what sits behind that. Reading back through what has been written here about well-designed…	+117	128	7.2k	14mo
Critiquing an observational claim about a class effect — one year on Evidence › Study critique observational data disputed risk of bias Pinned Critiquing an observational claim about a class effect — one year on — setting out what I have, and where I think it stops being reliable. I was wrong about Critiquing an observational claim in a thread last…	+5	9	3.2k	8mo
Immortal time bias in a claims-database study — one year on Evidence › Study critique effect size risk of bias confounding Solved Posting this under the heading it deserves: Immortal time bias in a claims-database study — one year on Everything below is what sits behind that. I would like to disagree carefully with the settled view on…	+41	46	3.7k	22h
Confounding by indication, explained with a concrete example Evidence › Study critique confounding observational data disputed On the subject in the title: Confounding by indication, explained with a concrete example Working notes rather than a conclusion. Comparing SURMOUNT-1 ( N Engl J Med , 2022) with SURPASS-2 ( N Engl J Med ,…	+21	25	21k	11mo

Related topics — sharing the tags confounding, risk of bias, effect size

Topic	Participants	Replies	Views	Activity
Why plausible mechanism is not evidence of effect Compounds › Repair & healing peptides TB-500 preprint observational data Solved Closed Asking directly, because I could not find a straight answer: Why plausible mechanism is not evidence of effect What changes if the standard account of plausible mechanism is wrong? I ask because I have been…	+14	18	57k	13mo
Journal club: LEADER as the historical anchor Evidence › Journal club estimand number needed to treat journal club effect size Pinned Journal club: LEADER as the historical anchor Writing it up because I had to work it out twice and would rather nobody else did. What changes if the standard account of LEADER is wrong? I ask because I have…	+103	110	8.1k	2h
Semaglutide half-life: where the 165 to 184 hour figure comes from Compounds › Semaglutide semaglutide effect size randomised trial type 2 diabetes Wiki Semaglutide half-life: where the 165 to 184 hour figure comes from — setting out what I have, and where I think it stops being reliable. Putting the numbers in the first post, because a question about a…	+128	145	3k	4mo
BPC-157: what the rodent literature actually shows, and what it does not — does this still hold? Compounds › Repair & healing peptides observational data needs a source BPC-157 TB-500 Solved The question in the title: BPC-157: what the rodent literature actually shows, and what it does not — does this still hold? I will give what I have already checked below so nobody repeats it. I would like to…	+6	10	3.1k	1d
Preprint servers and what screening they do — the long version Evidence › Preprints risk of bias effect size needs a source On the subject in the title: Preprint servers and what screening they do — the long version Working notes rather than a conclusion. A preprint question rather than a question about the finding in it. The…	+7	11	26k	4mo