Topic summary

A well-designed study with a badly written abstract

Evidence › Study critique disputed effect size risk of bias observational data

This is a generated summary. It shows the 9 most-liked posts from a topic of 129, in their original order, with the accepted answer included where one exists. It is a reading aid and it will miss nuance — the full topic is the record.

m.ilungaTL229 Mar 2025#2

Reporting bias within a paper — a result in a figure that is absent from the abstract — is checkable and is worth checking when the claim matters.

28 likes 16mo

n.bridgewaterTL2Member10 Apr 2025#18

Well-designed study would be much easier to settle if anyone reported the denominator. Almost nobody reports the denominator.

28 likes 16mo

k.farrugiaTL3Regular13 Apr 2025#22

e.silva, post #12: I think the well-designed study question is answerable and has not been answered, which is a more optimistic position than most of this thread. Go to post

Where I part company with post #21, and it is a narrow parting.

Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.

29 likes in reply to #12 15mo

e.krastevTL217 Apr 2025#29

Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters.

If the premise is wrong, everything after it is decoration.

28 likes 15mo

am.wikstromTL220 Apr 2025#34

h.koodziej, post #14: This follows post #13 rather than contradicting it. A note on scope: what I am saying about well-designed study applies to the case in the first post and I would not extend it further without checking. Go to post

My experience of well-designed study contradicts the reply above. I am posting it as a data point rather than as a refutation, because one person's experience is exactly that.

32 likes in reply to #14 15mo

r.mensaTL224 Apr 2025#41

Speaking only to well-designed study as I have actually seen it, rather than as it is usually described: the effect is real, it is smaller than the thread suggests, and the variance between people is larger than the effect.

32 likes 15mo

y.asanteTL212 May 2025#76

Reading back through, this was answered upthread and I missed it. My fault.

33 likes 15mo

l.chevalierTL3Regular17 May 2025 · edited#87

Post #84 is right about the mechanism and I think understates the practical bit.

Adding a reference point for well-designed study. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.

30 likes 14mo

e.vargaTL218 May 2025#91

Understood. Thank you for being specific about the limits of it.

28 likes 14mo

Read the full topic (129 posts)

This topic was referenced in

Measurement error in a self-reported exposureEvidence › Study critique · 129 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Coming back to: Measurement error in a self-reported exposure Evidence › Study critique risk of bias observational data effect size disputed Measurement error in a self-reported exposure — setting out what I have, and where I think it stops being reliable. Reading back through what has been written here about Measurement error, three questions…	+23	27	49k	3mo
Selection into a registry and what it does to the estimate Evidence › Study critique risk of bias disputed observational data Selection into a registry and what it does to the estimate — setting out what I have, and where I think it stops being reliable. Working notes on selection rather than a conclusion. I would rather post the…		2	55k	7mo
A claim built entirely on a subgroup analysis — what changed since Evidence › Study critique disputed effect size observational data risk of bias Closed On the subject in the title: A claim built entirely on a subgroup analysis — what changed since Working notes rather than a conclusion. What changes if the standard account of claim built is wrong? I ask…		2	2k	18mo
Criticising the method without criticising the authors Evidence › Study critique disputed observational data confounding effect size Criticising the method without criticising the authors Writing it up because I had to work it out twice and would rather nobody else did. I was wrong about criticising the method without criticising in a…	+55	61	60k	15mo
Critiquing an observational claim about a class effect Evidence › Study critique disputed risk of bias effect size observational data Posting this under the heading it deserves: Critiquing an observational claim about a class effect Everything below is what sits behind that. I was wrong about critiquing an observational claim in a thread…		2	6.9k	14mo

Related topics — sharing the tags disputed, effect size, risk of bias

Topic	Participants	Replies	Views	Activity
Publication bias detection and its low power Evidence › Meta-analyses heterogeneity publication bias effect size On the subject in the title: Publication bias detection and its low power Working notes rather than a conclusion. Working notes on publication bias detection rather than a conclusion. I would rather post the…		2	3.2k	1d
Stability of BPC-157 in solution: what has been measured Compounds › Repair & healing peptides BPC-157 observational data needs a source TB-500 Stability of BPC-157 in solution: what has been measured — setting out what I have, and where I think it stops being reliable. Something about stability of BPC-157 does not reconcile and I would like a second…		2	805	30d
Follow-up: A supplier that answers analytical questions with restated claims Sourcing › Red flags documentation quality disputed mislabelling moderation A supplier that answers analytical questions with restated claims — setting out what I have, and where I think it stops being reliable. A follow-up question about supplier that I did not know to ask the first…	+74	82	2.8k	6mo
Second pass at: Journal club: LEADER as the historical anchor Evidence › Journal club randomised trial journal club effect size estimand Posting this under the heading it deserves: Second pass at: Journal club: LEADER as the historical anchor Everything below is what sits behind that. Two things I would like separated before anyone answers on…	+75	81	2k	2y
Revisiting: Hepatic effects of glucagon receptor agonism: the mechanistic worry Compounds › Retatrutide retatrutide randomised trial effect size preprint Solved Revisiting: Hepatic effects of glucagon receptor agonism: the mechanistic worry Writing it up because I had to work it out twice and would rather nobody else did. A methods question rather than a substantive…	+10	14	8.4k	16h