Evidence · Study critique

A claim built entirely on a subgroup analysis — what changed since

Evidence › Study critique disputed effect size observational data risk of bias

Closed

a.weissTL27 Jan 2025#1

On the subject in the title: A claim built entirely on a subgroup analysis — what changed since Working notes rather than a conclusion.

What changes if the standard account of claim built is wrong? I ask because I have been treating it as settled and I noticed this week that I could not say why.

Working through the consequences rather than the evidence, since others here are better placed on the evidence.

36 likes 19mo

z.cardosoTL223 Jan 2025 · edited#2

Where a critique rests on a subgroup, check whether the interaction was tested. A difference between subgroups that is not statistically supported is not a difference.

0 likes 18mo

q.zhao_qaTL3Quality assurance3 Feb 2025#3

Picking up post #2: that is the part I would want checked first.

Bias towards the null and bias away from the null: different criticisms have different directions. Differential dropout might bias away from null; conservative statistical analysis might bias toward null.

That is what I would do. It may not be what is correct.

0 likes 18mo

This topic was closed 60 days after the last reply. Closing is automatic for quiet topics so that a settled answer does not collect new questions underneath it. If you have a follow-up, open a new topic and link back to this one — that keeps both readable and gives your question its own title.

Suggested topics

Topic	Participants	Replies	Views	Activity
A structured critique template this community uses Evidence › Study critique observational data effect size risk of bias A structured critique template this community uses — setting out what I have, and where I think it stops being reliable. Structured critique template — I have the observation and I do not trust my…		2	18k	1d
[2026 update] Confounding by indication, explained with a concrete example Evidence › Study critique observational data disputed risk of bias Confounding by indication, explained with a concrete example Writing it up because I had to work it out twice and would rather nobody else did. I have seen SURPASS-4 ( Lancet , 2021) cited in support of a…	+103	116	22k	5mo
A well-designed study with a badly written abstract Evidence › Study critique disputed effect size risk of bias observational data Posting this under the heading it deserves: A well-designed study with a badly written abstract Everything below is what sits behind that. Reading back through what has been written here about well-designed…	+117	128	7.2k	14mo
Reverse causation in a cohort study of weight and outcome — a second dataset Evidence › Study critique risk of bias observational data confounding Reverse causation in a cohort study of weight and outcome — a second dataset Writing it up because I had to work it out twice and would rather nobody else did. Posting a small dataset on Reverse causation. It…	+112	131	1.5k	10mo
About the Study critique category Evidence › Study critique risk of bias confounding observational data Pinned Wiki Structured criticism of a specific published study. Criticise the method, not the author. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its…		4	114	16mo

Related topics — sharing the tags disputed, effect size, observational data

Topic	Participants	Replies	Views	Activity
Journal club: how we handle discontinuation in obesity trials — what changed since Evidence › Journal club number needed to treat estimand risk of bias randomised trial Solved Journal club: how we handle discontinuation in obesity trials — what changed since — setting out what I have, and where I think it stops being reliable. Session topic: STEP 4 ( JAMA , 2021). Please read it…	+14	18	35k	8mo
BPC-157: what the rodent literature actually shows, and what it does not Compounds › Repair & healing peptides observational data preprint BPC-157 needs a source BPC-157: what the rodent literature actually shows, and what it does not — setting out what I have, and where I think it stops being reliable. I would like to know what people here actually do about BPC-157,…	+44	48	3k	10mo
About the Datasets category Data & Tools › Datasets data table observational data confounding Pinned Wiki Community-collected datasets, their collection methods, and their limitations. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its author and a…	+6	11	42k	12mo
The "under review" status pill and when a moderator applies it Evidence › Preprints risk of bias effect size preprint Posting this under the heading it deserves: The "under review" status pill and when a moderator applies it Everything below is what sits behind that. A question about how this site should cite something that…	+96	119	17k	3mo
Sample size in a voluntary survey: the selection problem Data & Tools › Datasets observational data data table site feedback Solved Posting this under the heading it deserves: Sample size in a voluntary survey: the selection problem Everything below is what sits behind that. Sample size: setting out the arithmetic in full, because I have…	+1	5	12k	15mo

Sponsored Sigma-Aldrich — certified reference materialsReference standards, analytical reagents and certified materials for method calibration. If you are validating a method, this is where the standard comes from.Reference standards →