Evidence · Trials · continued

Comparators chosen for regulatory reasons rather than clinical ones — one year on posts 61–88

Evidence › Trials randomised trial intention to treat open label estimand Solved

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.

b.dumitruTL225 Jun 2025#61

a.weiss, post #27: I read post #23 twice before replying, because I had assumed the opposite. What I would tell a new member reading about Comparators chosen for regulatory reasons for the first time: the confident posts are not the reliable ones, and the reliable ones are longer. Go to post

Nothing in a trial report is medical advice about an individual, and the gap between a population estimate and a person is exactly where clinical judgement lives.

If anyone has run this properly I would rather read that than my own guess.

27 likes in reply to #27 13mo

electrolyte_notesTL2Regular26 Jun 2025#62

i.almeida, post #14: Comparators chosen for regulatory reasons was covered in the wiki last year and the page has a review date on it, which is a better starting point than my memory of a thread. Go to post

Open-label design: unblinded trials admit expectation effects. For weight-loss trials where one arm loses substantial weight and the other does not, complete blinding is impossible anyway. The unblinded nature is a limitation worth noting.

Not a conclusion. A place to stand while looking for one.

13 likes in reply to #14 13mo

z.cardosoTL227 Jun 2025 · edited#63

On post #60 — agreed on the reasoning, with one qualification.

Genuine question rather than a rhetorical one: has anyone here actually observed Comparators chosen for regulatory reasons, as opposed to read about it? The thread is long and I cannot tell.

2 likes 13mo

d.moreauTL2Regular28 Jun 2025#64

Picking up post #63: that is the part I would want checked first.

Practical answer on Comparators chosen for regulatory reasons, since the theoretical one is upthread: do the simplest check first, write down the result, and only then decide whether the complicated explanation is needed. It usually is not.

0 likes 13mo

s.ivaturiTL230 Jun 2025#65

Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.

0 likes 13mo

k.redgraveTL2Member1 Jul 2025#66

KLindqvist, post #22: A note on scope: what I am saying about Comparators chosen for regulatory reasons applies to the case in the first post and I would not extend it further without checking. Go to post

Number needed to treat is only interpretable with the duration attached. The same NNT over one year and over five years describes very different clinical situations.

18 likes in reply to #22 13mo

v.bruunTL22 Jul 2025 · edited#67

Right — I had this wrong and I am glad to have read it before it mattered.

4 likes 13mo

nl_translatorTL2Translator · NL3 Jul 2025#68

Comparators chosen for regulatory reasons is a question about a distribution, not about a value, and treating it as a value is what produces the confident wrong answers.

0 likes 13mo

n.villalobosTL24 Jul 2025#69

glossary_desk, post #45: Coming back to post #41, because the follow-up matters more than the original answer. The bit of Comparators chosen for regulatory reasons that nobody enjoys is that the answer changes depending on what you are trying to decide with it. Say what the decision is and the thread will converge. Go to post

Two people in this thread mean different things by Comparators chosen for regulatory reasons and are disagreeing about the definition while believing they are disagreeing about the facts. Worth pausing to define it.

13 likes in reply to #45 13mo

blank_injectionTL2Analytical chemist6 Jul 2025#70

Post #68 answers the question as asked. The question underneath it is different.

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

Worth one more sentence than it usually gets.

5 likes 13mo

a.novakTL27 Jul 2025#71

Taking post #70 at face value and following it one step further.

An open-label trial is not worthless and its subjective endpoints deserve more scepticism than its objective ones. That is a graded judgement rather than a verdict.

Second-hand, so weight it accordingly.

0 likes 13mo

bench_notesTL4 Moderator8 Jul 2025#72

r.ilunga, post #57: On Comparators chosen for regulatory reasons, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit. If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about… Go to post

Post #68 and I disagree about the size of the effect, not about the direction.

Non-inferiority margins are chosen, and the choice is an argument rather than a fact. A wide margin can make a worse treatment look acceptable.

Posting it because the silence on this was starting to look like agreement.

0 likes in reply to #57 13mo

e.vargaTL29 Jul 2025#73

l.ibarra, post #2: Open-label design: unblinded trials admit expectation effects. For weight-loss trials where one arm loses substantial weight and the other does not, complete blinding is impossible anyway. The unblinded nature is a limitation worth noting. That is what the documentation says. What happens in practice is usually close. Go to post

Practical note on Comparators chosen for regulatory reasons: write down what you expect before you look. The number of times I have found what I went looking for is higher than chance would allow.

8 likes in reply to #2 13mo

r.aldana_pharmdTL4Pharmacist10 Jul 2025#74

Comparators chosen for regulatory reasons is one of those subjects where the general answer and the answer for a specific case diverge, and the thread will go in circles until someone says which one is being asked for.

20 likes 13mo

s.okaforTL212 Jul 2025#75

This follows post #74 rather than contradicting it.

Reading it again, the caveat matters more than the finding.

28 likes 13mo

lc_gradientTL3Analytical chemist13 Jul 2025#76

nl_translator, post #68: Comparators chosen for regulatory reasons is a question about a distribution, not about a value, and treating it as a value is what produces the confident wrong answers. Go to post

That is consistent with mine, for whatever one more account is worth.

0 likes in reply to #68 13mo

d.vukovicTL214 Jul 2025#77

Collapsed as off-topic by two members at trust level 3 or above

Two things can be true about Comparators chosen for regulatory reasons at once: the mechanism is plausible and the evidence for the size of the effect is thin. Most of the argument here is people defending the first against attacks on the second.

dr_bhattacharyaTL3Physician15 Jul 2025 · edited#78

Coming back to post #75, because the follow-up matters more than the original answer.

Before the thread moves on from Comparators chosen for regulatory reasons — what is the sample size behind the claim? I am not being difficult; I have seen the same figure quoted from an n of four and from an n of four hundred.

14 likes 12mo

m.stephanopoulosTL3Regular16 Jul 2025#79

A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.

21 likes 12mo

n.norgaardTL217 Jul 2025#80

Two claims get bundled together under Comparators chosen for regulatory reasons and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.

Almost every disagreement in threads like this one dissolves once you say which of the two you are making.

0 likes 12mo

l.chevalierTL3Regular19 Jul 2025#81

j.lokken, post #50: Building on post #49 rather than restating it. Multiplicity and multiple comparisons: if a trial tests many hypotheses, the chance of a false positive on at least one by random chance increases. This is why pre-specification of the primary endpoint matters and why secondary endpoints are weaker evidence. Go to post

I had written a reply contradicting post #78 and deleted it. Here is what survived.

I would call the community position on Comparators chosen for regulatory reasons likely rather than established, and I would be comfortable defending that hedge.

15 likes in reply to #50 12mo

m.adebayoTL220 Jul 2025#82

n.bridgewater, post #33: Confirming post #30 from a second method, which matters more than confirming it from a second person. A methods point on Comparators chosen for regulatory reasons rather than a substantive one: if the comparison is not like for like, the difference you are measuring is the difference in method. Go to post

Composite endpoints should be read component by component. A composite driven entirely by its softest component is a different finding from one where the components move together.

Correct me on the arithmetic if it is wrong; I would rather know.

5 likes in reply to #33 12mo

taper_tableTL3Regular21 Jul 2025#83

Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.

I would call that likely rather than established.

1 like 12mo

p.boatengTL222 Jul 2025#84

One more thing on Comparators chosen for regulatory reasons that took me far too long to see: the two figures people quote are not measuring the same quantity. Once you notice that, the apparent contradiction disappears.

0 likes 12mo

IsaksenTL3Regular23 Jul 2025#85

electrolyte_notes, post #62: Open-label design: unblinded trials admit expectation effects. For weight-loss trials where one arm loses substantial weight and the other does not, complete blinding is impossible anyway. The unblinded nature is a limitation worth noting. Not a conclusion. A place to stand while looking for one. Go to post

If anyone can point at the primary source I would be grateful.

21 likes in reply to #62 12mo

t.ibarraTL224 Jul 2025#86

I have no financial interest in anything named in this thread and I want to say so before I comment on Comparators chosen for regulatory reasons, because it is the sort of subject where it matters.

9 likes 12mo

vial_deskTL3Regular26 Jul 2025 · edited#87

Helpful, and easy to find again, which is half of what a good reply is.

2 likes 12mo

a.eriksenTL227 Jul 2025#88

The arithmetic in post #85 is right; the assumption feeding it is the part to check.

The failure mode on Comparators chosen for regulatory reasons is boring rather than dramatic. It is almost always the step everyone assumes was done correctly because it is too simple to get wrong.

0 likes 12mo

This topic was referenced in

Second pass at: Trial registration and comparing the protocol with the paperEvidence › Trials · 34 replies

Promoted into the documentation commons. The content of this topic is maintained at STEP 2 — trial digest, with named maintainers and a review date. The promotion was discussed in doc review. Corrections are best raised against the document, which is the version that gets kept current.

Topic	Participants	Replies	Views	Activity
Adjudicated events and why the definition matters — the long version Evidence › Trials intention to treat estimand randomised trial discontinuation & dropout Adjudicated events and why the definition matters — the long version Writing it up because I had to work it out twice and would rather nobody else did. A narrow question about Adjudicated events, deliberately…	+125	146	13k	15mo
Interim analyses and stopping rules Evidence › Trials intention to treat surrogate endpoints open label discontinuation & dropout Interim analyses and stopping rules Writing it up because I had to work it out twice and would rather nobody else did. Asking about interim analyses and stopping rules on behalf of the question I keep seeing…	+25	29	35k	16d
About the Trials category Evidence › Trials randomised trial estimand intention to treat Pinned Wiki Individual randomised trials: design, population, endpoints, results, and limitations. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its…	+6	10	27k	12mo
Trial registration and comparing the protocol with the paper Evidence › Trials randomised trial open label intention to treat Trial registration and comparing the protocol with the paper — setting out what I have, and where I think it stops being reliable. I have spent a fortnight trying to pin trial registration down and I want to…	+38	44	4.8k	10mo
Second pass at: Trial registration and comparing the protocol with the paper Evidence › Trials estimand discontinuation & dropout randomised trial open label Posting this under the heading it deserves: Second pass at: Trial registration and comparing the protocol with the paper Everything below is what sits behind that. Two things I would like separated before…	+30	34	33k	13mo

Topic	Participants	Replies	Views	Activity
Journal club: what we got wrong in an earlier session Evidence › Journal club number needed to treat effect size randomised trial journal club On the subject in the title: Journal club: what we got wrong in an earlier session Working notes rather than a conclusion. Bringing a claim rather than a paper, because the claim is the thing circulating. The…	+114	125	2.5k	23h
Journal club: SELECT, absolute risk, and how it was reported — a second dataset Evidence › Journal club risk of bias effect size randomised trial journal club On the subject in the title: Journal club: SELECT, absolute risk, and how it was reported — a second dataset Working notes rather than a conclusion. Posting a small dataset on SELECT. It is mine, it is…		4	1.3k	1mo
How to read a forest plot, properly, from scratch Evidence › Trials randomised trial intention to treat surrogate endpoints estimand Solved The question in the title: How to read a forest plot, properly, from scratch I will give what I have already checked below so nobody repeats it. Reading STEP 4 ( JAMA , 2021) for the population rather than…	+73	81	17k	7mo
Why fasting instructions for oral semaglutide are not optional advice Compounds › Oral incretins PIONEER programme orforglipron randomised trial SOUL trial Solved Why fasting instructions for oral semaglutide are not optional advice I have a specific reason for asking rather than idle curiosity, and the context is below. Fasting instructions for oral semaglutide, from…	+70	74	1.1k	23d
Journal club: SURPASS-2 and the semaglutide 1 mg comparator Evidence › Journal club estimand effect size risk of bias journal club Solved On the subject in the title: Journal club: SURPASS-2 and the semaglutide 1 mg comparator Working notes rather than a conclusion. Session topic: SURPASS-2 ( N Engl J Med , 2021). Please read it before posting;…	+71	76	52k	21mo

Comparators chosen for regulatory reasons rather than clinical ones — one year on posts 61–88

This topic was referenced in

Suggested topics

Related topics — sharing the tags randomised trial, intention to treat, open label