Evidence · Trials · continued

Comparators chosen for regulatory reasons rather than clinical ones — one year on posts 31–60

Evidence › Trials randomised trial intention to treat open label estimand Solved

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.

v.milanoviTL3Regular16 May 2025#31

a.ibarra, post #23: Everything in post #19 holds. The case it does not cover is the one I have. Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot. Go to post

An open-label trial is not worthless and its subjective endpoints deserve more scepticism than its objective ones. That is a graded judgement rather than a verdict.

That is all I can say without guessing.

22 likes in reply to #23 14mo

ne.laurentTL217 May 2025#32

Agreed on Comparators chosen for regulatory reasons, with one qualification that I think matters. The reasoning holds for the case as described. Change the starting assumption and it does not, and the starting assumption is the part nobody states.

0 likes 14mo

n.bridgewaterTL2Member19 May 2025 · edited#33

Confirming post #30 from a second method, which matters more than confirming it from a second person.

A methods point on Comparators chosen for regulatory reasons rather than a substantive one: if the comparison is not like for like, the difference you are measuring is the difference in method.

3 likes 14mo

h.fonsecaTL220 May 2025#34

I had written a reply contradicting post #32 and deleted it. Here is what survived.

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

10 likes 14mo

integrator_traceTL2Member22 May 2025#35

This follows post #34 rather than contradicting it.

Comparators chosen for regulatory reasons is well covered in the tag pages, and the older discussions are better than the recent ones because they were argued out properly. Worth twenty minutes before adding to this one.

30 likes in reply to #23 14mo

n.chowdhuryTL223 May 2025#36

v.milanovi, post #31: An open-label trial is not worthless and its subjective endpoints deserve more scepticism than its objective ones. That is a graded judgement rather than a verdict. That is all I can say without guessing. Go to post

Non-inferiority margins are chosen, and the choice is an argument rather than a fact. A wide margin can make a worse treatment look acceptable.

The step people skip is the one I have spelled out.

0 likes in reply to #31 14mo

ambient_draftTL3Regular24 May 2025#37

Useful. I had the fact and not the reason, which turns out to be the important half.

5 likes 14mo

s.lundgrenTL226 May 2025#38

Coming back to post #36, because the follow-up matters more than the original answer.

Marking my uncertainty on Comparators chosen for regulatory reasons explicitly. I am confident about the direction, much less confident about the size, and not confident at all that it generalises past the case in the first post.

15 likes 14mo

e.silvaTL227 May 2025#39

The arithmetic in post #38 is right; the assumption feeding it is the part to check.

Adding a reference point for Comparators chosen for regulatory reasons. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.

0 likes 14mo

an.adeyemiTL229 May 2025#40

Answering the question post #36 raises rather than the one it answers.

A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.

1 like 14mo

aliquot_lineTL3Regular30 May 2025#41

e.ferrari, post #28: Adding a note of thanks rather than an opinion. I did not know most of that. Go to post

Where the Comparators chosen for regulatory reasons discussion usually stalls is that nobody wants to say "I do not know" and everyone is willing to say "it varies". Those are the same sentence with different clothes on.

32 likes in reply to #28 14mo

e.nilsenTL231 May 2025#42

The arithmetic on Comparators chosen for regulatory reasons is the easy part and it is where the errors are, which is an uncomfortable combination. Show your working and someone will catch it.

16 likes 14mo

NicolaidesTL32 Jun 2025#43

Collapsed as off-topic by two members at trust level 3 or above

A trial that answers a slightly different question from the one you have is the normal situation rather than a failure of the trial. The skill is describing the gap precisely.

v.stanescuTL23 Jun 2025#44

an.adeyemi, post #40: Answering the question post #36 raises rather than the one it answers. A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers. Go to post

Adding the measurement that post #41 says would settle it.

If you are new and reading this thread for the answer to Comparators chosen for regulatory reasons: the answer is conditional, the conditions are in the third reply, and the rest of the thread is worth skipping.

1 like in reply to #40 14mo

glossary_deskTL3Regular4 Jun 2025#45

a.weiss, post #27: I read post #23 twice before replying, because I had assumed the opposite. What I would tell a new member reading about Comparators chosen for regulatory reasons for the first time: the confident posts are not the reliable ones, and the reliable ones are longer. Go to post

Coming back to post #41, because the follow-up matters more than the original answer.

The bit of Comparators chosen for regulatory reasons that nobody enjoys is that the answer changes depending on what you are trying to decide with it. Say what the decision is and the thread will converge.

24 likes in reply to #27 14mo

f.piresTL26 Jun 2025#46

Composite endpoints should be read component by component. A composite driven entirely by its softest component is a different finding from one where the components move together.

I am reporting what happened, not recommending it.

11 likes 14mo

DKwiatkowskiTL3Regular7 Jun 2025 · edited#47

Something worth flagging about Comparators chosen for regulatory reasons: the strongest-sounding claims in this thread are the ones with no source attached, which is the usual pattern and not a coincidence.

3 likes 14mo

l.krastevTL28 Jun 2025#48

This is the sort of exchange that makes the archive worth searching.

0 likes 14mo

s.stavrianosTL2Member10 Jun 2025#49

v.stanescu, post #44: Adding the measurement that post #41 says would settle it. If you are new and reading this thread for the answer to Comparators chosen for regulatory reasons: the answer is conditional, the conditions are in the third reply, and the rest of the thread is worth skipping. Go to post

Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot.

0 likes in reply to #44 14mo

j.lokkenTL211 Jun 2025#50

h.fonseca, post #34: I had written a reply contradicting post #32 and deleted it. Here is what survived. Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers. Go to post

Building on post #49 rather than restating it.

Multiplicity and multiple comparisons: if a trial tests many hypotheses, the chance of a false positive on at least one by random chance increases. This is why pre-specification of the primary endpoint matters and why secondary endpoints are weaker evidence.

30 likes in reply to #34 14mo

d.nilsenTL212 Jun 2025#51

k.roos, post #26: Composite endpoints should be read component by component. A composite driven entirely by its softest component is a different finding from one where the components move together. Go to post

Building on post #49 rather than restating it.

A request rather than an answer: could whoever has the primary source for Comparators chosen for regulatory reasons post it? I have seen the claim three times this month and each version had lost a qualifier.

0 likes in reply to #26 14mo

OTeixeiraTL3Regular13 Jun 2025#52

Post #50 put the caveat in the right place and I want to underline it.

Source for the Comparators chosen for regulatory reasons figure, since it was asked for. It is in the discussion rather than the abstract, which is why the version circulating is stronger than the paper is.

Reading the surrounding paragraph is worth the two minutes. The authors are more careful than their summarisers.

1 like 13mo

n.zielinskiTL215 Jun 2025#53

Non-inferiority margins are chosen, and the choice is an argument rather than a fact. A wide margin can make a worse treatment look acceptable.

7 likes 13mo

MJayawardenaTL3Regular16 Jun 2025#54

Speaking only to Comparators chosen for regulatory reasons as I have actually seen it, rather than as it is usually described: the effect is real, it is smaller than the thread suggests, and the variance between people is larger than the effect.

17 likes 13mo

b.teixeiraTL217 Jun 2025#55

Post #52 is the version of this I will quote in future. One addition.

I changed my mind about Comparators chosen for regulatory reasons after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.

0 likes in reply to #34 13mo

DOdendaalTL3Regular18 Jun 2025#56

f.pires, post #46: Composite endpoints should be read component by component. A composite driven entirely by its softest component is a different finding from one where the components move together. I am reporting what happened, not recommending it. Go to post

Pre-specification is the property that makes a primary endpoint trustworthy. An endpoint chosen after seeing the data can be the best endpoint in the world and it no longer carries the same guarantee.

0 likes in reply to #46 13mo

r.ilungaTL220 Jun 2025#57

Collapsed as off-topic by two members at trust level 3 or above

On Comparators chosen for regulatory reasons, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.

If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.

e.dalgleishTL3Regular21 Jun 2025#58

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

It took me longer than it should have to see that.

12 likes 13mo

e.roosTL222 Jun 2025#59

Sensible. I would want the same detail before I acted on it either.

1 like 13mo

s.grahameTL2Member23 Jun 2025#60

Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.

6 likes 13mo