Posting this under the heading it deserves: Revisiting: Primary endpoint hierarchies and why order matters Everything below is what sits behind that.
I was wrong about Primary endpoint hierarchies in a thread last spring and I would like to correct it publicly rather than quietly.
The error was in the units, which changed the conclusion by an order of magnitude. Setting out the corrected version, and the way I now check for that class of mistake.
Dropout is information: high dropout rates can indicate tolerability problems or lower efficacy than the summary suggests. Where the analysis handled dropouts matters. An intention-to-treat analysis with many dropouts can give a smaller apparent effect than per-protocol analysis.
Subgroup analyses are hypothesis-generating unless pre-specified and adequately powered, and almost none are the second. The interaction test matters more than the subgroup point estimate.
That is the version I use. It may not be the version that is correct.
Where I part company with post #3, and it is a narrow parting.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
I would be interested in a counterexample if anyone has one.
Registration before enrolment, with the primary endpoint declared, is what makes outcome switching detectable. Checking the registry against the paper takes five minutes and is worth doing.
Primary endpoint hierarchies is a question about a distribution, not about a value, and treating it as a value is what produces the confident wrong answers.
Worth separating two things that post #6 runs together.
Pre-specification is the property that makes a primary endpoint trustworthy. An endpoint chosen after seeing the data can be the best endpoint in the world and it no longer carries the same guarantee.
Marking that as an opinion rather than a finding.
Primary endpoint hierarchies came up in a thread eighteen months ago and was answered well. I cannot find it, which is itself the problem, so here is the reconstruction.
Post #10 describes the usual case. This is about the unusual one.
Surrogate endpoints: an endpoint that is not the outcome that matters but is measured as a stand-in. HbA1c is a surrogate for long-term glucose control and the short-term complications it prevents. Weight loss is a surrogate for metabolic health and long-term outcomes. Surrogates are useful but not identical to the endpoint that matters.
If that is already documented somewhere, ignore me and link it.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
Caveat: everything above assumes the paperwork is what it says it is.
Seconded. It reads as careful rather than confident, which is the right register.
On post #13 — agreed on the reasoning, with one qualification.
If you are new and reading this thread for the answer to Primary endpoint hierarchies: the answer is conditional, the conditions are in the third reply, and the rest of the thread is worth skipping.
Picking up post #17: that is the part I would want checked first.
The figure that circulates in coverage is almost always whichever estimand gives the larger effect. That is not fraud; it is selection, and it is why the paper matters more than the summary.
Nothing above should be read as advice about what anyone else should do.
Coming back to post #17, because the follow-up matters more than the original answer.
Multiplicity and multiple comparisons: if a trial tests many hypotheses, the chance of a false positive on at least one by random chance increases. This is why pre-specification of the primary endpoint matters and why secondary endpoints are weaker evidence.
It took me longer than it should have to see that.
Post #17 is right about the mechanism and I think understates the practical bit.
What I can speak to on Primary endpoint hierarchies is narrow, so I will keep it narrow rather than generalising from it. Beyond that boundary I do not know.
Adding a small correction to the Primary endpoint hierarchies summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
Worth separating two things that post #20 runs together.
Pre-specification is the property that makes a primary endpoint trustworthy. An endpoint chosen after seeing the data can be the best endpoint in the world and it no longer carries the same guarantee.
Post #22 answers the question as asked. The question underneath it is different.
The thing about Primary endpoint hierarchies that took me longest to accept is that a plausible mechanism is not evidence of an effect. It is a reason to look, not a result.
Adding a note of thanks rather than an opinion. I did not know most of that.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Run-in periods and the population they select — the long version
Run-in periods and the population they select — the long version Writing it up because I had to work it out twice and would rather nobody else did. A narrow question about Run-in periods, deliberately narrow,…
|
2 | 8.9k | 4mo | |
|
About the Trials category
Individual randomised trials: design, population, endpoints, results, and limitations. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its…
|
+6 | 10 | 27k | 12mo |
|
Non-inferiority margins: how they are chosen and how they are abused
Posting this under the heading it deserves: Non-inferiority margins: how they are chosen and how they are abused Everything below is what sits behind that. A question about non-inferiority margins that I…
|
+42 | 46 | 10k | 5mo |
|
Interim analyses and stopping rules
Interim analyses and stopping rules Writing it up because I had to work it out twice and would rather nobody else did. Asking about interim analyses and stopping rules on behalf of the question I keep seeing…
|
+25 | 29 | 35k | 16d |
|
Coming back to: Sample size calculations, read backwards from the published number
On the subject in the title: Sample size calculations, read backwards from the published number Working notes rather than a conclusion. Sample size calculations — I have the observation and I do not trust my…
|
+82 | 88 | 1.8k | 11mo |
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Adjudicated events and why the definition matters — the long version
Adjudicated events and why the definition matters — the long version Writing it up because I had to work it out twice and would rather nobody else did. A narrow question about Adjudicated events, deliberately…
|
+125 | 146 | 13k | 15mo |
|
Follow-up: Open-label extensions: what survives and what does not
Open-label extensions: what survives and what does not — setting out what I have, and where I think it stops being reliable. I have spent a fortnight trying to pin Open-label extensions down and I want to set…
|
+22 | 26 | 397 | 16mo |
|
Interim analyses and stopping rules — does this still hold?
Interim analyses and stopping rules — does this still hold? — that is the question, and I have not found it answered plainly anywhere I have looked. Interim analyses and stopping rules, from the point of view…
|
+30 | 34 | 29k | 13mo |
|
Journal club: the CagriSema phase 2 combination paper
On the subject in the title: Journal club: the CagriSema phase 2 combination paper Working notes rather than a conclusion. CagriSema phase 2 combination paper — I have the observation and I do not trust my…
|
2 | 329 | 21h | |
|
About the Journal club category
Our recurring session. One named paper per topic, methods first, conclusions last. Everyone reads before posting. This post is a community wiki: any member at trust level 3 or above can edit it, and every…
|
+2 | 6 | 12k | 16mo |