Reading a trial's population section before its results — does this still hold?

Post #14 is the version of this I will quote in future. One addition.

Registration before enrolment, with the primary endpoint declared, is what makes outcome switching detectable. Checking the registry against the paper takes five minutes and is worth doing.

Noting that I have skin in this question and have tried to discount for it.

Where I part company with post #13, and it is a narrow parting.

Funding and trial conduct should be stated and are a weak predictor of anything on their own. Design quality is the stronger signal and it is checkable.

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

Happy to be the one who is wrong here if it settles the question.

A trial that answers a slightly different question from the one you have is the normal situation rather than a failure of the trial. The skill is describing the gap precisely.

That is what I would do. It may not be what is correct.

Trial duration determines what can be observed. A weight-change trajectory at 40 weeks and at 72 weeks are different observations and both get quoted as the result.

That has held every time I have looked, which is not the same as always.

Reading the supplementary appendix is where most of the real information is, and it is where almost nobody goes. The baseline table alone answers half the generalisability questions asked here.

Anyone who has looked at this more carefully, please correct the record.

Everything in post #17 holds. The case it does not cover is the one I have.

Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.

Not a conclusion. A place to stand while looking for one.

The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions.

I have written this out at length because the short version keeps being misread.

Reading back through, this was answered upthread and I missed it. My fault.

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.

The general case is well covered; this is the awkward specific one.

Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.

Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot.

Building on post #25 rather than restating it.

Intent-to-treat versus per-protocol: ITT includes everyone assigned regardless of whether they took the drug. Per-protocol includes only those who completed it as intended. The two can give substantially different results.

If anyone has run this properly I would rather read that than my own guess.

Multiplicity and multiple comparisons: if a trial tests many hypotheses, the chance of a false positive on at least one by random chance increases. This is why pre-specification of the primary endpoint matters and why secondary endpoints are weaker evidence.