This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.
The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions.
It is one reading of the data and not the only reasonable one.
Coming back to post #58, because the follow-up matters more than the original answer.
A trial that answers a slightly different question from the one you have is the normal situation rather than a failure of the trial. The skill is describing the gap precisely.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
The interesting part of this is the exception, and I do not understand the exception.
Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at the cost of external validity.
Adding a source would improve this post and I do not have one to hand.
Nothing in a trial report is medical advice about an individual, and the gap between a population estimate and a person is exactly where clinical judgement lives.
Not the answer, but possibly the question that gets there.
This follows post #64 rather than contradicting it.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
Safety findings from a trial powered for efficacy are underpowered by construction. Absence of a signal in that setting is weak evidence of absence.
Discontinuation handling is the methodological detail that most changes a result and gets the least attention. Read how missing data was imputed before reading the effect size.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
It is the sort of thing that seems obvious in retrospect and was not at the time.
That is consistent with mine, for whatever one more account is worth.
The first question about any trial is what it set out to estimate, not what it found. Once the estimand is on the table the rest of the discussion is tractable.
Taking post #73 at face value and following it one step further.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
One of those cases where knowing the mechanism does not help the decision.
Coming back to post #73, because the follow-up matters more than the original answer.
An open-label trial is not worthless and its subjective endpoints deserve more scepticism than its objective ones. That is a graded judgement rather than a verdict.
I would treat that as a working assumption and revisit it.
Post #73 is right about the mechanism and I think understates the practical bit.
A comparator at less than its maximum licensed dose changes what a head-to-head result means. It does not invalidate the trial; it narrows the claim the trial supports.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
Filing this under things that are true until someone shows me otherwise.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
That holds under the stated conditions and I have stated them.
That is a cleaner way of putting what I was circling around.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
A weak preference rather than a position.
Narrowing post #82, because the general version has more than one answer.
Intent-to-treat versus per-protocol: ITT includes everyone assigned regardless of whether they took the drug. Per-protocol includes only those who completed it as intended. The two can give substantially different results.
Everything in post #81 holds. The case it does not cover is the one I have.
Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot.
Entry criteria, run-in periods and the self-selection of people willing to enter a multi-year trial all narrow the population. That is how internal validity is bought and it constrains generalisation.
I am describing what is, rather than arguing for what should be.
The arithmetic in post #85 is right; the assumption feeding it is the part to check.
Trial duration determines what can be observed. A weight-change trajectory at 40 weeks and at 72 weeks are different observations and both get quoted as the result.
Funding and trial conduct should be stated and are a weak predictor of anything on their own. Design quality is the stronger signal and it is checkable.
Correct me on the arithmetic if it is wrong; I would rather know.
Post #87 is the version of this I will quote in future. One addition.
Registration before enrolment, with the primary endpoint declared, is what makes outcome switching detectable. Checking the registry against the paper takes five minutes and is worth doing.
Where I part company with post #88, and it is a narrow parting.
Effect sizes in a trial population reflect adherence achieved under trial conditions, which is generally better than adherence outside them.