Run-in periods and the population they select posts 91–120

That is a cleaner way of putting what I was circling around.

Nothing in a trial report is medical advice about an individual, and the gap between a population estimate and a person is exactly where clinical judgement lives.

Reading it back, the second half matters more than the first.

I had written a reply contradicting post #103 and deleted it. Here is what survived.

One more thing on run-in periods that took me far too long to see: the two figures people quote are not measuring the same quantity. Once you notice that, the apparent contradiction disappears.

Confirming post #107 from a second method, which matters more than confirming it from a second person.

Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.

I read the earlier replies on run-in periods twice before writing this, because I had assumed the opposite and wanted to be sure I was disagreeing with what was said rather than what I expected.

Taking post #107 at face value and following it one step further.

The honest answer on run-in periods is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter.

Most people get the first two right and then argue about the fourth.

Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.

A single observation, in a thread that deserves better than single observations.

Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at the cost of external validity.

I would put the burden of proof on the interesting explanation, not the dull one.

Building on post #110 rather than restating it.

I keep a log for run-in periods specifically because my memory of it turned out to be systematically wrong in one direction. Six weeks of notes cost nothing and settled it.

Clear enough that I do not think I have a follow-up, which is unusual.

The confident answers on run-in periods and the well-sourced answers are not the same answers, which is the most useful thing I have learned reading this category.

A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.

That is the honest state of it as of this week.

Before the thread moves on from run-in periods — what is the sample size behind the claim? I am not being difficult; I have seen the same figure quoted from an n of four and from an n of four hundred.

Where I part company with post #116, and it is a narrow parting.

Adding the boring version of run-in periods, because the interesting version keeps getting posted and the boring one is usually right.

Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the time the second one does.

Picking up post #118: that is the part I would want checked first.

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

I would put moderate confidence on the mainstream reading of run-in periods and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.