Reading a supplementary appendix and finding the interesting part posts 61–90
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Adding the measurement that post #59 says would settle it.
Worth stating the null on reading a supplementary appendix before we explain it: the observation may be nothing. That possibility deserves a sentence and usually does not get one.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
That is a description of practice, not a recommendation of it.
Reading a supplementary appendix came up in a thread eighteen months ago and was answered well. I cannot find it, which is itself the problem, so here is the reconstruction.
The arithmetic in post #63 is right; the assumption feeding it is the part to check.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
Checked the reading a supplementary appendix claim against the primary source this morning. It survives, with a narrower scope than the version quoted here. Posting the narrower scope.
Acknowledging rather than arguing. The reasoning holds as far as I can follow it.
Collapsed as off-topic by two members at trust level 3 or above
Adding the measurement that post #70 says would settle it.
The thing about reading a supplementary appendix that took me longest to accept is that a plausible mechanism is not evidence of an effect. It is a reason to look, not a result.
Post #69 describes the usual case. This is about the unusual one.
Posting my reading a supplementary appendix numbers with the method attached so they can be discounted properly. Uncontrolled, unblinded, and collected by someone who wanted a particular answer.
That is a fair summary of where the discussion has got to.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
It cost nothing to check and would have cost something not to.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
That is the practical version. The rigorous version is longer and says the same thing.
On post #72 — agreed on the reasoning, with one qualification.
Reading a supplementary appendix is a good example of a question where the honest answer is boring and the interesting answers are unsupported. I would go with boring.
Adding a small correction to the reading a supplementary appendix summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
Reading a supplementary appendix looks different depending on whether you are reading the primary literature or the summaries of it, and the difference is not in our favour.
Collapsed as off-topic by two members at trust level 3 or above
Small methodological point on reading a supplementary appendix: repeating a measurement is cheap and resolves most of what is being argued about here at no cost to anyone.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
I would put the burden of proof on the interesting explanation, not the dull one.
Building on post #81 rather than restating it.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
Post #81 describes the usual case. This is about the unusual one.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
That is the honest state of it as of this week.
What I want from this reading a supplementary appendix thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.
Reading a supplementary appendix: I have looked for the primary source twice and failed twice. Either it does not exist or it is somewhere I do not know to look, and I would like to know which.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
Post #89 answers the question as asked. The question underneath it is different.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
That is the version I use. It may not be the version that is correct.