Reading a supplementary appendix and finding the interesting part — setting out what I have, and where I think it stops being reliable.
Collecting what is known about reading a supplementary appendix in one place, because it is currently spread across a category, two tag pages and a thread that is hard to find.
This is a summary rather than new work, and I have attributed each part to where I found it.
Confirming the opening post from a second method, which matters more than confirming it from a second person.
Two sentences on reading a supplementary appendix and then I will stop, because the rest is speculation and the thread is better without mine.
What is documented is narrow. What is inferred from it is broad. The gap between them is where every argument here lives.
On the opening post — agreed on the reasoning, with one qualification.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
Reading a supplementary appendix: I would want to see the raw numbers rather than the summary before agreeing. Summaries lose exactly the information that would settle this.
Filing a mild objection to the consensus on reading a supplementary appendix. Mild because I might be wrong; an objection because nobody has addressed the case that does not fit.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
For what it is worth, the same held on the two occasions I checked.
Following, with nothing to contribute beyond having asked the same thing elsewhere.
This follows post #6 rather than contradicting it.
I would be cautious about generalising from the reading a supplementary appendix example above. It is a good example. It is one example.
Discontinuation handling is the methodological detail that most changes a result and gets the least attention. Read how missing data was imputed before reading the effect size.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
If anyone has run this properly I would rather read that than my own guess.
Picking up post #8: that is the part I would want checked first.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
Old habit: I write down the expected answer before I calculate it.
The documentation on reading a supplementary appendix is better than this thread and I say that as someone who has posted in the thread.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
The general answer and the answer for your case may diverge here.
Two claims get bundled together under reading a supplementary appendix and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
No notes. Posting so the count is not one.
Reading the supplementary appendix is where most of the real information is, and it is where almost nobody goes. The baseline table alone answers half the generalisability questions asked here.
That is the version I would defend. It is not the version I started with.
I read post #14 twice before replying, because I had assumed the opposite.
On reading a supplementary appendix, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
Reading a supplementary appendix has been discussed here with more heat than it deserves, mostly because two definitions have been in play the whole time.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
The rule of thumb is fine; the edge cases are where it earns its keep.
Worth separating two things that post #19 runs together.
Adding a reference point for reading a supplementary appendix. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
Noting that the question and the thing people usually mean by it are different.
Post #23 and I disagree about the size of the effect, not about the direction.
One more thing on reading a supplementary appendix that took me far too long to see: the two figures people quote are not measuring the same quantity. Once you notice that, the apparent contradiction disappears.
I would call the community position on reading a supplementary appendix likely rather than established, and I would be comfortable defending that hedge.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
I would treat that as a working assumption and revisit it.
Narrowing post #26, because the general version has more than one answer.
The failure mode on reading a supplementary appendix is boring rather than dramatic. It is almost always the step everyone assumes was done correctly because it is too simple to get wrong.
Composite endpoints should be read component by component. A composite driven entirely by its softest component is a different finding from one where the components move together.