Asking directly, because I could not find a straight answer: Journal club: STEP 1 and the treatment-policy estimand — does this still hold?
Working notes on STEP 1 rather than a conclusion. I would rather post the reasoning while it can still be corrected than post the answer once I am committed to it.
Everything below is checkable. Where I have taken a figure from somewhere else I have said where; where I have estimated, I have said that too.
I read the opening post twice before replying, because I had assumed the opposite.
Where the group cannot agree, record the disagreement rather than resolving it by seniority. The recorded disagreement is more honest and more useful later.
I would not lead a decision with this, but I would not ignore it either.
Summarising the STEP 1 thread so far, since it is long and the answer is buried: the first reply has the method, the fourth has the correction to it, and the rest is people agreeing at length.
Narrowing post #6, because the general version has more than one answer.
The figures usually contain the finding and the text usually contains the interpretation. Separating them for the first twenty minutes is a discipline worth keeping.
That is the practical version. The rigorous version is longer and says the same thing.
The most useful reply I ever got about STEP 1 was a request to state my units. It sounds like pedantry and it has saved me twice.
Post #8 describes the usual case. This is about the unusual one.
Reading order that works for these sessions: registry entry, methods, baseline table, primary result, then abstract last. Reading the abstract first anchors everything that follows.
Everything in post #7 holds. The case it does not cover is the one I have.
The arithmetic on STEP 1 is the easy part and it is where the errors are, which is an uncomfortable combination. Show your working and someone will catch it.
My position on STEP 1 is current rather than settled. I have revised it once already and I expect to again, so treat it accordingly.
The most common failure mode is spending fifty minutes on the effect size and ten on the population. Reversing that ratio would improve most sessions.
Posting it because the silence on this was starting to look like agreement.
Fine by me. I had wanted a stronger conclusion and there is not one available.
Where the STEP 1 discussion usually stalls is that nobody wants to say "I do not know" and everyone is willing to say "it varies". Those are the same sentence with different clothes on.
The baseline characteristics table is the single most useful page for the questions that get asked here, because it tells you who the result applies to.
Somebody will have a better source than mine, and I hope they post it.
Confirming post #20 from a second method, which matters more than confirming it from a second person.
The figures usually contain the finding and the text usually contains the interpretation. Separating them for the first twenty minutes is a discipline worth keeping.
If anyone has run this properly I would rather read that than my own guess.
Post #22 describes the usual case. This is about the unusual one.
Session format: read the paper before posting. The discussion is much better when everyone has. Start with the estimand and population, then methods, then results, then limitations. That order makes critique coherent.
The general case is well covered; this is the awkward specific one.
The question underneath STEP 1 is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it.
Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help.
The arithmetic in post #24 is right; the assumption feeding it is the part to check.
Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend.
None of the above is medical advice and I am not qualified to give any.
Answering the question post #26 raises rather than the one it answers.
The most common failure mode is spending fifty minutes on the effect size and ten on the population. Reversing that ratio would improve most sessions.
I would call that likely rather than established.
Posting my STEP 1 numbers with the method attached so they can be discounted properly. Uncontrolled, unblinded, and collected by someone who wanted a particular answer.