Collapsed as off-topic by two members at trust level 3 or above
The bit of STEP 1 that nobody enjoys is that the answer changes depending on what you are trying to decide with it. Say what the decision is and the thread will converge.
Evidence · Journal club · continued
Journal club: STEP 1 and the treatment-policy estimand — does this still hold? posts 61–90
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Post #60 is the version of this I will quote in future. One addition.
Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend.
That is the practical version. The rigorous version is longer and says the same thing.
0 likes 15mo
p.novotny, post #44: Adding a reference point for STEP 1. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately. Go to post
Offering a way to settle STEP 1 rather than another opinion about it. Two measurements, taken the same way, a fortnight apart. If the difference is within the noise, the question was not answerable at this precision.
r.mcalister, post #59: This is the answer, and the reason it is the answer is the more useful part. Go to post
Reframing STEP 1 slightly, because I think the disagreement is about the question rather than the answer. If the question is "does it happen", yes. If it is "how often", nobody here knows.
Noted, and I have changed what I was going to do on the strength of it.
31 likes 15mo
Confirming post #64 from a second method, which matters more than confirming it from a second person.
The baseline characteristics table is the single most useful page for the questions that get asked here, because it tells you who the result applies to.
0 likes 15mo
Where I part company with post #67, and it is a narrow parting.
Where the group cannot agree, record the disagreement rather than resolving it by seniority. The recorded disagreement is more honest and more useful later.
I would not lead a decision with this, but I would not ignore it either.
3 likes 15mo
STEP 1 (N Engl J Med 2021): The pivotal obesity trial for semaglutide and the reference point for most subsequent comparison. Mean weight reduction was substantially larger than anything previously achieved pharmacologically.
I would want to see it done twice before believing it once.
11 likes 15mo
tracked_parcel, post #48: Quietly grateful for the plain phrasing. Not every thread gets that. Go to post
Understood, and I withdraw the assumption I opened with.
Coming back to post #69, because the follow-up matters more than the original answer.
Session format: read the paper before posting. The discussion is much better when everyone has. Start with the estimand and population, then methods, then results, then limitations. That order makes critique coherent.
14 likes 15mo
Post #73 is right about the mechanism and I think understates the practical bit.
Taking STEP 1 seriously for a moment rather than deflecting: the honest position is that the community has observations and no controlled comparison, and those two things support very different sentences.
5 likes 15mo
The most common failure mode is spending fifty minutes on the effect size and ten on the population. Reversing that ratio would improve most sessions.
0 likes 15mo
p.boateng, post #26: Having read the whole STEP 1 thread before replying: the question in the first post has not actually been answered yet, and three of us have answered a nearby one instead. Go to post
What I would want before treating STEP 1 as settled: the method, the sample, and whether anyone tried to find the opposite result. Two of the three are usually missing.
Confirming post #77 from a second method, which matters more than confirming it from a second person.
Reading order that works for these sessions: registry entry, methods, baseline table, primary result, then abstract last. Reading the abstract first anchors everything that follows.
I am aware this is the third time this month I have made this point.
8 likes 15mo
Adding the measurement that post #77 says would settle it.
The baseline characteristics table is the single most useful page for the questions that get asked here, because it tells you who the result applies to.
15 likes 15mo
Where the group cannot agree, record the disagreement rather than resolving it by seniority. The recorded disagreement is more honest and more useful later.
One more caveat and then I will stop qualifying: the sample selected itself.
0 likes 15mo
STEP 1 (N Engl J Med 2021): The pivotal obesity trial for semaglutide and the reference point for most subsequent comparison. Mean weight reduction was substantially larger than anything previously achieved pharmacologically.
The answer changed when I changed how I was measuring, which was informative.
0 likes 15mo
owen.brady, post #76: The most common failure mode is spending fifty minutes on the effect size and ten on the population. Reversing that ratio would improve most sessions. Go to post
Worth separating two things that post #80 runs together.
My position on STEP 1 is current rather than settled. I have revised it once already and I expect to again, so treat it accordingly.
Collapsed as off-topic by two members at trust level 3 or above
Where the STEP 1 discussion usually stalls is that nobody wants to say "I do not know" and everyone is willing to say "it varies". Those are the same sentence with different clothes on.
a.westergaard, post #2: Where the group cannot agree, record the disagreement rather than resolving it by seniority. The recorded disagreement is more honest and more useful later. Go to post
Post #84 and I disagree about the size of the effect, not about the direction.
Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend.
s.okafor, post #31: A definition problem is doing most of the work in this STEP 1 discussion. Once the term is pinned down I suspect the disagreement mostly goes away and what is left is small. Go to post
STEP 1 is worth one more sentence than it usually gets, and the sentence is the one about how the number was arrived at.
Everything in post #88 holds. The case it does not cover is the one I have.
An update on my earlier STEP 1 post: the pattern held for another six weeks and then stopped, which I did not predict and cannot explain.
4 likes 15mo