A structured critique template this community uses — a second dataset

On the subject in the title: A structured critique template this community uses — a second dataset Working notes rather than a conclusion.

Posting a small dataset on structured critique template. It is mine, it is uncontrolled, and the method is stated so it can be discounted appropriately.

What I would like is not agreement but a second dataset collected by someone with no stake in mine. If one exists I would rather read it than argue for this one.

A methods point on structured critique template rather than a substantive one: if the comparison is not like for like, the difference you are measuring is the difference in method.

Confirming post #15 from a second method, which matters more than confirming it from a second person.

Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.

Post #29 put the caveat in the right place and I want to underline it.

A note on how structured critique template gets discussed rather than on structured critique template itself: the confident posts get the replies and the careful ones get ignored, and the careful ones have been right more often.

Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for.

That is a description of practice, not a recommendation of it.

Post #42 is the version of this I will quote in future. One addition.

Adding the boring version of structured critique template, because the interesting version keeps getting posted and the boring one is usually right.

Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the time the second one does.