This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Adding the measurement that post #28 says would settle it.
Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.
I would treat the number as indicative rather than as a measurement.
Post #30 describes the usual case. This is about the unusual one.
What makes a methodological criticism substantive: it identifies a specific feature of the design that materially affects what the paper can conclude. "Small sample size" alone is weak. "Small sample size for a rare outcome, so the confidence interval is wide" is stronger.
The rule of thumb is fine; the edge cases are where it earns its keep.
Reverse causation sits at the boundary between what this community can usefully discuss and what it cannot, and I think it falls on the discussable side, narrowly.
Distinguishing three things in the Reverse causation discussion that keep getting used interchangeably: the observation, the proposed mechanism, and the recommendation that gets attached to both.
Marking my place. If it changes for me I will come back and say so.
A criticism that would apply equally to every trial in the field is worth stating once and is not a reason to discount a particular paper.
Having read the whole Reverse causation thread before replying: the question in the first post has not actually been answered yet, and three of us have answered a nearby one instead.
The strongest argument against my own position on Reverse causation, stated as well as I can state it, since nobody else has yet.
Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not.
The failure mode on Reverse causation is boring rather than dramatic. It is almost always the step everyone assumes was done correctly because it is too simple to get wrong.
The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.
Saving this. It is the version I will quote when the question comes round again.
Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.
This is where my knowledge stops and I would rather mark the edge than blur it.
Answering the question post #43 raises rather than the one it answers.
Careful with the language on Reverse causation. "Not detected" and "not present" are different findings and the first is a statement about the method.
Reframing Reverse causation slightly, because I think the disagreement is about the question rather than the answer. If the question is "does it happen", yes. If it is "how often", nobody here knows.
Post #48 describes the usual case. This is about the unusual one.
Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence.
None of the above is medical advice and I am not qualified to give any.
A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.
Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for.
Agreed, and I will stop repeating the version of this I had been repeating.
Something worth flagging about Reverse causation: the strongest-sounding claims in this thread are the ones with no source attached, which is the usual pattern and not a coincidence.
Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.
Written quickly, so the reasoning may be tighter than the wording.
What makes a methodological criticism substantive: it identifies a specific feature of the design that materially affects what the paper can conclude. "Small sample size" alone is weak. "Small sample size for a rare outcome, so the confidence interval is wide" is stronger.