This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Taking post #90 at face value and following it one step further.
A note on scope: what I am saying about Reverse causation applies to the case in the first post and I would not extend it further without checking.
Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.
I read post #92 twice before replying, because I had assumed the opposite.
Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters.
I would put the burden of proof on the interesting explanation, not the dull one.
The most useful thing anyone has posted about Reverse causation in this category was a table of what had been measured and by whom. That is what I would want again.
Narrowing post #94, because the general version has more than one answer.
Hold a trial to the standard something could actually have met. A criticism that no achievable design could have answered is a criticism of the field rather than of the paper.
That is one dataset and I would not build a rule on it.
Everything in post #96 holds. The case it does not cover is the one I have.
I would rather this thread reach "we do not know" about Reverse causation than reach a confident answer that nobody can support when asked.
On Reverse causation, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
Where I part company with post #96, and it is a narrow parting.
Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for.
The answer changed when I changed how I was measuring, which was informative.
Two claims get bundled together under Reverse causation and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
I changed my mind about Reverse causation after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.
Where I part company with post #102, and it is a narrow parting.
The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.
What would change my mind on Reverse causation is a second dataset collected by someone with no stake in the first. Until then I hold it loosely and I would rather say so than pretend to more.
Useful. I have added it to my own notes with the date on it.
Post #107 put the caveat in the right place and I want to underline it.
The useful distinction on Reverse causation is between what was measured and what was inferred from it. Both end up in the same sentence and only one of them has error bars.
Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.
Everything in post #107 holds. The case it does not cover is the one I have.
Speaking only to Reverse causation as I have actually seen it, rather than as it is usually described: the effect is real, it is smaller than the thread suggests, and the variance between people is larger than the effect.
I would be cautious about generalising from the Reverse causation example above. It is a good example. It is one example.
Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not.
I would not lead a decision with this, but I would not ignore it either.
Same experience here, different supplier, so it is at least not unique to one of them.
Narrowing post #112, because the general version has more than one answer.
A criticism that would apply equally to every trial in the field is worth stating once and is not a reason to discount a particular paper.
For what it is worth, the same held on the two occasions I checked.
A definition problem is doing most of the work in this Reverse causation discussion. Once the term is pinned down I suspect the disagreement mostly goes away and what is left is small.
The arithmetic in post #117 is right; the assumption feeding it is the part to check.
Filing a mild objection to the consensus on Reverse causation. Mild because I might be wrong; an objection because nobody has addressed the case that does not fit.
Where I part company with post #117, and it is a narrow parting.
Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for.
Reading it back, the second half matters more than the first.
Reverse causation was covered in the wiki last year and the page has a review date on it, which is a better starting point than my memory of a thread.