Taking post #8 at face value and following it one step further. A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary. I would treat that as a working assumption and revisit it.
A claim built entirely on a subgroup analysis Writing it up because I had to work it out twice and would rather nobody else did.
I would like to disagree carefully with the settled view on claim built, and I would like to be argued out of it if the disagreement is bad.
The disagreement is about one step, not about the conclusion. If the step holds I withdraw it entirely.
I had written a reply contradicting the opening post and deleted it. Here is what survived.
Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.
The literature is thinner on this than the confidence in the thread implies.
The opening post describes the usual case. This is about the unusual one.
An observation about claim built that I cannot explain and am posting anyway, on the principle that unexplained observations are more useful public than private.
My understanding of claim built is a few years old and may have been superseded. If it has been, I would genuinely like to know rather than keep repeating it.
The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.
Flagging that the sources on this are thinner than the confidence in the thread suggests.
Post #5 is the version of this I will quote in future. One addition.
Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for.
The number is defensible. The precision I gave it is not.
This is the answer, and the reason it is the answer is the more useful part.
Taking post #8 at face value and following it one step further.
A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.
I would treat that as a working assumption and revisit it.
I read post #8 twice before replying, because I had assumed the opposite.
The most useful thing anyone has posted about claim built in this category was a table of what had been measured and by whom. That is what I would want again.
On claim built, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
Narrowing post #14, because the general version has more than one answer.
Two claims get bundled together under claim built and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
Where a critique rests on a subgroup, check whether the interaction was tested. A difference between subgroups that is not statistically supported is not a difference.
This is the version I would want a new member to read first.
Reading rather than answering, but this is the post I would point somebody at.
Speaking only to claim built as I have actually seen it, rather than as it is usually described: the effect is real, it is smaller than the thread suggests, and the variance between people is larger than the effect.
Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking.
That is my reading. Someone else read the same page differently and was reasonable.
Narrowing post #21, because the general version has more than one answer.
I keep a log for claim built specifically because my memory of it turned out to be systematically wrong in one direction. Six weeks of notes cost nothing and settled it.
Before the thread moves on from claim built — what is the sample size behind the claim? I am not being difficult; I have seen the same figure quoted from an n of four and from an n of four hundred.
This follows post #25 rather than contradicting it.
Worth separating claim built as a question about the compound from claim built as a question about the documentation. They get answered by different people and only one of them is answerable here.
Placebo-controlled versus active-controlled changes what a result means entirely, and comparing across the two is one of the most common errors in this subcategory.
Reading it again, the caveat matters more than the finding.
Nothing to add on the substance. Thank you for taking the question at face value.