This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Coming back to post #27, because the follow-up matters more than the original answer.
Well-designed study is well covered in the tag pages, and the older discussions are better than the recent ones because they were argued out properly. Worth twenty minutes before adding to this one.
Post #31 is right about the mechanism and I think understates the practical bit.
Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not.
It is a small point and it changes the answer, which is an awkward combination.
Grateful for the specificity. Vague answers to this question are what sent me looking.
My experience of well-designed study contradicts the reply above. I am posting it as a data point rather than as a refutation, because one person's experience is exactly that.
Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for.
Confirming post #35 from a second method, which matters more than confirming it from a second person.
Answering the well-designed study question as asked, then the question I think is meant. As asked: yes, with the qualification below. As meant: it depends on how the first measurement was taken.
Post #35 describes the usual case. This is about the unusual one.
On well-designed study: the maintained page in the documentation commons covers the general case with citations and a review date, which is more reliable than any reply here including this one.
A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.
It is worth checking rather than assuming, which costs nothing.
The question underneath well-designed study is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it.
Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help.
Sensible. I would want the same detail before I acted on it either.
I had written a reply contradicting post #38 and deleted it. Here is what survived.
Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.
The useful distinction on well-designed study is between what was measured and what was inferred from it. Both end up in the same sentence and only one of them has error bars.
Two claims get bundled together under well-designed study and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for.
Noting that I have skin in this question and have tried to discount for it.
The arithmetic in post #44 is right; the assumption feeding it is the part to check.
Reporting bias within a paper — a result in a figure that is absent from the abstract — is checkable and is worth checking when the claim matters.
On well-designed study, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
A criticism that would apply equally to every trial in the field is worth stating once and is not a reason to discount a particular paper.
This is the version I would want a new member to read first.
Well-designed study is a good example of a question where the honest answer is boring and the interesting answers are unsupported. I would go with boring.
Post #49 and I disagree about the size of the effect, not about the direction.
Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking.
What I want from this well-designed study thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.
Posting my well-designed study numbers with the method attached so they can be discounted properly. Uncontrolled, unblinded, and collected by someone who wanted a particular answer.
Post #53 put the caveat in the right place and I want to underline it.
Two questions I would want answered before drawing anything from the well-designed study data above: how were the cases selected, and what happened to the ones that dropped out.
Building on post #57 rather than restating it.
Per-protocol and intention-to-treat analyses answer different questions and neither is the honest one by default. Reporting both is the practice worth insisting on.
Appreciated. The plain phrasing does more work here than a longer post would.