Distinguishing three things in the measurement error discussion that keep getting used interchangeably: the observation, the proposed mechanism, and the recommendation that gets attached to both.
Posting this under the heading it deserves: Measurement error in a self-reported exposure Everything below is what sits behind that.
The question about measurement error that I actually want answered is the second one below. The first is context and I have kept it short.
Both are stated with units, and I have said what I already checked so that nobody repeats it.
The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.
Two sources, same conclusion, and I could not rule out that one copied the other.
Thank you for the correction. I would rather find out here than later.
Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.
This is the sort of thing the wiki should carry and currently does not.
Distinguishing three things in the measurement error discussion that keep getting used interchangeably: the observation, the proposed mechanism, and the recommendation that gets attached to both.
The arithmetic in post #4 is right; the assumption feeding it is the part to check.
Trying to state the measurement error position in a way that someone who disagrees would recognise as fair, because I do not think the version in this thread passes that test.
Answering the question post #6 raises rather than the one it answers.
Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking.
This is the sort of thing that ought to be settled and apparently is not.
An honest declaration on measurement error: I have a prior here and it is strong enough that you should weight what I say downward. Stating it rather than hiding it.
Post #6 put the caveat in the right place and I want to underline it.
Measurement error is worth one more sentence than it usually gets, and the sentence is the one about how the number was arrived at.
Hold a trial to the standard something could actually have met. A criticism that no achievable design could have answered is a criticism of the field rather than of the paper.
Filing this under things that are true until someone shows me otherwise.
On post #9 — agreed on the reasoning, with one qualification.
Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not.
The literature is thinner on this than the confidence in the thread implies.
Picking up post #13: that is the part I would want checked first.
What I want from this measurement error thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.
Post #13 describes the usual case. This is about the unusual one.
Attrition is the failure mode most likely to invalidate a result and the least likely to be discussed. Differential attrition between arms is the specific thing to look for.
On balance I think that is right, and I would not bet much on it.
One caution on measurement error: everything above assumes the underlying documentation is what it claims to be. That assumption is doing real work and is rarely stated.
Post #17 put the caveat in the right place and I want to underline it.
Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters.
Post #18 is the version of this I will quote in future. One addition.
A request rather than an answer: could whoever has the primary source for measurement error post it? I have seen the claim three times this month and each version had lost a qualifier.
Where I part company with post #20, and it is a narrow parting.
A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.
The interesting part of this is the exception, and I do not understand the exception.
Surrogate endpoints are not automatically bad and their validity is compound-specific and population-specific. The question is whether this surrogate has been validated for this use.
It is one reading of the data and not the only reasonable one.
Building on post #22 rather than restating it.
Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.
If that reads as pedantic, it is, and it has saved me twice.
What would change my mind on measurement error is a second dataset collected by someone with no stake in the first. Until then I hold it loosely and I would rather say so than pretend to more.
The useful distinction on measurement error is between what was measured and what was inferred from it. Both end up in the same sentence and only one of them has error bars.
Helpful, and easy to find again, which is half of what a good reply is.
Adding the boring version of measurement error, because the interesting version keeps getting posted and the boring one is usually right.
Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the time the second one does.