A structured critique template this community uses — a second dataset posts 31–52
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.
Where a critique rests on a subgroup, check whether the interaction was tested. A difference between subgroups that is not statistically supported is not a difference.
This has been discussed before and I could not find the thread, so, again.
Post #29 put the caveat in the right place and I want to underline it.
A note on how structured critique template gets discussed rather than on structured critique template itself: the confident posts get the replies and the careful ones get ignored, and the careful ones have been right more often.
Structured critique template came up in a thread eighteen months ago and was answered well. I cannot find it, which is itself the problem, so here is the reconstruction.
Agreed, and I will stop repeating the version of this I had been repeating.
Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters.
Collapsed as off-topic by two members at trust level 3 or above
Post #33 and I disagree about the size of the effect, not about the direction.
Where a critique changes your position, saying so in the thread is worth more than being right, and it is the thing this subcategory does better than most places.
Posting it because the silence on this was starting to look like agreement.
Post #38 describes the usual case. This is about the unusual one.
Offering a way to settle structured critique template rather than another opinion about it. Two measurements, taken the same way, a fortnight apart. If the difference is within the noise, the question was not answerable at this precision.
Two things can be true about structured critique template at once: the mechanism is plausible and the evidence for the size of the effect is thin. Most of the argument here is people defending the first against attacks on the second.
Post #42 is the version of this I will quote in future. One addition.
Adding the boring version of structured critique template, because the interesting version keeps getting posted and the boring one is usually right.
Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the time the second one does.
The confident answers on structured critique template and the well-sourced answers are not the same answers, which is the most useful thing I have learned reading this category.
Adding the measurement that post #42 says would settle it.
A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.
I have kept the units in throughout, for the obvious reason.
Post #44 describes the usual case. This is about the unusual one.
Worth separating structured critique template as a question about the compound from structured critique template as a question about the documentation. They get answered by different people and only one of them is answerable here.
Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking.
Genuinely open to being wrong about this one.
The question underneath structured critique template is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it.
Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help.
Picking up post #46: that is the part I would want checked first.
I would put moderate confidence on the mainstream reading of structured critique template and no more. That is not scepticism for its own sake; it is where the sourcing actually stops.
The practical version of structured critique template is three sentences long. The rigorous version is three pages and reaches the same conclusion with the conditions attached.
This topic was referenced in
- Criticising the method without criticising the authors — one year onEvidence › Study critique · 20 replies
Suggested topics
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Reverse causation in a cohort study of weight and outcome — a second dataset
Reverse causation in a cohort study of weight and outcome — a second dataset Writing it up because I had to work it out twice and would rather nobody else did. Posting a small dataset on Reverse causation. It…
|
+112 | 131 | 1.5k | 10mo |
|
Measurement error in a self-reported exposure
Posting this under the heading it deserves: Measurement error in a self-reported exposure Everything below is what sits behind that. The question about measurement error that I actually want answered is the…
|
+117 | 129 | 45k | 5mo |
|
Immortal time bias in a claims-database study — one year on
Posting this under the heading it deserves: Immortal time bias in a claims-database study — one year on Everything below is what sits behind that. I would like to disagree carefully with the settled view on…
|
+41 | 46 | 3.7k | 22h |
|
Immortal time bias in a claims-database study
Immortal time bias in a claims-database study — setting out what I have, and where I think it stops being reliable. Asking about immortal time bias directly, because I have read four threads on it and each…
|
+29 | 34 | 28k | 11mo |
|
Critiquing an observational claim about a class effect
Posting this under the heading it deserves: Critiquing an observational claim about a class effect Everything below is what sits behind that. I was wrong about critiquing an observational claim in a thread…
|
2 | 6.9k | 14mo |
Related topics — sharing the tags observational data, effect size, confounding
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
TB-500 and thymosin beta-4: the fragment versus the protein
TB-500 and thymosin beta-4: the fragment versus the protein Writing it up because I had to work it out twice and would rather nobody else did. TB-500 and thymosin beta-4: what I expected, what I found, and…
|
+107 | 118 | 22k | 11mo |
|
Contributing data without breaching anyone's privacy
Contributing data without breaching anyone's privacy Writing it up because I had to work it out twice and would rather nobody else did. Asking about contributing data without breaching directly, because I…
|
+65 | 71 | 839 | 5mo |
|
Revisiting: Sample size in a voluntary survey: the selection problem
Revisiting: Sample size in a voluntary survey: the selection problem — setting out what I have, and where I think it stops being reliable. I have spent a fortnight trying to pin Sample size down and I want to…
|
2 | 56k | 9mo | |
|
How this community labels and discusses unrefereed work
How this community labels and discusses unrefereed work — that is the question, and I have not found it answered plainly anywhere I have looked. Reading an unrefereed manuscript properly, and asking whether I…
|
+112 | 119 | 1.3k | 2y |
|
Journal club: STEP-HFpEF and symptom endpoints
Journal club: STEP-HFpEF and symptom endpoints — setting out what I have, and where I think it stops being reliable. The question about step-hfpef and symptom endpoints that I actually want answered is the…
|
+18 | 22 | 2.5k | 11mo |