Subgroup analyses: pre-specified versus discovered — a second dataset posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Posting my Subgroup analyses numbers with the method attached so they can be discounted properly. Uncontrolled, unblinded, and collected by someone who wanted a particular answer.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
I have kept the units in throughout, for the obvious reason.
The practical version of Subgroup analyses is three sentences long. The rigorous version is three pages and reaches the same conclusion with the conditions attached.
Where I part company with post #37, and it is a narrow parting.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
Worth one more sentence than it usually gets.
Collapsed as off-topic by two members at trust level 3 or above
Intent-to-treat versus per-protocol: ITT includes everyone assigned regardless of whether they took the drug. Per-protocol includes only those who completed it as intended. The two can give substantially different results.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
I would rather be precise about what I do not know than vague about what I do.
Taking post #42 at face value and following it one step further.
Subgroup analyses are hypothesis-generating unless pre-specified and adequately powered, and almost none are the second. The interaction test matters more than the subgroup point estimate.
Post #44 and I disagree about the size of the effect, not about the direction.
Offering a way to settle Subgroup analyses rather than another opinion about it. Two measurements, taken the same way, a fortnight apart. If the difference is within the noise, the question was not answerable at this precision.
Second-hand on Subgroup analyses, so weight it accordingly — someone whose method I trust told me this and I have not verified it myself.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
The part I am sure of is shorter than the part I have written.
An observation about Subgroup analyses that I cannot explain and am posting anyway, on the principle that unexplained observations are more useful public than private.
Worth separating two things that post #48 runs together.
Practical experience of Subgroup analyses, offered as one case with the conditions stated, not as a general finding. Conditions first, because they are what make it interpretable.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
Second-hand, so weight it accordingly.
Confirming post #51 from a second method, which matters more than confirming it from a second person.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
Post #51 describes the usual case. This is about the unusual one.
Subgroup analyses is well covered in the tag pages, and the older discussions are better than the recent ones because they were argued out properly. Worth twenty minutes before adding to this one.
Where I part company with post #51, and it is a narrow parting.
What I would want before treating Subgroup analyses as settled: the method, the sample, and whether anyone tried to find the opposite result. Two of the three are usually missing.
Post #55 is the version of this I will quote in future. One addition.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
Nobody has said the unglamorous part of Subgroup analyses yet, so: most of the variation is explained by things that are boring to write about and easy to check.
Post #55 put the caveat in the right place and I want to underline it.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.