Subgroup analyses: pre-specified versus discovered — a second dataset posts 61–80

Where the Subgroup analyses reasoning breaks down for me is the step from the group result to the individual case. That step is almost never argued for.

The arithmetic in post #61 is right; the assumption feeding it is the part to check.

Adding a null result on Subgroup analyses. I looked, carefully, and found nothing, and null results deserve posting precisely because they never are.

I think the Subgroup analyses question is answerable and has not been answered, which is a more optimistic position than most of this thread.

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

Noted, and I have changed what I was going to do on the strength of it.

Adding the measurement that post #64 says would settle it.

Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.

I am confident about the direction and much less about the magnitude.

Post #67 describes the usual case. This is about the unusual one.

Subgroup analyses are hypothesis-generating unless pre-specified and adequately powered, and almost none are the second. The interaction test matters more than the subgroup point estimate.

If that is already documented somewhere, ignore me and link it.

Post #68 is right about the mechanism and I think understates the practical bit.

Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.

I would rather say I do not know than round it up to an answer.

Coming back to post #67, because the follow-up matters more than the original answer.

An update on my earlier Subgroup analyses post: the pattern held for another six weeks and then stopped, which I did not predict and cannot explain.

Filing a mild objection to the consensus on Subgroup analyses. Mild because I might be wrong; an objection because nobody has addressed the case that does not fit.

Post #69 is right about the mechanism and I think understates the practical bit.

A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.

Nothing to add, except that this is the answer I would give if asked.

Subgroup analyses was covered in the wiki last year and the page has a review date on it, which is a better starting point than my memory of a thread.