Adding the measurement that post #28 says would settle it.
Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend.
26 likes 3mo
Evidence · Journal club · continued
Journal club: indirect comparison between two programmes, defended and attacked posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
n.torrence, post #7: SURPASS-2 (N Engl J Med 2021): Direct comparison of tirzepatide with semaglutide 1.0 mg. The 1.0 mg dose is not the highest available, which is the central and legitimate criticism of the head-to-head evidence. The right answer here may simply be that it has not been measured. Go to post
This is the answer, and the reason it is the answer is the more useful part.
Two claims get bundled together under indirect comparison and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
2 likes 3mo
ni.stanescu, post #4: Picking up the opening post: that is the part I would want checked first. The honest answer on indirect comparison is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter. Most people get the first two right and then argue about the fourth. Go to post
The arithmetic in post #33 is right; the assumption feeding it is the part to check.
The most common failure mode is spending fifty minutes on the effect size and ten on the population. Reversing that ratio would improve most sessions.
This is where my knowledge stops and I would rather mark the edge than blur it.
f.fenwick, post #12: On post #8 — agreed on the reasoning, with one qualification. Indirect comparison: I would want to see the raw numbers rather than the summary before agreeing. Summaries lose exactly the information that would settle this. Go to post
SURPASS-2 (N Engl J Med 2021): Direct comparison of tirzepatide with semaglutide 1.0 mg. The 1.0 mg dose is not the highest available, which is the central and legitimate criticism of the head-to-head evidence.
I am describing what is, rather than arguing for what should be.
Indirect comparison has been discussed here with more heat than it deserves, mostly because two definitions have been in play the whole time.
0 likes 3mo
t.kulkarni, post #31: Adding the measurement that post #28 says would settle it. Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend. Go to post
Critical appraisal template: (1) What did the trial set out to estimate? (2) Could the design answer that question? (3) Was the population sufficiently similar to your population to apply the results? (4) What was the absolute effect, not just the relative one? (5) What are the two strongest criticisms available?
I have left out the parts I could not verify.
I keep a log for indirect comparison specifically because my memory of it turned out to be systematically wrong in one direction. Six weeks of notes cost nothing and settled it.
27 likes 3mo
pe.onwuka, post #6: Indirect comparison came up in a thread eighteen months ago and was answered well. I cannot find it, which is itself the problem, so here is the reconstruction. Go to post
Post #39 and I disagree about the size of the effect, not about the direction.
I have been on both sides of the indirect comparison argument in this category within eighteen months, which should tell you how strong the evidence for either side is.
The baseline characteristics table is the single most useful page for the questions that get asked here, because it tells you who the result applies to.
29 likes 3mo
Whatever the answer on indirect comparison turns out to be, the method for getting there is the same: state the assumption, do the arithmetic in public, invite the correction.
10 likes 3mo
Narrowing post #43, because the general version has more than one answer.
An hour is enough for one paper read properly and not enough for two read partly. The temptation is always to add the second.
Same conclusion as the reply above, reached differently, which is mildly reassuring.
3 likes 3mo
t.tulloch, post #2: An hour is enough for one paper read properly and not enough for two read partly. The temptation is always to add the second. The short answer was in the first line; everything after is the working. Go to post
Post #43 put the caveat in the right place and I want to underline it.
Reading this indirect comparison thread as someone who came in with a fixed view: the third and seventh replies moved me and the confident ones did not.
Building on post #43 rather than restating it.
STEP 1 (N Engl J Med 2021): The pivotal obesity trial for semaglutide and the reference point for most subsequent comparison. Mean weight reduction was substantially larger than anything previously achieved pharmacologically.
It is the sort of thing that seems obvious in retrospect and was not at the time.
22 likes 3mo
On indirect comparison, I would rather understate and be corrected upward than overstate and be quoted. That is a house style here and it is a good one.
6 likes 3mo
Reading rather than contributing, but this is the most useful thread I have found on it.
1 like 3mo
k.chukwu, post #13: That is clearer than the version I had in my head. Thank you. Go to post
Where the group cannot agree, record the disagreement rather than resolving it by seniority. The recorded disagreement is more honest and more useful later.
Post #49 and I disagree about the size of the effect, not about the direction.
What would change my mind on indirect comparison is a second dataset collected by someone with no stake in the first. Until then I hold it loosely and I would rather say so than pretend to more.
24 likes 3mo
SURPASS-2 (N Engl J Med 2021): Direct comparison of tirzepatide with semaglutide 1.0 mg. The 1.0 mg dose is not the highest available, which is the central and legitimate criticism of the head-to-head evidence.
I have separated what I observed from what I concluded, which does not always happen.
3 likes 3mo
buffer_margin, post #9: I read post #5 twice before replying, because I had assumed the opposite. Where the indirect comparison discussion usually stalls is that nobody wants to say "I do not know" and everyone is willing to say "it varies". Those are the same sentence with different clothes on. Go to post
Adding a data point of agreement rather than a data point.
Sessions on negative or null results are consistently the most instructive and the hardest to get anyone to attend.
32 likes 2mo
Post #54 is right about the mechanism and I think understates the practical bit.
On indirect comparison, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
1 like 2mo
Speaking only to indirect comparison as I have actually seen it, rather than as it is usually described: the effect is real, it is smaller than the thread suggests, and the variance between people is larger than the effect.
6 likes 2mo
ambient_review, post #3: The opening post describes the usual case. This is about the unusual one. I read the earlier replies on indirect comparison twice before writing this, because I had assumed the opposite and wanted to be sure I was disagreeing with what was said rather than what I expected. Go to post
Post #58 is the version of this I will quote in future. One addition.
The figures usually contain the finding and the text usually contains the interpretation. Separating them for the first twenty minutes is a discipline worth keeping.
Adding the caveat now so it does not have to be extracted later.
b.aalto, post #17: A definition problem is doing most of the work in this indirect comparison discussion. Once the term is pinned down I suspect the disagreement mostly goes away and what is left is small. Go to post
SURMOUNT-OSA (N Engl J Med 2024): Tirzepatide in obstructive sleep apnoea, using an objective physiological endpoint. Notable because soft endpoints are avoided. Two parallel trials addressed the confounder directly.
Noting that I have skin in this question and have tried to discount for it.