The Peptide CommonsEst. May 2024
Independent. We sell nothing and are affiliated with no manufacturer or pharmacy. Every moderation action is logged in public
Evidence · Trials

Trial registration and comparing the protocol with the paper

AI
a.iyerTL231 Mar 2025#1

Trial registration and comparing the protocol with the paper — setting out what I have, and where I think it stops being reliable.

I have spent a fortnight trying to pin trial registration down and I want to set out where I have got to, because I suspect the honest answer is duller than the thread this will produce.

What I have: a consistent observation across a small number of cases, collected the same way each time. What I do not have: any controlled comparison, or any reason to think my cases are representative.

The specific question is whether the pattern survives once the obvious confounder is removed. I cannot remove it with what I have.

0 likes 16mo
RD
r.danquahTL210 Apr 2025#2

I would be cautious about generalising from the trial registration example above. It is a good example. It is one example.

29 likes 16mo
RV
r.villalobosTL217 Apr 2025#3

Trial duration determines what can be observed. A weight-change trajectory at 40 weeks and at 72 weeks are different observations and both get quoted as the result.

The literature is thinner on this than the confidence in the thread implies.

14 likes 15mo
OV
o.vogelTL223 Apr 2025#4

Useful. I had the fact and not the reason, which turns out to be the important half.

5 likes 15mo
AV
a.vukovicTL229 Apr 2025#5
r.danquah, post #2: I would be cautious about generalising from the trial registration example above. It is a good example. It is one example. Go to post

Post #3 and I disagree about the size of the effect, not about the direction.

Nothing in a trial report is medical advice about an individual, and the gap between a population estimate and a person is exactly where clinical judgement lives.

0 likes in reply to #2 15mo
BN
bench_notesTL4 Moderator5 May 2025#6

A trial that answers a slightly different question from the one you have is the normal situation rather than a failure of the trial. The skill is describing the gap precisely.

21 likes 15mo
KP
k.perrinTL210 May 2025#7
KV
k.vanheckeTL215 May 2025#8

Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot.

It took me longer than it should have to see that.

2 likes 14mo
RS
r.scholtenTL2Member20 May 2025#9

Grateful for the specificity. Vague answers to this question are what sent me looking.

2 likes 14mo
GV
g.verhoevenTL225 May 2025#10
NP
n.petrovTL230 May 2025#11

The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions.

11 likes 14mo
BN
b.nilsenTL24 Jun 2025#12

The first question about any trial is what it set out to estimate, not what it found. Once the estimand is on the table the rest of the discussion is tractable.

24 likes 14mo
VK
v.krastevTL28 Jun 2025#13

Entry criteria, run-in periods and the self-selection of people willing to enter a multi-year trial all narrow the population. That is how internal validity is bought and it constrains generalisation.

Flagging that the sources on this are thinner than the confidence in the thread suggests.

0 likes 14mo
MA
m.achebeTL213 Jun 2025#14
k.perrin, post #7: The figure that circulates in coverage is almost always whichever estimand gives the larger effect. That is not fraud; it is selection, and it is why the paper matters more than the summary. On balance I think that is right, and I would not bet much on it. Go to post

I had written a reply contradicting post #11 and deleted it. Here is what survived.

Funding and trial conduct should be stated and are a weak predictor of anything on their own. Design quality is the stronger signal and it is checkable.

Genuinely open to being wrong about this one.

1 like in reply to #7 13mo
P
PSkarbekTL317 Jun 2025#15
AS
a.silvaTL222 Jun 2025 · edited#16

The honest answer on trial registration is that it depends, and the useful part is the list of what it depends on. Four items, in rough order of how much they matter.

Most people get the first two right and then argue about the fourth.

17 likes 13mo
BM
buffer_marginTL3Regular26 Jun 2025#17

Seconded. It reads as careful rather than confident, which is the right register.

33 likes 13mo
KH
k.haddadTL230 Jun 2025#18
n.petrov, post #11: The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions. Go to post

Coming back to post #14, because the follow-up matters more than the original answer.

An open-label trial is not worthless and its subjective endpoints deserve more scepticism than its objective ones. That is a graded judgement rather than a verdict.

I would rather be precise about what I do not know than vague about what I do.

0 likes in reply to #11 13mo
EC
excursion_checkTL3Regular4 Jul 2025#19
buffer_margin, post #17: Seconded. It reads as careful rather than confident, which is the right register. Go to post

Narrowing post #16, because the general version has more than one answer.

The bit of trial registration that nobody enjoys is that the answer changes depending on what you are trying to decide with it. Say what the decision is and the thread will converge.

4 likes in reply to #17 13mo
AH
a.hartmannTL28 Jul 2025#20

Everything in post #18 holds. The case it does not cover is the one I have.

Registration before enrolment, with the primary endpoint declared, is what makes outcome switching detectable. Checking the registry against the paper takes five minutes and is worth doing.

That is all I can say without guessing.

12 likes 13mo
CE
crossover_entryTL3Regular12 Jul 2025#21

Post #19 and I disagree about the size of the effect, not about the direction.

Adding a small correction to the trial registration summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.

1 like 13mo
LV
l.vermeulenTL217 Jul 2025#22
v.krastev, post #13: Entry criteria, run-in periods and the self-selection of people willing to enter a multi-year trial all narrow the population. That is how internal validity is bought and it constrains generalisation. Flagging that the sources on this are thinner than the confidence in the thread suggests. Go to post

Reading rather than answering, but this is the post I would point somebody at.

0 likes in reply to #13 12mo
N
NardoneTL2Member20 Jul 2025#23

Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at the cost of external validity.

15 likes 12mo
TV
t.vargaTL224 Jul 2025#24
OA
o.abrahamsenTL3Regular28 Jul 2025#25

Post #23 put the caveat in the right place and I want to underline it.

Trial registration: I have looked for the primary source twice and failed twice. Either it does not exist or it is somewhere I do not know to look, and I would like to know which.

3 likes 12mo
MN
m.nwosuTL21 Aug 2025#26
k.haddad, post #18: Coming back to post #14, because the follow-up matters more than the original answer. An open-label trial is not worthless and its subjective endpoints deserve more scepticism than its objective ones. That is a graded judgement rather than a verdict. I would rather be precise about what I do not know than vague about what I do. Go to post

Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.

I would want a second opinion before relying on that.

0 likes in reply to #18 12mo
EK
e.kjeldsenTL2Member5 Aug 2025#27
v.krastev, post #13: Entry criteria, run-in periods and the self-selection of people willing to enter a multi-year trial all narrow the population. That is how internal validity is bought and it constrains generalisation. Flagging that the sources on this are thinner than the confidence in the thread suggests. Go to post

Effect sizes in a trial population reflect adherence achieved under trial conditions, which is generally better than adherence outside them.

22 likes in reply to #13 12mo
PL
p.lindqvistTL29 Aug 2025#28

The arithmetic in post #27 is right; the assumption feeding it is the part to check.

Funding and trial conduct should be stated and are a weak predictor of anything on their own. Design quality is the stronger signal and it is checkable.

Worth checking against a second source before it gets quoted onward.

10 likes 12mo
MM
methods_marginTL3Regular13 Aug 2025#29

Multiplicity and multiple comparisons: if a trial tests many hypotheses, the chance of a false positive on at least one by random chance increases. This is why pre-specification of the primary endpoint matters and why secondary endpoints are weaker evidence.

Somebody will have a better source than mine, and I hope they post it.

5 likes 11mo
EB
e.bakkenTL216 Aug 2025#30

The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions.

0 likes 11mo