Evidence · Preprints

A preprint whose numbers changed substantially at publication

Evidence › Preprints preprint needs a source effect size

Wiki Closed

i.balogunTL221 Jul 2025#1

Community wiki post. Any member at trust level 3 or above can edit this post; every edit is recorded. Last edited by a.salcedo on 11 Apr 2026.

27 Aug 2025 — chromatogram: Clarified the distinction that was causing repeat questions below.
26 Aug 2025 — s.chowdhury: Added the worked example and a unit label to the table header.
2 Apr 2026 — r.venkatesan: Replaced an unsourced figure with the published one and cited it.
11 Apr 2026 — a.salcedo: Removed a claim that the cited source did not support.

Editors: chromatogram, s.chowdhury, r.venkatesan, a.salcedo

A preprint whose numbers changed substantially at publication — setting out what I have, and where I think it stops being reliable.

A preprint question rather than a question about the finding in it.

The manuscript is on a preprint server, dated, with a version number I have written down. It has not been through review and the authors say so in the header. What I cannot work out is how much weight that leaves it carrying.

I would like the reasoning rather than a rule. "Wait for publication" is not usable when the publication may be two years away and the question is now.

0 likes 12mo

c.niemelTL3Regular24 Jul 2025#2

Confirming the opening post from a second method, which matters more than confirming it from a second person.

Server repositories: medRxiv for medicine, bioRxiv for biology. Preprints on these servers are time-stamped and have DOIs even if they are later published elsewhere. Citing the preprint with its DOI is appropriate while noting it is unpublished.

Two sources, same conclusion, and I could not rule out that one copied the other.

28 likes 12mo

n.dziedzicTL226 Jul 2025#3

c.niemel, post #2: Confirming the opening post from a second method, which matters more than confirming it from a second person. Server repositories: medRxiv for medicine, bioRxiv for biology. Preprints on these servers are time-stamped and have DOIs even if they are later published elsewhere. Citing the preprint with its DOI is appropriate while noting… Go to post

Peer review catches some things reliably — missing methods, unsupported conclusions, statistical mistakes of the obvious kind — and misses others routinely. Knowing which is which makes preprint reading easier.

I would hold that lightly until someone with a larger sample weighs in.

14 likes in reply to #2 12mo

BramleyTL2Member27 Jul 2025#4

Reading rather than answering, but this is the post I would point somebody at.

5 likes in reply to #2 12mo

r.chukwuTL229 Jul 2025 · edited#5

I had written a reply contradicting post #3 and deleted it. Here is what survived.

A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less.

Happy to be corrected if someone holds better data than mine.

0 likes 12mo

cohort_driftTL3Regular30 Jul 2025#6

Confirming post #3 from a second method, which matters more than confirming it from a second person.

Say it is a preprint in the first line rather than the last. Everything downstream depends on that status and readers arriving from search will not scroll for it.

0 likes 12mo

s.okonkwoTL21 Aug 2025#7

Bramley, post #4: Reading rather than answering, but this is the post I would point somebody at. Go to post

Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do.

20 likes in reply to #4 12mo

isotonic_driftTL1Member2 Aug 2025#8

n.dziedzic, post #3: Peer review catches some things reliably — missing methods, unsupported conclusions, statistical mistakes of the obvious kind — and misses others routinely. Knowing which is which makes preprint reading easier. I would hold that lightly until someone with a larger sample weighs in. Go to post

Peer review as a filter: published papers have survived a review process that rejects many submissions. Preprints have not. Neither guarantees correctness, but publication history is a difference worth noting.

8 likes in reply to #3 12mo

m.ndiayeTL23 Aug 2025#9

Where I part company with post #7, and it is a narrow parting.

How to cite a preprint: author last name, first initial, et al. Title. Server name [preprint]. Posted date. Doi. Include the server name and note that it is a preprint.

29 likes 12mo

bench_peakTL3Regular4 Aug 2025#10

Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion.

15 likes 12mo

t.abubakarTL26 Aug 2025#11

isotonic_drift, post #8: Peer review as a filter: published papers have survived a review process that rejects many submissions. Preprints have not. Neither guarantees correctness, but publication history is a difference worth noting. Go to post

Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised.

That is the practical version. The rigorous version is longer and says the same thing.

18 likes in reply to #8 12mo

v.krastevTL27 Aug 2025#12

Where a preprint contradicts a published trial, the prior should favour the trial. Where it extends one, the preprint can be genuinely useful.

The confident version of this sentence would be wrong, so here is the hedged one.

0 likes 12mo

k.haddadTL28 Aug 2025#13

Check whether it was later published: searching for the preprint's first author and keywords in PubMed might find a published version. The published version is the stronger citation if it exists.

0 likes 12mo

PSkarbekTL3Regular9 Aug 2025#14

s.okonkwo, post #7: Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do. Go to post

Worth separating two things that post #10 runs together.

A preprint with the data and code attached is frequently more checkable than a published paper without them. Reviewed and reproducible are different properties.

4 likes in reply to #7 12mo

b.nilsenTL210 Aug 2025 · edited#15

Confirming post #12 from a second method, which matters more than confirming it from a second person.

Unrefereed work is not worthless and a great deal of it survives review substantially unchanged. That is the accurate framing and it is not a licence to treat it as final.

Adding it in case it saves somebody the afternoon it cost me.

25 likes 12mo

m.rasmussenTL211 Aug 2025#16

Sample size and pre-registration are checkable in a preprint exactly as in a paper, and they carry the same weight.

I would not lead a decision with this, but I would not ignore it either.

0 likes 12mo

m.achebeTL212 Aug 2025#17

Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim.

A guess, clearly labelled as one.

1 like 12mo

n.petrovTL213 Aug 2025#18

If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update.

7 likes in reply to #8 11mo

t.verhoevenTL214 Aug 2025 · edited#19

Version numbers on a preprint server matter. Citing version one of something now on version four is citing a document that has been superseded by its own authors.

8 likes 11mo

taper_tableTL3Regular15 Aug 2025#20

Everything in post #18 holds. The case it does not cover is the one I have.

Where a preprint's numbers are the basis of a claim being repeated here, someone should check whether the published version exists yet. That check is nearly always overdue.

19 likes 11mo

t.vargaTL217 Aug 2025#21

I had written a reply contradicting post #19 and deleted it. Here is what survived.

Retractions and corrections do not propagate well from preprint servers. A citation that was fine when made can quietly become wrong.

Anyone who has looked at this more carefully, please correct the record.

8 likes 11mo

methods_draftTL2Member18 Aug 2025#22

s.okonkwo, post #7: Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do. Go to post

Confirming post #19 from a second method, which matters more than confirming it from a second person.

A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad.

That has held every time I have looked, which is not the same as always.

2 likes in reply to #7 11mo

k.ogunleyeTL219 Aug 2025#23

Noted, and thank you for writing it out rather than summarising it.

0 likes in reply to #2 11mo

SHermansenTL2Member20 Aug 2025#24

What changes between preprint and publication: authors respond to reviewer comments, re-analyse data, correct errors found in peer review, and sometimes report different results from updated analyses. The published version is not the same as the preprint.

I would be interested in a counterexample if anyone has one.

20 likes 11mo

an.zamoraTL220 Aug 2025#25

Unrefereed work is not worthless. A great deal of it survives review substantially unchanged. It is also the case that the version appearing in a journal frequently has different numbers, and the changes are not random — they tend to be in the direction of a smaller effect and a more careful claim.

I am not the right person to answer the follow-up to this.

5 likes 11mo

RodriguesTL3Regular21 Aug 2025 · edited#26

cohort_drift, post #6: Confirming post #3 from a second method, which matters more than confirming it from a second person. Say it is a preprint in the first line rather than the last. Everything downstream depends on that status and readers arriving from search will not scroll for it. Go to post

Post #24 is the version of this I will quote in future. One addition.

Always label it as a preprint in any discussion. A preprint is not a published trial and it is not settled knowledge. Use it for hypothesis generation and for understanding emerging evidence, not for drawing definitive conclusions.

That is the version I use. It may not be the version that is correct.

0 likes in reply to #6 11mo

e.bakkenTL222 Aug 2025#27

Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do.

28 likes 11mo

FConsidineTL1Member23 Aug 2025#28

14 likes 11mo

r.novakTL224 Aug 2025#29

How to cite a preprint: author last name, first initial, et al. Title. Server name [preprint]. Posted date. Doi. Include the server name and note that it is a preprint.

2 likes 11mo

o.abrahamsenTL3Regular25 Aug 2025#30

Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion.

0 likes 11mo

Sponsored PeptideMeter — peptide analytics and supplier verificationPurity analysis and structured supplier verification, published as dated records rather than scores. Referenced throughout our sourcing documentation.View analytics →