Evidence · Preprints

When a preprint is the best available evidence and we say so

Evidence › Preprints risk of bias effect size preprint

Solved

Solved by sourced_claims in post #8

Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised.

Jump to the accepted answer →

chromatogramTL4Analytical chemist25 Jul 2025#1

When a preprint is the best available evidence and we say so — that is the question, and I have not found it answered plainly anywhere I have looked.

A preprint question rather than a question about the finding in it.

The manuscript is on a preprint server, dated, with a version number I have written down. It has not been through review and the authors say so in the header. What I cannot work out is how much weight that leaves it carrying.

I would like the reasoning rather than a rule. "Wait for publication" is not usable when the publication may be two years away and the question is now.

0 likes 12mo

priorauth_notesTL2Regular26 Jul 2025#2

The opening post put the caveat in the right place and I want to underline it.

Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do.

Same conclusion as the reply above, reached differently, which is mildly reassuring.

3 likes 12mo

m.kjaerTL227 Jul 2025#3

The arithmetic in the opening post is right; the assumption feeding it is the part to check.

Retractions and corrections do not propagate well from preprint servers. A citation that was fine when made can quietly become wrong.

If it helps: the failure mode here is usually boring rather than dramatic.

15 likes 12mo

microgramsTL2Regular27 Jul 2025#4

m.kjaer, post #3: The arithmetic in the opening post is right; the assumption feeding it is the part to check. Retractions and corrections do not propagate well from preprint servers. A citation that was fine when made can quietly become wrong. If it helps: the failure mode here is usually boring rather than dramatic. Go to post

Where the preprint reasoning breaks down for me is the step from the group result to the individual case. That step is almost never argued for.

31 likes in reply to #3 12mo

j.erdoganTL228 Jul 2025#5

priorauth_notes, post #2: The opening post put the caveat in the right place and I want to underline it. Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do. Same conclusion as the reply above, reached differently, which is mildly reassuring. Go to post

Adding a data point of agreement rather than a data point.

1 like in reply to #2 12mo

np_gilmoreTL3Nurse practitioner28 Jul 2025#6

Post #2 put the caveat in the right place and I want to underline it.

Adding a null result on preprint. I looked, carefully, and found nothing, and null results deserve posting precisely because they never are.

6 likes 12mo

m.vukovicTL229 Jul 2025#7

Where a preprint's numbers are the basis of a claim being repeated here, someone should check whether the published version exists yet. That check is nearly always overdue.

22 likes 12mo

sourced_claimsTL3Regular Solution29 Jul 2025#8

Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised.

7 likes 12mo

m.onwukaTL230 Jul 2025#9

Taking post #8 at face value and following it one step further.

A preprint with the data and code attached is frequently more checkable than a published paper without them. Reviewed and reproducible are different properties.

That holds for the case as described. Change the assumptions and it may not.

3 likes 12mo

owen.bradyTL4 Moderator30 Jul 2025#10

A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less.

10 likes 12mo

y.adeyemiTL230 Jul 2025#11

I had read the opposite somewhere and cannot now find where, which tells me something.

7 likes 12mo

maintenance_modeTL3Regular31 Jul 2025#12

How to cite a preprint: author last name, first initial, et al. Title. Server name [preprint]. Posted date. Doi. Include the server name and note that it is a preprint.

Reporting the observation and leaving the explanation open deliberately.

1 like 12mo

g.radichTL231 Jul 2025#13

Always label it as a preprint in any discussion. A preprint is not a published trial and it is not settled knowledge. Use it for hypothesis generation and for understanding emerging evidence, not for drawing definitive conclusions.

0 likes 12mo

two_year_lineTL3Regular1 Aug 2025#14

Adding the measurement that post #13 says would settle it.

The honest position on most preprints in this field is that they are interesting and provisional, and that saying both is not fence-sitting.

The rule of thumb is fine; the edge cases are where it earns its keep.

23 likes in reply to #3 12mo

s.kuuselaTL21 Aug 2025#15

Collapsed as off-topic by two members at trust level 3 or above

Peer review catches some things reliably — missing methods, unsupported conclusions, statistical mistakes of the obvious kind — and misses others routinely. Knowing which is which makes preprint reading easier.

Adding the caveat now so it does not have to be extracted later.

batchlogTL3Regular1 Aug 2025#16

Practical experience of preprint, offered as one case with the conditions stated, not as a general finding. Conditions first, because they are what make it interpretable.

3 likes 12mo

d.ferreiraTL22 Aug 2025#17

m.onwuka, post #9: Taking post #8 at face value and following it one step further. A preprint with the data and code attached is frequently more checkable than a published paper without them. Reviewed and reproducible are different properties. That holds for the case as described. Change the assumptions and it may not. Go to post

Answering the question post #13 raises rather than the one it answers.

What I would check first on preprint is whether the thing being measured moved or whether the way of measuring it moved. Those look identical in a graph.

0 likes in reply to #9 12mo

bias_varianceTL4Biostatistician2 Aug 2025#18

np_gilmore, post #6: Post #2 put the caveat in the right place and I want to underline it. Adding a null result on preprint. I looked, carefully, and found nothing, and null results deserve posting precisely because they never are. Go to post

The arithmetic in post #17 is right; the assumption feeding it is the part to check.

Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion.

This is the version I would want a new member to read first.

31 likes in reply to #6 12mo

n.ramosTL23 Aug 2025#19

I keep a log for preprint specifically because my memory of it turned out to be systematically wrong in one direction. Six weeks of notes cost nothing and settled it.

16 likes 12mo

SHermansenTL2Member3 Aug 2025#20

Taking post #17 at face value and following it one step further.

What changes between preprint and publication: authors respond to reviewer comments, re-analyse data, correct errors found in peer review, and sometimes report different results from updated analyses. The published version is not the same as the preprint.

6 likes 12mo

ni.stanescuTL23 Aug 2025#21

Post #18 is the version of this I will quote in future. One addition.

A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad.

That is the shape of it. The detail is where I would expect to be corrected.

9 likes 12mo

ambient_reviewTL3Regular4 Aug 2025#22

sourced_claims, post #8: Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised. Go to post

Where I part company with post #20, and it is a narrow parting.

Where a preprint contradicts a published trial, the prior should favour the trial. Where it extends one, the preprint can be genuinely useful.

21 likes in reply to #8 12mo

t.tullochTL24 Aug 2025#23

s.kuusela, post #15: Peer review catches some things reliably — missing methods, unsupported conclusions, statistical mistakes of the obvious kind — and misses others routinely. Knowing which is which makes preprint reading easier. Adding the caveat now so it does not have to be extracted later. Go to post

Thank you for the correction. I would rather find out here than later.

0 likes in reply to #15 12mo

j.habermannTL3Regular4 Aug 2025#24

Source for the preprint figure, since it was asked for. It is in the discussion rather than the abstract, which is why the version circulating is stronger than the paper is.

Reading the surrounding paragraph is worth the two minutes. The authors are more careful than their summarisers.

2 likes 12mo

m.agyemanTL25 Aug 2025 · edited#25

Building on post #22 rather than restating it.

Author track record is a weak signal and it is a signal. A group that has published carefully for years is a different prior from an unknown one.

14 likes 12mo

n.torrenceTL3Regular5 Aug 2025#26

Building anything durable on a preprint means committing to rebuild it. That is a reasonable trade for a fast-moving question and it should be an explicit one.

28 likes 12mo

pe.onwukaTL25 Aug 2025#27

ni.stanescu, post #21: Post #18 is the version of this I will quote in future. One addition. A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad. That is the shape of it. The detail is where I would expect to be corrected. Go to post

Sample size and pre-registration are checkable in a preprint exactly as in a paper, and they carry the same weight.

I would rather post the uncertainty than round it away.

0 likes in reply to #21 12mo

endpoint_traceTL1Member6 Aug 2025#28

Server repositories: medRxiv for medicine, bioRxiv for biology. Preprints on these servers are time-stamped and have DOIs even if they are later published elsewhere. Citing the preprint with its DOI is appropriate while noting it is unpublished.

If anyone can point at the primary source I would be grateful.

5 likes 12mo

a.lindqvistTL26 Aug 2025#29

owen.brady, post #10: A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less. Go to post

If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update.

2 likes in reply to #10 12mo

j.vandermolenTL3Regular6 Aug 2025#30

batchlog, post #16: Practical experience of preprint, offered as one case with the conditions stated, not as a general finding. Conditions first, because they are what make it interpretable. Go to post

Thank you — that answers what I came here to find out.

10 likes in reply to #16 12mo

Sponsored Sigma-Aldrich — certified reference materialsReference standards, analytical reagents and certified materials for method calibration. If you are validating a method, this is where the standard comes from.Reference standards →