Answering the question post #59 raises rather than the one it answers.
Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion.
9 likes 10mo
Evidence · Preprints · continued
A preprint whose numbers changed substantially at publication posts 61–90
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
FConsidine, post #28: Peer review as a filter: published papers have survived a review process that rejects many submissions. Preprints have not. Neither guarantees correctness, but publication history is a difference worth noting. Go to post
The arithmetic in post #59 is right; the assumption feeding it is the part to check.
Unrefereed work is not worthless and a great deal of it survives review substantially unchanged. That is the accurate framing and it is not a licence to treat it as final.
If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update.
One of those cases where knowing the mechanism does not help the decision.
0 likes 10mo
Peer review as a filter: published papers have survived a review process that rejects many submissions. Preprints have not. Neither guarantees correctness, but publication history is a difference worth noting.
The variance between people here is larger than the effect being discussed.
21 likes 10mo
Post #63 describes the usual case. This is about the unusual one.
Building anything durable on a preprint means committing to rebuild it. That is a reasonable trade for a fast-moving question and it should be an explicit one.
14 likes 10mo
dr.villanueva, post #36: Where I part company with post #32, and it is a narrow parting. Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised. Reading it back, the second half matters more… Go to post
Server repositories: medRxiv for medicine, bioRxiv for biology. Preprints on these servers are time-stamped and have DOIs even if they are later published elsewhere. Citing the preprint with its DOI is appropriate while noting it is unpublished.
Adding the caveat now so it does not have to be extracted later.
methods_draft, post #22: Confirming post #19 from a second method, which matters more than confirming it from a second person. A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad. That has held every time I have looked, which is not the same as always. Go to post
Peer review catches some things reliably — missing methods, unsupported conclusions, statistical mistakes of the obvious kind — and misses others routinely. Knowing which is which makes preprint reading easier.
That matches what I was told, which is not the same as knowing it.
k.pereira, post #60: Where I part company with post #59, and it is a narrow parting. A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less. Go to post
Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised.
This is the version I would want a new member to read first.
Check whether it was later published: searching for the preprint's first author and keywords in PubMed might find a published version. The published version is the stronger citation if it exists.
16 likes 10mo
Retractions and corrections do not propagate well from preprint servers. A citation that was fine when made can quietly become wrong.
Genuinely open to being wrong about this one.
31 likes 10mo
n.petrov, post #18: If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update. Go to post
The arithmetic in post #70 is right; the assumption feeding it is the part to check.
How to cite a preprint: author last name, first initial, et al. Title. Server name [preprint]. Posted date. Doi. Include the server name and note that it is a preprint.
The strength of my opinion here exceeds the strength of my evidence.
e.iyer, post #35: If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update. Go to post
Answering the question post #72 raises rather than the one it answers.
Unrefereed work is not worthless. A great deal of it survives review substantially unchanged. It is also the case that the version appearing in a journal frequently has different numbers, and the changes are not random — they tend to be in the direction of a smaller effect and a more careful claim.
That is a cleaner way of putting what I was circling around.
10 likes 10mo
e.bakken, post #27: Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do. Go to post
Narrowing post #74, because the general version has more than one answer.
Say it is a preprint in the first line rather than the last. Everything downstream depends on that status and readers arriving from search will not scroll for it.
l.dziedzic, post #45: Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim. Go to post
A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less.
Post #76 and I disagree about the size of the effect, not about the direction.
Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised.
The disagreement above is smaller than it looks once the terms are fixed.
16 likes 10mo
Sample size and pre-registration are checkable in a preprint exactly as in a paper, and they carry the same weight.
It reads as pedantry until the day it does not.
3 likes 10mo
e.almeida, post #51: Post #48 is right about the mechanism and I think understates the practical bit. Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do. Written in the hope of being told what I have missed. Go to post
The arithmetic in post #81 is right; the assumption feeding it is the part to check.
Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim.
Post #81 put the caveat in the right place and I want to underline it.
Where a preprint's numbers are the basis of a claim being repeated here, someone should check whether the published version exists yet. That check is nearly always overdue.
32 likes 10mo
Collapsed as off-topic by two members at trust level 3 or above
Everything in post #81 holds. The case it does not cover is the one I have.
Always label it as a preprint in any discussion. A preprint is not a published trial and it is not settled knowledge. Use it for hypothesis generation and for understanding emerging evidence, not for drawing definitive conclusions.
y.adebayo, post #62: The arithmetic in post #59 is right; the assumption feeding it is the part to check. Unrefereed work is not worthless and a great deal of it survives review substantially unchanged. That is the accurate framing and it is not a licence to treat it as final. Go to post
Narrowing post #84, because the general version has more than one answer.
A preprint with the data and code attached is frequently more checkable than a published paper without them. Reviewed and reproducible are different properties.
What changes between preprint and publication: authors respond to reviewer comments, re-analyse data, correct errors found in peer review, and sometimes report different results from updated analyses. The published version is not the same as the preprint.
That is my reading. Someone else read the same page differently and was reasonable.
0 likes 10mo
Bookmarking this. I will come back when I have something worth adding.
23 likes 10mo
I read post #86 twice before replying, because I had assumed the opposite.
Peer review catches some things reliably — missing methods, unsupported conclusions, statistical mistakes of the obvious kind — and misses others routinely. Knowing which is which makes preprint reading easier.
11 likes 10mo
Check whether it was later published: searching for the preprint's first author and keywords in PubMed might find a published version. The published version is the stronger citation if it exists.
It is worth stating the boring hypothesis before the interesting one.
3 likes 9mo