The most useful reply I ever got about tracking a preprint was a request to state my units. It sounds like pedantry and it has saved me twice.
19 likes 8mo
Evidence · Preprints · continued
Tracking a preprint to publication: a worked example posts 61–90
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
o.vogel, post #13: Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim. Go to post
Adding the measurement that post #61 says would settle it.
Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim.
Reading it back, the second half matters more than the first.
Where I part company with post #61, and it is a narrow parting.
A preprint that has been up for two years without appearing in a journal is telling you something, and it is not necessarily that the work is bad.
This is the sort of thing the wiki should carry and currently does not.
2 likes 8mo
Version numbers on a preprint server matter. Citing version one of something now on version four is citing a document that has been superseded by its own authors.
0 likes 8mo
h.ferrari, post #22: Where a preprint contradicts a published trial, the prior should favour the trial. Where it extends one, the preprint can be genuinely useful. Go to post
Building anything durable on a preprint means committing to rebuild it. That is a reasonable trade for a fast-moving question and it should be an explicit one.
r.vukovic, post #7: Worth separating two things that post #5 runs together. Screening on a preprint server is minimal and is not review. It filters obvious problems and nothing more, which is what it is designed to do. Adding it in case it saves somebody the afternoon it cost me. Go to post
Acknowledging rather than arguing. The reasoning holds as far as I can follow it.
Post #65 put the caveat in the right place and I want to underline it.
Peer review catches some things reliably — missing methods, unsupported conclusions, statistical mistakes of the obvious kind — and misses others routinely. Knowing which is which makes preprint reading easier.
The confident version of this sentence would be wrong, so here is the hedged one.
0 likes 8mo
Collapsed as off-topic by two members at trust level 3 or above
Building on post #65 rather than restating it.
Author track record is a weak signal and it is a signal. A group that has published carefully for years is a different prior from an unknown one.
Worth reading the earlier posts in this thread before acting on mine.
r.scholten, post #16: I read post #12 twice before replying, because I had assumed the opposite. Always label it as a preprint in any discussion. A preprint is not a published trial and it is not settled knowledge. Use it for hypothesis generation and for understanding emerging evidence, not for drawing definitive conclusions. That is a description of… Go to post
A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less.
Adding this to the thread rather than to the wiki, because I am not confident enough for the wiki.
If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update.
18 likes 8mo
s.roos, post #51: Post #50 answers the question as asked. The question underneath it is different. Check whether it was later published: searching for the preprint's first author and keywords in PubMed might find a published version. The published version is the stronger citation if it exists. That matches what I was told, which is not the same as… Go to post
Narrowing post #69, because the general version has more than one answer.
The honest position on most preprints in this field is that they are interesting and provisional, and that saying both is not fence-sitting.
That is one dataset and I would not build a rule on it.
Collapsed as off-topic by two members at trust level 3 or above
A preprint with the data and code attached is frequently more checkable than a published paper without them. Reviewed and reproducible are different properties.
MJayawardena, post #37: Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised. Go to post
This is the first time the answer has come with its own limits attached. Appreciated.
Post #73 answers the question as asked. The question underneath it is different.
Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised.
0 likes 8mo
Peer review as a filter: published papers have survived a review process that rejects many submissions. Preprints have not. Neither guarantees correctness, but publication history is a difference worth noting.
3 likes 8mo
f.chowdhury, post #47: Peer review as a filter: published papers have survived a review process that rejects many submissions. Preprints have not. Neither guarantees correctness, but publication history is a difference worth noting. One more caveat and then I will stop qualifying: the sample selected itself. Go to post
Post #75 and I disagree about the size of the effect, not about the direction.
Distinguishing three things in the tracking a preprint discussion that keep getting used interchangeably: the observation, the proposed mechanism, and the recommendation that gets attached to both.
Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim.
5 likes 8mo
s.oyelaran, post #38: I had written a reply contradicting post #34 and deleted it. Here is what survived. Adding the boring version of tracking a preprint, because the interesting version keeps getting posted and the boring one is usually right. Check the ordinary explanations, in order, and stop when one of them accounts for what you are seeing. Most of the… Go to post
Server repositories: medRxiv for medicine, bioRxiv for biology. Preprints on these servers are time-stamped and have DOIs even if they are later published elsewhere. Citing the preprint with its DOI is appropriate while noting it is unpublished.
The answer changed when I changed how I was measuring, which was informative.
I changed my mind about tracking a preprint after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.
28 likes 8mo
r.coelho, post #32: Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised. Posting it because the silence on this was starting to look like agreement. Go to post
Two claims get bundled together under tracking a preprint and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
o.vogel, post #13: Where numbers change between preprint and published version, the changes are not random. They tend toward a smaller effect and a more carefully hedged claim. Go to post
Reading rather than answering, but this is the post I would point somebody at.
Always label it as a preprint in any discussion. A preprint is not a published trial and it is not settled knowledge. Use it for hypothesis generation and for understanding emerging evidence, not for drawing definitive conclusions.
For what it is worth, the same held on the two occasions I checked.
9 likes 8mo
Worth separating two things that post #87 runs together.
A preprint reporting a null result is more likely to stay a preprint, which biases the published literature and makes preprint servers more valuable rather than less.
I would call that likely rather than established.
13 likes 8mo
dexa_twice_yearly, post #4: Picking up the opening post: that is the part I would want checked first. Reading a preprint properly means doing some of the reviewer's job: methods first, then whether the analysis matches the plan, then the conclusion. Correct me on the arithmetic if it is wrong; I would rather know. Go to post
If you cite a preprint in a maintained document here, the convention is to label it in the sentence and to note the check date. The document review will catch the update.
None of the above is medical advice and I am not qualified to give any.