How this community labels and discusses unrefereed work — the long version

Asking directly, because I could not find a straight answer: How this community labels and discusses unrefereed work — the long version

Reading an unrefereed manuscript properly, and asking whether I am doing it in the right order.

What I do at the moment: check the version and the posting date, read the methods, check whether the analysis matches whatever plan is described, then read the conclusion last. The abstract I read only afterwards, on the theory that reading it first anchors everything.

Is there a step in that sequence which is doing no work, and a step I have left out?

Where I part company with the opening post, and it is a narrow parting.

Posting my community labels numbers with the method attached so they can be discounted properly. Uncontrolled, unblinded, and collected by someone who wanted a particular answer.

Community labels: I have looked for the primary source twice and failed twice. Either it does not exist or it is somewhere I do not know to look, and I would like to know which.

The most useful convention this subcategory has: state the status, state the version, state the date you checked. Three short clauses and the citation stays honest.

Confirming post #2 from a second method, which matters more than confirming it from a second person.

Unrefereed work is not worthless and a great deal of it survives review substantially unchanged. That is the accurate framing and it is not a licence to treat it as final.

That is my reading. Someone else read the same page differently and was reasonable.

I had written a reply contradicting post #4 and deleted it. Here is what survived.

On community labels the community has more anecdote than the confidence in this thread implies, and I include my own contribution in that.

The practical version of community labels is three sentences long. The rigorous version is three pages and reaches the same conclusion with the conditions attached.

I had read the opposite somewhere and cannot now find where, which tells me something.

Post #6 is right about the mechanism and I think understates the practical bit.

Unrefereed work is not worthless. A great deal of it survives review substantially unchanged. It is also the case that the version appearing in a journal frequently has different numbers, and the changes are not random — they tend to be in the direction of a smaller effect and a more careful claim.

I keep a log of this specifically because memory is unreliable about it.

Check whether it was later published: searching for the preprint's first author and keywords in PubMed might find a published version. The published version is the stronger citation if it exists.

One case, stated as one case.

The useful distinction on community labels is between what was measured and what was inferred from it. Both end up in the same sentence and only one of them has error bars.

Taking post #11 at face value and following it one step further.

Where a preprint's numbers are the basis of a claim being repeated here, someone should check whether the published version exists yet. That check is nearly always overdue.

I read post #11 twice before replying, because I had assumed the opposite.

I changed my mind about community labels after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens.

Two claims get bundled together under community labels and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.

Almost every disagreement in threads like this one dissolves once you say which of the two you are making.