Aggregate verification data across the directory, with caveats

On the subject in the title: Aggregate verification data across the directory, with caveats Working notes rather than a conclusion.

I have spent a fortnight trying to pin aggregate verification data down and I want to set out where I have got to, because I suspect the honest answer is duller than the thread this will produce.

What I have: a consistent observation across a small number of cases, collected the same way each time. What I do not have: any controlled comparison, or any reason to think my cases are representative.

The specific question is whether the pattern survives once the obvious confounder is removed. I cannot remove it with what I have.

Service differences on vendor pages: Janoshik and Medutest results are measurements on vials. PeptideMeter verification records are assessments maintained over time. VendorInvestigate reviews are documentation assessments. We cite all four because they answer different questions.

I would not lead a decision with this, but I would not ignore it either.

Narrowing post #2, because the general version has more than one answer.

What I want from this aggregate verification data thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.

The thing about aggregate verification data that took me longest to accept is that a plausible mechanism is not evidence of an effect. It is a reason to look, not a result.

A record with four results over sixteen months is a thin record rather than a bad one, and the honest description says which.

Post #4 put the caveat in the right place and I want to underline it.

Repeat determination on the same product line months apart is the measurement that speaks to consistency, and consistency is what most buyers actually want.

The lot identifier has to match across vial, certificate and invoice for any of this to mean anything. It is the floor and it is worth checking every time.

Two sources, same conclusion, and I could not rule out that one copied the other.

Nothing in a verification record is a guarantee about the vial in your hand. That check is yours and nobody can do it for you.

It cost nothing to check and would have cost something not to.

Post #6 is the version of this I will quote in future. One addition.

The record shows what was measured. Everything beyond that is inference and this site tries to keep the line visible.

Helpful, and short, which on this subject is harder than long.

Where I part company with post #7, and it is a narrow parting.

Repeat testing: if you want to verify the same lot a second time, testing with a different service strengthens the conclusion. Testing with the same service verifies reproducibility but uses the same method.

I am reporting what happened, not recommending it.

Post #11 is the version of this I will quote in future. One addition.

I have no financial interest in anything named in this thread and I want to say so before I comment on aggregate verification data, because it is the sort of subject where it matters.

Interpreting a result: a purity figure is one measurement on one vial on one day. Agreement across two services is much stronger. A single high number does not establish a pattern.

Understood, and I withdraw the assumption I opened with.