Absence of evidence and evidence of absence

Absence of evidence and evidence of absence Writing it up because I had to work it out twice and would rather nobody else did.

Asking about absence of evidence on behalf of the question I keep seeing asked badly, including by me.

Framed properly it is answerable. Framed the usual way it is not, and that is most of why the previous threads went nowhere.

This follows the opening post rather than contradicting it.

Adding a null result on absence of evidence. I looked, carefully, and found nothing, and null results deserve posting precisely because they never are.

Baseline imbalance in a randomised trial is expected by chance and adjusting for it post hoc is a choice that should have been pre-specified.

A modest claim, modestly supported.

Adding thanks rather than a view. I do not have a view worth the space.

The most useful thing anyone has posted about absence of evidence in this category was a table of what had been measured and by whom. That is what I would want again.

Absence of evidence has been discussed here with more heat than it deserves, mostly because two definitions have been in play the whole time.

Coming back to post #5, because the follow-up matters more than the original answer.

Number needed to treat: how many people need to be treated to prevent one bad outcome or achieve one good outcome. More intuitive than relative risk reduction.

Small point, but it is the one that usually catches people.

Absence of evidence and evidence of absence: if a study is small and finds no effect, that is absence of evidence, not evidence of absence. A larger study might find an effect that a small study missed.

It is one reading of the data and not the only reasonable one.

On post #5 — agreed on the reasoning, with one qualification.

Effect sizes: the magnitude of a difference, not just whether it is statistically significant. A difference that is significant (p<0.05) might be too small to matter. A large effect might not be significant if sample size is small.

Not disagreeing with anyone above, just adding the bit I keep having to look up.

Absence of evidence is worth one more sentence than it usually gets, and the sentence is the one about how the number was arrived at.

I have no financial interest in anything named in this thread and I want to say so before I comment on absence of evidence, because it is the sort of subject where it matters.

Post #8 and I disagree about the size of the effect, not about the direction.

Confounding: a third variable explains an apparent association. In randomised data, randomisation balances confounders. In observational data, confounders can be adjusted for but unknown ones cannot.

Genuine question rather than a rhetorical one: has anyone here actually observed absence of evidence, as opposed to read about it? The thread is long and I cannot tell.

Adding a reference point for absence of evidence. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.