Posting this under the heading it deserves: Pre-registering a personal experiment, seriously — a second dataset Everything below is what sits behind that.
Posting a small dataset on Pre-registering a personal experiment. It is mine, it is uncontrolled, and the method is stated so it can be discounted appropriately.
What I would like is not agreement but a second dataset collected by someone with no stake in mine. If one exists I would rather read it than argue for this one.
The reason Pre-registering a personal experiment is hard to answer is that the obvious measurement and the relevant quantity are not the same thing, and substituting one for the other is silent.
Bookmarking this. I will come back when I have something worth adding.
Regression to the mean explains a great many personal experiments. People start when things are at their worst, and things at their worst tend to improve.
Someone will know this better than I do and I hope they say so.
Post #4 is right about the mechanism and I think understates the practical bit.
Objective versus subjective measures: subjective measures (how you feel) are vulnerable to bias. Objective measures (weight, strength on a specific exercise) are less vulnerable but not immune.
Coming back to post #2, because the follow-up matters more than the original answer.
A definition problem is doing most of the work in this Pre-registering a personal experiment discussion. Once the term is pinned down I suspect the disagreement mostly goes away and what is left is small.
A single-person experiment can be genuinely informative if it has a pre-specified outcome, a defined period, and a plan written before the data arrives. Most accounts here have none of those.
I have seen it go both ways, which is why I hedge.
Practical experience of Pre-registering a personal experiment, offered as one case with the conditions stated, not as a general finding. Conditions first, because they are what make it interpretable.
Designing a personal experiment that could actually change your mind: that is the standard for an n-of-1 design. An experiment designed so that any result confirms what you already believed has not changed anything.
The right answer here may simply be that it has not been measured.
Worth separating two things that post #9 runs together.
Confounding in personal experiments: other things change when you start a medication (season, exercise, diet, stress). Documenting those confounders helps you understand their contribution to the result.
I would rather be precise about what I do not know than vague about what I do.
Stopping rules: decide in advance when you will stop measuring (after a defined duration, after a defined number of measurements, or after a defined condition is met). Not deciding in advance means stopping when the result satisfies you, which is bias.
The literature is thinner on this than the confidence in the thread implies.
Nobody has said the unglamorous part of Pre-registering a personal experiment yet, so: most of the variation is explained by things that are boring to write about and easy to check.
Taking post #13 at face value and following it one step further.
Duration should be set by how long an effect would take to appear rather than by patience. Stopping early is the commonest reason a personal experiment answers nothing.
Flagging that the sources on this are thinner than the confidence in the thread suggests.
Nothing to add, except that this is the answer I would give if asked.
Reading back through the Pre-registering a personal experiment threads from last year, the same three questions come up every time and only one of them has ever been answered properly. That seems like a documentation gap rather than a knowledge gap.
Worth separating two things that post #20 runs together.
Adding a small correction to the Pre-registering a personal experiment summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
Post #22 answers the question as asked. The question underneath it is different.
Write down what you would accept as evidence that it did not work, before starting. That one sentence turns an anecdote into an experiment.
Record everything that changed, not just the intervention. Personal experiments are confounded by life, and the confounders are only recoverable if they were written down.
It is one reading of the data and not the only reasonable one.
The version of Pre-registering a personal experiment that circulates here is a simplification of a simplification. It is not wrong, but it has lost the conditions under which it holds, and those conditions are where the interesting cases live.
Narrowing post #26, because the general version has more than one answer.
Designing a personal experiment that could actually change your mind: that is the standard for an n-of-1 design. An experiment designed so that any result confirms what you already believed has not changed anything.
Thank you for taking the time. That was more work than a reply usually is.
The interesting personal experiments are the ones where the answer was no. They are also the ones least likely to be written up.
That has been true for the cases I have seen and I have not seen many.