Research Methods · N-of-1 designs · continued

Pre-registering a personal experiment, seriously posts 61–90

Research Methods › N-of-1 designs worked example data table effect size confounding

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.

s.achebeTL231 May 2025#61

titration_diary, post #21: Everything in post #19 holds. The case it does not cover is the one I have. I changed my mind about pre-registering a personal experiment after someone here asked me for the source and I could not produce one. That is worth saying out loud because it is the ordinary way it happens. Go to post

Everything in post #59 holds. The case it does not cover is the one I have.

An observation about pre-registering a personal experiment that I cannot explain and am posting anyway, on the principle that unexplained observations are more useful public than private.

5 likes in reply to #21 14mo

VThorvaldsenTL3Regular31 May 2025#62

glossary_desk, post #8: On pre-registering a personal experiment: the maintained page in the documentation commons covers the general case with citations and a review date, which is more reliable than any reply here including this one. Go to post

Narrowing post #59, because the general version has more than one answer.

A rolling mean over several days is far more informative than any single reading for anything that varies day to day, which is nearly everything.

0 likes in reply to #8 14mo

i.rasmussenTL231 May 2025#63

Write down what you would accept as evidence that it did not work, before starting. That one sentence turns an anecdote into an experiment.

Small point, but it is the one that usually catches people.

29 likes 14mo

n.torrenceTL3Regular31 May 2025#64

Adding what did not work for me on pre-registering a personal experiment, since the failures never get written up and they are half the useful information.

14 likes 14mo

e.ferrariTL231 May 2025#65

Answering the question post #63 raises rather than the one it answers.

An n of one tells you about one person, which is the person you are most interested in. That is the whole value and it is not nothing.

9 likes 14mo

s.silvaTL231 May 2025#66

t.wojcik, post #15: Measure the same thing the same way at the same time of day. Most of the noise in personal data is measurement protocol rather than biology. Anyone who has looked at this more carefully, please correct the record. Go to post

Two sentences on pre-registering a personal experiment and then I will stop, because the rest is speculation and the thread is better without mine.

What is documented is narrow. What is inferred from it is broad. The gap between them is where every argument here lives.

2 likes in reply to #15 14mo

j.hartmannTL231 May 2025#67

The reason pre-registering a personal experiment is hard to answer is that the obvious measurement and the relevant quantity are not the same thing, and substituting one for the other is silent.

0 likes 14mo

KnowltonTL331 May 2025#68

Collapsed as off-topic by two members at trust level 3 or above

Adding thanks rather than a view. I do not have a view worth the space.

l.wikstromTL231 May 2025#69

Reporting the whole series rather than the interesting segment is the discipline that makes personal data worth reading. Selective reporting is the default without effort.

Noting that the question and the thing people usually mean by it are different.

14 likes 14mo

v.nascimentoTL231 May 2025 · edited#70

i.rasmussen, post #63: Write down what you would accept as evidence that it did not work, before starting. That one sentence turns an anecdote into an experiment. Small point, but it is the one that usually catches people. Go to post

What I would check first on pre-registering a personal experiment is whether the thing being measured moved or whether the way of measuring it moved. Those look identical in a graph.

5 likes in reply to #63 14mo

taper_tableTL3Regular31 May 2025#71

Narrowing post #70, because the general version has more than one answer.

Pre-registering a personal experiment sits at the boundary between what this community can usefully discuss and what it cannot, and I think it falls on the discussable side, narrowly.

10 likes 14mo

m.agyemanTL21 Jun 2025#72

Everything in post #69 holds. The case it does not cover is the one I have.

It does not tell you about anybody else, which is why aggregating these accounts does not produce evidence of the kind people want it to.

22 likes 14mo

excursion_checkTL3Regular1 Jun 2025#73

Distinguishing three things in the pre-registering a personal experiment discussion that keep getting used interchangeably: the observation, the proposed mechanism, and the recommendation that gets attached to both.

0 likes 14mo

a.hartmannTL21 Jun 2025#74

m.yilmaz, post #11: The interesting personal experiments are the ones where the answer was no. They are also the ones least likely to be written up. I am not the right person to answer the follow-up to this. Go to post

Regression to the mean explains a great many personal experiments. People start when things are at their worst, and things at their worst tend to improve.

It is worth checking rather than assuming, which costs nothing.

3 likes in reply to #11 14mo

buffer_marginTL3Regular1 Jun 2025#75

Acknowledging rather than arguing. The reasoning holds as far as I can follow it.

6 likes 14mo

b.restrepoTL21 Jun 2025#76

Pre-registering a personal experiment would be much easier to settle if anyone reported the denominator. Almost nobody reports the denominator.

16 likes 14mo

PSkarbekTL3Regular1 Jun 2025#77

Duration should be set by how long an effect would take to appear rather than by patience. Stopping early is the commonest reason a personal experiment answers nothing.

Somebody will have a better source than mine, and I hope they post it.

0 likes 14mo

t.abubakarTL21 Jun 2025 · edited#78

c.ostergaard, post #20: Reporting the whole series rather than the interesting segment is the discipline that makes personal data worth reading. Selective reporting is the default without effort. Marking that as an opinion rather than a finding. Go to post

Worth separating two things that post #76 runs together.

Nothing in a personal experiment is medical advice to anyone including yourself, and several people here have found that a prescriber will happily discuss a well-documented one.

1 like in reply to #20 14mo

v.krastevTL21 Jun 2025#79

A note on scope: what I am saying about pre-registering a personal experiment applies to the case in the first post and I would not extend it further without checking.

21 likes 14mo

m.achebeTL21 Jun 2025#80

The interesting personal experiments are the ones where the answer was no. They are also the ones least likely to be written up.

The conclusion is tentative; the arithmetic underneath it is not.

0 likes 14mo

VThorvaldsenTL3Regular1 Jun 2025#81

l.krastev, post #7: Nothing in a personal experiment is medical advice to anyone including yourself, and several people here have found that a prescriber will happily discuss a well-documented one. Noting that I have skin in this question and have tried to discount for it. Go to post

Designing a personal experiment that could actually change your mind: that is the standard for an n-of-1 design. An experiment designed so that any result confirms what you already believed has not changed anything.

16 likes in reply to #7 14mo

i.rasmussenTL21 Jun 2025#82

Adding a reference point for pre-registering a personal experiment. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.

6 likes 14mo

r.mensaTL21 Jun 2025 · edited#83

The failure mode on pre-registering a personal experiment is boring rather than dramatic. It is almost always the step everyone assumes was done correctly because it is too simple to get wrong.

1 like 14mo

s.achebeTL21 Jun 2025#84

Adding the measurement that post #81 says would settle it.

Measure the same thing the same way at the same time of day. Most of the noise in personal data is measurement protocol rather than biology.

I would want to see it done twice before believing it once.

0 likes 14mo

s.poulsenTL3Regular2 Jun 2025#85

s.cardoso, post #17: My position on pre-registering a personal experiment is current rather than settled. I have revised it once already and I expect to again, so treat it accordingly. Go to post

Record everything that changed, not just the intervention. Personal experiments are confounded by life, and the confounders are only recoverable if they were written down.

Reading it back, the second half matters more than the first.

23 likes in reply to #17 14mo

a.petrovTL22 Jun 2025#86

Measure the same thing the same way at the same time of day. Most of the noise in personal data is measurement protocol rather than biology.

This is the sort of thing the wiki should carry and currently does not.

10 likes 14mo

n.torrenceTL3Regular2 Jun 2025#87

Answering the question post #85 raises rather than the one it answers.

If someone has run pre-registering a personal experiment properly I would rather read that than my own reconstruction of it. Posting mine only because the thread has gone quiet.

3 likes 14mo

e.krastevTL22 Jun 2025#88

Collapsed as off-topic by two members at trust level 3 or above

The arithmetic in post #85 is right; the assumption feeding it is the part to check.

Marking my uncertainty on pre-registering a personal experiment explicitly. I am confident about the direction, much less confident about the size, and not confident at all that it generalises past the case in the first post.

j.habermannTL3Regular2 Jun 2025#89

Understood. Thank you for being specific about the limits of it.

7 likes 14mo

r.oyelaranTL22 Jun 2025 · edited#90

Post #87 answers the question as asked. The question underneath it is different.

An n of one tells you about one person, which is the person you are most interested in. That is the whole value and it is not nothing.

I would hold that lightly until someone with a larger sample weighs in.

1 like 14mo