Research Methods · N-of-1 designs · continued

Second pass at: Designing a personal experiment that could change your mind posts 31–45

Research Methods › N-of-1 designs worked example effect size confounding data table Solved

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1 · go to the accepted answer.

l.solbergTL220 May 2025#31

Confirming post #28 from a second method, which matters more than confirming it from a second person.

Reporting the whole series rather than the interesting segment is the discipline that makes personal data worth reading. Selective reporting is the default without effort.

Reading it again, the caveat matters more than the finding.

16 likes 14mo

e.verhoevenTL220 May 2025#32

Write down what you would accept as evidence that it did not work, before starting. That one sentence turns an anecdote into an experiment.

32 likes 14mo

d.bakkerTL220 May 2025 · edited#33

s.kuusela, post #13: Where I would push back on the Designing a personal experiment consensus is the confidence, not the direction. The direction looks right. The confidence is borrowed. Go to post

The useful distinction on Designing a personal experiment is between what was measured and what was inferred from it. Both end up in the same sentence and only one of them has error bars.

0 likes in reply to #13 14mo

p.marchettiTL221 May 2025#34

m.balogun, post #21: I read post #17 twice before replying, because I had assumed the opposite. Write down what you would accept as evidence that it did not work, before starting. That one sentence turns an anecdote into an experiment. Go to post

Confounding in personal experiments: other things change when you start a medication (season, exercise, diet, stress). Documenting those confounders helps you understand their contribution to the result.

It is a small point and it changes the answer, which is an awkward combination.

3 likes in reply to #21 14mo

c.boatengTL221 May 2025#35

Sensible. I would want the same detail before I acted on it either.

23 likes 14mo

owen.bradyTL4 Moderator21 May 2025#36

Coming back to post #34, because the follow-up matters more than the original answer.

Record everything that changed, not just the intervention. Personal experiments are confounded by life, and the confounders are only recoverable if they were written down.

It is worth stating the boring hypothesis before the interesting one.

0 likes 14mo

r.laurentTL221 May 2025#37

I disagree with the framing of Designing a personal experiment above, and I think it is a substantive disagreement rather than a terminological one. Setting out why, so it can be checked.

The reasoning depends on an assumption that is doing a lot of work and is never stated. If the assumption holds, the conclusion follows. I do not think it holds generally.

1 like 14mo

mira.patelTL4 Admin21 May 2025#38

h.brandt, post #19: Generalisability: a robust n-of-1 result applies to you. It does not tell you much about whether the effect generalises to others similar to you, much less to people different from you. That holds for the case as described. Change the assumptions and it may not. Go to post

Practical note on Designing a personal experiment: write down what you expect before you look. The number of times I have found what I went looking for is higher than chance would allow.

6 likes in reply to #19 14mo

OstrowskiTL2Member21 May 2025#39

j.ivaturi, post #2: Duration should be set by how long an effect would take to appear rather than by patience. Stopping early is the commonest reason a personal experiment answers nothing. Go to post

Multiple outcomes measured without a primary one means something will move. Nominate the primary in advance and report the rest as secondary.

Not the answer, but possibly the question that gets there.

30 likes in reply to #2 14mo

j.silvaTL221 May 2025#40

A photograph or a device reading is a harder record than a recollection, and where one is available it should be the record.

I have kept the units in throughout, for the obvious reason.

0 likes 14mo

v.szaboTL3Analytical chemist21 May 2025#41

Where I part company with post #39, and it is a narrow parting.

Nothing in a personal experiment is medical advice to anyone including yourself, and several people here have found that a prescriber will happily discuss a well-documented one.

The confident version of this sentence would be wrong, so here is the hedged one.

8 likes 14mo

h.vargaTL222 May 2025#42

Post #39 is the version of this I will quote in future. One addition.

Blinding yourself is harder than it sounds and is not impossible. Somebody else preparing labelled containers is the usual approach and it requires a second person you trust.

2 likes 14mo

d.oyelaranTL3Pharmacist22 May 2025#43

The interesting personal experiments are the ones where the answer was no. They are also the ones least likely to be written up.

A single observation, in a thread that deserves better than single observations.

14 likes in reply to #13 14mo

c.tullochTL222 May 2025#44

Adding a note of thanks rather than an opinion. I did not know most of that.

20 likes 14mo

coldchain_liuTL3Regular22 May 2025#45

Post #43 put the caveat in the right place and I want to underline it.

The question underneath Designing a personal experiment is usually "how would I tell?" rather than "what is true?", and that one has a method attached to it.

Write down what you would expect to see under each hypothesis before you collect anything. If they predict the same observation, collecting it will not help.

13 likes 14mo

This topic was referenced in

Coming back to: Why most self-reports here are not experiments, and that is fineResearch Methods › N-of-1 designs · 62 replies
Pre-registering a personal experiment, seriously — a second datasetResearch Methods › N-of-1 designs · 66 replies

Topic	Participants	Replies	Views	Activity
An ABAB design with a data table and honest limitations — a second dataset Research Methods › N-of-1 designs confounding effect size data table worked example Solved On the subject in the title: An ABAB design with a data table and honest limitations — a second dataset Working notes rather than a conclusion. Posting a small dataset on ABAB design. It is mine, it is…	+106	118	25k	4mo
Why most self-reports here are not experiments, and that is fine Research Methods › N-of-1 designs confounding worked example data table effect size Closed Why most self-reports here are not experiments, and that is fine — that is the question, and I have not found it answered plainly anywhere I have looked. A question about what a single-person experiment can…		2	46k	11mo
Designing a personal experiment that could change your mind — one year on Research Methods › N-of-1 designs effect size data table worked example Designing a personal experiment that could change your mind — one year on — setting out what I have, and where I think it stops being reliable. Two things I would like separated before anyone answers on…	+74	80	17k	2d
Washout with a one-week half-life: the arithmetic Research Methods › N-of-1 designs effect size data table worked example Wiki Washout with a one-week half-life: the arithmetic — setting out what I have, and where I think it stops being reliable. Washout: setting out the arithmetic in full, because I have had to do it twice and I…	+90	96	44k	20mo
An ABAB design with a data table and honest limitations Research Methods › N-of-1 designs effect size confounding worked example An ABAB design with a data table and honest limitations Writing it up because I had to work it out twice and would rather nobody else did. An honest uncertainty about ABAB design rather than a disguised…		2	45k	5mo

Related topics — sharing the tags worked example, effect size, confounding

Topic	Participants	Replies	Views	Activity
Coming back to: Column chemistry choices for a 40-residue peptide Analytics › HPLC & UHPLC peak integration system suitability gradient method data table On the subject in the title: Column chemistry choices for a 40-residue peptide Working notes rather than a conclusion. Column chemistry choices — I have the observation and I do not trust my interpretation of…		2	43k	16mo
[2026 update] Reporting a disappointing outcome, and why it belongs here Outcomes › Progress reports DXA long read data table Pinned Closed On the subject in the title: Reporting a disappointing outcome, and why it belongs here Working notes rather than a conclusion. Proposing that Reporting a disappointing outcome deserves a maintained page…		2	11k	8mo
How this community labels and discusses unrefereed work Evidence › Preprints risk of bias effect size preprint needs a source How this community labels and discusses unrefereed work — that is the question, and I have not found it answered plainly anywhere I have looked. Reading an unrefereed manuscript properly, and asking whether I…	+112	119	1.3k	2y
Revisiting: List price versus negotiated price versus what you pay Access › Pricing cost per milligram compounding data table formulary Revisiting: List price versus negotiated price versus what you pay — setting out what I have, and where I think it stops being reliable. Working notes on List price versus negotiated price rather than a…	+24	28	50k	12mo
Follow-up: Why the titration planner refuses to produce a schedule below a floor Data & Tools › Calculators worked example syringe units site feedback The question in the title: Why the titration planner refuses to produce a schedule below a floor I will give what I have already checked below so nobody repeats it. A follow-up question about titration…	+120	146	14k	10mo

This topic was referenced in

Suggested topics

Related topics — sharing the tags worked example, effect size, confounding