Correlation in a self-tracked dataset: what it can support — setting out what I have, and where I think it stops being reliable.
Posting a small dataset on correlation. It is mine, it is uncontrolled, and the method is stated so it can be discounted appropriately.
What I would like is not agreement but a second dataset collected by someone with no stake in mine. If one exists I would rather read it than argue for this one.
Time-to-event analysis handles differing follow-up in a way a simple proportion cannot, which is why event rates and Kaplan-Meier estimates can differ.
Reading it back, the second half matters more than the first.
Post #3 describes the usual case. This is about the unusual one.
Adding a small correction to the correlation summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
Correlation is a good example of a question where the honest answer is boring and the interesting answers are unsupported. I would go with boring.
P-values and significance: p<0.05 means the data would be surprising if the null hypothesis were true, not that the null hypothesis is false. A non-significant p-value does not mean "no effect".
Two sources, same conclusion, and I could not rule out that one copied the other.
Small methodological point on correlation: repeating a measurement is cheap and resolves most of what is being argued about here at no cost to anyone.
Taking post #8 at face value and following it one step further.
Rounding and significant figures carry information about precision. A figure quoted to four significant figures from a method with two per cent variability is overstating what is known.
Post #10 and I disagree about the size of the effect, not about the direction.
Where an analysis was changed after seeing the data, the honest thing is to report both and say which was pre-specified.
I am confident about the direction and much less about the magnitude.
No disagreement from me. Posting only so the question does not look ignored.
Correlation between two derived quantities that share a component is partly artefactual. It is a common trap in analyses of ratios.
Worth separating two things that post #15 runs together.
What I can speak to on correlation is narrow, so I will keep it narrow rather than generalising from it. Beyond that boundary I do not know.
One more thing on correlation that took me far too long to see: the two figures people quote are not measuring the same quantity. Once you notice that, the apparent contradiction disappears.
Counterpoint on correlation, offered without confidence: the same observation is consistent with a much duller explanation, and nobody has ruled the dull one out.
Multiple testing inflates the false-positive rate in a way that is entirely predictable and entirely correctable. The correction should be declared in advance.
The literature is thinner on this than the confidence in the thread implies.
Where I part company with post #19, and it is a narrow parting.
Adding a reference point for correlation. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.
Post #19 is the version of this I will quote in future. One addition.
Practical answer on correlation, since the theoretical one is upthread: do the simplest check first, write down the result, and only then decide whether the complicated explanation is needed. It usually is not.
Baseline imbalance in a randomised trial is expected by chance and adjusting for it post hoc is a choice that should have been pre-specified.
Scoping that to what I have actually seen rather than what I have read.
Nothing to add on the substance. Thank you for taking the question at face value.
Two people in this thread mean different things by correlation and are disagreeing about the definition while believing they are disagreeing about the facts. Worth pausing to define it.
Building on post #23 rather than restating it.
A methods point on correlation rather than a substantive one: if the comparison is not like for like, the difference you are measuring is the difference in method.
Marking my uncertainty on correlation explicitly. I am confident about the direction, much less confident about the size, and not confident at all that it generalises past the case in the first post.
Number needed to treat: how many people need to be treated to prevent one bad outcome or achieve one good outcome. More intuitive than relative risk reduction.
Reading it again, the caveat matters more than the finding.