On the subject in the title: Whether an aggregate is worth publishing at all: a disputed topic Working notes rather than a conclusion.
An honest uncertainty about whether an aggregate rather than a disguised assertion.
I do not know the answer and I have not been able to find one. What I have is the shape of the question, which may be worth more than my guess at the answer.
Practical note on whether an aggregate: write down what you expect before you look. The number of times I have found what I went looking for is higher than chance would allow.
Publishing the raw records alongside the summary is what makes a dataset checkable. A summary alone asks for trust that nobody has earned.
A request rather than an answer: could whoever has the primary source for whether an aggregate post it? I have seen the claim three times this month and each version had lost a qualifier.
Limitations of datasets: all community-collected data has limitations. The population is self-selected (people in this community are not representative of all people using these compounds). Reporting bias is real (remarkable outcomes get reported; mundane outcomes do not).
The confident answers on whether an aggregate and the well-sourced answers are not the same answers, which is the most useful thing I have learned reading this category.
Speaking only to whether an aggregate as I have actually seen it, rather than as it is usually described: the effect is real, it is smaller than the thread suggests, and the variance between people is larger than the effect.
A structured format with one observation per row is worth the initial effort. Wide formats are easier to enter and much harder to analyse.
The number is defensible. The precision I gave it is not.
I disagree with the framing of whether an aggregate above, and I think it is a substantive disagreement rather than a terminological one. Setting out why, so it can be checked.
The reasoning depends on an assumption that is doing a lot of work and is never stated. If the assumption holds, the conclusion follows. I do not think it holds generally.
Adding a data point of agreement rather than a data point.
On whether an aggregate, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit.
If you can post the two or three numbers you are working from, several people here will check the arithmetic rather than argue about the conclusion.
Post #16 answers the question as asked. The question underneath it is different.
Licence and reuse terms are worth stating even informally. People will reuse it, and the honest position is to say what you are happy with.
I would rather be precise about what I do not know than vague about what I do.
Whether an aggregate has been discussed here with more heat than it deserves, mostly because two definitions have been in play the whole time.
This follows post #19 rather than contradicting it.
A codebook describing each field takes twenty minutes and is what makes the file usable by anyone but you. Most shared datasets here do not have one.
Post #19 put the caveat in the right place and I want to underline it.
Where the whether an aggregate discussion usually stalls is that nobody wants to say "I do not know" and everyone is willing to say "it varies". Those are the same sentence with different clothes on.
Self-reported, unblinded, self-selected data has known biases and is still worth collecting, provided every one of those words appears in the description.
I am reporting what happened, not recommending it.
That is the distinction I keep failing to hold on to. Written down now.
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Whether an aggregate is worth publishing at all: a disputed topic — does this still hold?
Whether an aggregate is worth publishing at all: a disputed topic — does this still hold? I have a specific reason for asking rather than idle curiosity, and the context is below. An honest uncertainty about…
|
+14 | 18 | 3.4k | 2mo |
|
Revisiting: Sample size in a voluntary survey: the selection problem
Revisiting: Sample size in a voluntary survey: the selection problem — setting out what I have, and where I think it stops being reliable. I have spent a fortnight trying to pin Sample size down and I want to…
|
2 | 56k | 9mo | |
|
About the Datasets category
Community-collected datasets, their collection methods, and their limitations. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its author and a…
|
+6 | 11 | 42k | 12mo |
|
Coming back to: A community side-effect dataset, with its response rate and biases
On the subject in the title: A community side-effect dataset, with its response rate and biases Working notes rather than a conclusion. Reading back through what has been written here about community…
|
+27 | 31 | 46k | 9mo |
|
A community side-effect dataset, with its response rate and biases
A community side-effect dataset, with its response rate and biases — setting out what I have, and where I think it stops being reliable. Community side-effect dataset: setting out the arithmetic in full,…
|
2 | 5.1k | 19h |
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
TB-500 and thymosin beta-4: the fragment versus the protein
TB-500 and thymosin beta-4: the fragment versus the protein Writing it up because I had to work it out twice and would rather nobody else did. TB-500 and thymosin beta-4: what I expected, what I found, and…
|
+107 | 118 | 22k | 11mo |
|
Sample size intuition for a personal experiment
On the subject in the title: Sample size intuition for a personal experiment Working notes rather than a conclusion. A follow-up question about sample size intuition that I did not know to ask the first time.…
|
4 | 62k | 18mo | |
|
Second pass at: The mass-error checker and its intended scope
Posting this under the heading it deserves: Second pass at: The mass-error checker and its intended scope Everything below is what sits behind that. Posting a small dataset on mass-error checker. It is mine,…
|
+70 | 74 | 1.7k | 8mo |
|
Trust level 3 requirements: a clarification — does this still hold?
Asking directly, because I could not find a straight answer: Trust level 3 requirements: a clarification — does this still hold? Two things I would like separated before anyone answers on Trust level 3…
|
+77 | 90 | 57k | 9h |
|
What a confidence interval means, from scratch
What a confidence interval means, from scratch — that is the question, and I have not found it answered plainly anywhere I have looked. Asking about confidence interval directly, because I have read four…
|
2 | 2.6k | 2mo |