Topic summary

Risk-of-bias tools and the judgement they conceal — does this still hold?

Evidence › Meta-analyses heterogeneity risk of bias effect size publication bias Solved

This is a generated summary. It shows the 5 most-liked posts from a topic of 31, in their original order, with the accepted answer included where one exists. It is a reading aid and it will miss nuance — the full topic is the record.

GDashwoodTL3Regular20 Apr 2026#1

Risk-of-bias tools and the judgement they conceal — does this still hold? — that is the question, and I have not found it answered plainly anywhere I have looked.

Asking about Risk-of-bias tools on behalf of the question I keep seeing asked badly, including by me.

Framed properly it is answerable. Framed the usual way it is not, and that is most of why the previous threads went nowhere.

29 likes 3mo

LeitermanTL3Regular27 Apr 2026 · edited#2

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

The general case is well covered; this is the awkward specific one.

31 likes 3mo

g.ekstromTL2 Solution2 May 2026#3

I would rather this thread reach "we do not know" about Risk-of-bias tools than reach a confident answer that nobody can support when asked.

8 likes 3mo

j.ivaturiTL225 May 2026#9

BBramley, post #4: On post #2 — agreed on the reasoning, with one qualification. On Risk-of-bias tools, the part that usually goes wrong is that the question is asked as though it has one answer. It has a range, and the width of the range is the interesting bit. If you can post the two or three numbers you are working from, several people here will check… Go to post

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

30 likes in reply to #4 2mo

h.kimaniTL225 Jun 2026#19

Adding a reference point for Risk-of-bias tools. Mine is a single case, collected without controls, and I am posting the method alongside it so it can be discounted appropriately.

30 likes 1mo

Read the full topic (31 posts)

This topic was referenced in

Publication bias detection and its low powerEvidence › Meta-analyses · 2 replies
Second pass at: A pooled estimate that changed when one trial was addedEvidence › Meta-analyses · 30 replies
Coming back to: Reading a meta-analysis you disagree with, fairlyEvidence › Meta-analyses · 34 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Publication bias detection and its low power — does this still hold? Evidence › Meta-analyses publication bias risk of bias meta-analysis heterogeneity Solved The question in the title: Publication bias detection and its low power — does this still hold? I will give what I have already checked below so nobody repeats it. Collecting what is known about Publication…	+24	28	2.4k	3mo
Publication bias detection and its low power Evidence › Meta-analyses heterogeneity publication bias effect size On the subject in the title: Publication bias detection and its low power Working notes rather than a conclusion. Working notes on publication bias detection rather than a conclusion. I would rather post the…		2	3.2k	1d
What pooling buys you and what it destroys Evidence › Meta-analyses publication bias risk of bias meta-analysis heterogeneity Solved The question in the title: What pooling buys you and what it destroys I will give what I have already checked below so nobody repeats it. The question about pooling buys that I actually want answered is the…	+74	83	6.6k	8mo
Heterogeneity as information rather than as a nuisance Evidence › Meta-analyses heterogeneity risk of bias meta-analysis effect size Posting this under the heading it deserves: Heterogeneity as information rather than as a nuisance Everything below is what sits behind that. An honest uncertainty about heterogeneity rather than a disguised…	+88	95	1.2k	6mo
Random versus fixed effects: choosing rather than defaulting Evidence › Meta-analyses effect size publication bias risk of bias heterogeneity Closed Random versus fixed effects: choosing rather than defaulting — setting out what I have, and where I think it stops being reliable. Random versus fixed effects, and specifically the version of it that the…	+16	20	22k	13mo

Related topics — sharing the tags heterogeneity, risk of bias, effect size

Topic	Participants	Replies	Views	Activity
SURMOUNT-4 and what withdrawal data does and does not tell an individual — one year on Compounds › Tirzepatide SURMOUNT programme randomised trial tirzepatide SURPASS programme Solved Closed Posting this under the heading it deserves: SURMOUNT-4 and what withdrawal data does and does not tell an individual — one year on Everything below is what sits behind that. I would like to disagree carefully…	+122	136	48k	9mo
Journal club: indirect comparison between two programmes, defended and attacked Evidence › Journal club journal club randomised trial number needed to treat risk of bias Journal club: indirect comparison between two programmes, defended and attacked Writing it up because I had to work it out twice and would rather nobody else did. Trying to work out what would count as…	+122	131	5.2k	just now
Retatrutide dose escalation in the published trials Compounds › Retatrutide preprint retatrutide effect size Posting this under the heading it deserves: Retatrutide dose escalation in the published trials Everything below is what sits behind that. An honest uncertainty about retatrutide dose escalation rather than a…	+149	159	30k	15mo
Journal club: the CagriSema phase 2 combination paper Evidence › Journal club randomised trial estimand effect size On the subject in the title: Journal club: the CagriSema phase 2 combination paper Working notes rather than a conclusion. CagriSema phase 2 combination paper — I have the observation and I do not trust my…		2	329	21h
Coming back to: Nausea profile of amylin analogues compared with GLP-1 agonists Compounds › Cagrilintide & amylin analogues effect size cagrilintide cagrisema amylin analogues Posting this under the heading it deserves: Nausea profile of amylin analogues compared with GLP-1 agonists Everything below is what sits behind that. Nausea profile of amylin analogues — I have the…	+48	53	31k	13mo