TY - JOUR
T1 - P value–driven methods were underpowered to detect publication bias
T2 - analysis of Cochrane review meta-analyses
AU - Furuya-Kanamori, Luis
AU - Xu, Chang
AU - Lin, Lifeng
AU - Doan, Tinh
AU - Chu, Haitao
AU - Thalib, Lukman
AU - Doi, Suhail A.R.
N1 - Publisher Copyright:
© 2019 Elsevier Inc.
PY - 2020/2
Y1 - 2020/2
N2 - Objectives: The aim of the study was to investigate the effect of number of studies in a meta-analysis on the detection of publication bias using P value–driven methods. Methods: The proportion of meta-analyses detected by Egger's, Harbord's, Peters', and Begg's tests to have asymmetry suggestive of publication bias were examined in 5,014 meta-analyses from Cochrane reviews. P values were also assessed in meta-analyses with varying number of studies, whereas symmetry was held constant. A simulation study was conducted to investigate if the above tests underestimate or overestimate the presence of publication bias. Results: The proportion of meta-analyses detected as asymmetrical via Egger's, Harbord's, Peters', and Begg's tests decreased by 42.6%, 41.1%, 29.3%, and 28.3%, respectively, when the median number of studies in the meta-analysis decreased from 87 to 14. P values decreased as the number of studies increased in the meta-analysis, despite the level of symmetry remaining constant. The simulation study confirmed that when publication bias is present, P value tests underestimate the presence of publication bias, particularly when study numbers are small. Conclusion: P value–based tests used for the detection of publication bias–related asymmetry in meta-analysis require careful examination, as they underestimate asymmetry. Alternative methods not dependent on the number of studies are preferable.
AB - Objectives: The aim of the study was to investigate the effect of number of studies in a meta-analysis on the detection of publication bias using P value–driven methods. Methods: The proportion of meta-analyses detected by Egger's, Harbord's, Peters', and Begg's tests to have asymmetry suggestive of publication bias were examined in 5,014 meta-analyses from Cochrane reviews. P values were also assessed in meta-analyses with varying number of studies, whereas symmetry was held constant. A simulation study was conducted to investigate if the above tests underestimate or overestimate the presence of publication bias. Results: The proportion of meta-analyses detected as asymmetrical via Egger's, Harbord's, Peters', and Begg's tests decreased by 42.6%, 41.1%, 29.3%, and 28.3%, respectively, when the median number of studies in the meta-analysis decreased from 87 to 14. P values decreased as the number of studies increased in the meta-analysis, despite the level of symmetry remaining constant. The simulation study confirmed that when publication bias is present, P value tests underestimate the presence of publication bias, particularly when study numbers are small. Conclusion: P value–based tests used for the detection of publication bias–related asymmetry in meta-analysis require careful examination, as they underestimate asymmetry. Alternative methods not dependent on the number of studies are preferable.
KW - Asymmetry
KW - Meta-analysis
KW - Publication bias
UR - http://www.scopus.com/inward/record.url?scp=85076027542&partnerID=8YFLogxK
U2 - 10.1016/j.jclinepi.2019.11.011
DO - 10.1016/j.jclinepi.2019.11.011
M3 - Article
SN - 0895-4356
VL - 118
SP - 86
EP - 92
JO - Journal of Clinical Epidemiology
JF - Journal of Clinical Epidemiology
ER -