TY - JOUR
T1 - Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury
AU - Gravesteijn, Benjamin Y.
AU - Nieboer, Daan
AU - Ercole, Ari
AU - Lingsma, Hester F.
AU - Nelson, David
AU - van Calster, Ben
AU - Steyerberg, Ewout W.
AU - Akerlund, Cecilia
AU - Amrein, Krisztina
AU - Andelic, Nada
AU - Andreassen, Lasse
AU - Anke, Audny
AU - Antoni, Anna
AU - Audibert, Gérard
AU - Azouvi, Philippe
AU - Azzolini, Maria Luisa
AU - Bartels, Ronald
AU - Barzó, Pál
AU - Beauvais, Romuald
AU - Beer, Ronny
AU - Bellander, Bo Michael
AU - Belli, Antonio
AU - Benali, Habib
AU - Berardino, Maurizio
AU - Beretta, Luigi
AU - Blaabjerg, Morten
AU - Bragge, Peter
AU - Brazinova, Alexandra
AU - Brinck, Vibeke
AU - Brooker, Joanne
AU - Brorsson, Camilla
AU - Buki, Andras
AU - Bullinger, Monika
AU - Cabeleira, Manuel
AU - Caccioppola, Alessio
AU - Calappi, Emiliana
AU - Calvi, Maria Rosa
AU - Cameron, Peter
AU - Lozano, Guillermo Carbayo
AU - Carbonara, Marco
AU - Chevallard, Giorgio
AU - Chieregato, Arturo
AU - Citerio, Giuseppe
AU - Cnossen, Maryse
AU - Co-Burn, Mark
AU - Coles, Jonathan
AU - Cooper, D. Jamie
AU - Correia, Marta
AU - Čović, Amra
AU - Gruen, Russell L.
N1 - Publisher Copyright:
© 2020 The Authors
PY - 2020/6/1
Y1 - 2020/6/1
N2 - Objective: We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury. Study Design and Setting: We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified. Results: In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study. Conclusion: ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations.
AB - Objective: We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury. Study Design and Setting: We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified. Results: In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study. Conclusion: ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations.
KW - Cohort study
KW - Data science
KW - Machine learning
KW - Prediction
KW - Prognosis
KW - Traumatic brain injury
UR - http://www.scopus.com/inward/record.url?scp=85082847791&partnerID=8YFLogxK
U2 - 10.1016/j.jclinepi.2020.03.005
DO - 10.1016/j.jclinepi.2020.03.005
M3 - Article
SN - 0895-4356
VL - 122
SP - 95
EP - 107
JO - Journal of Clinical Epidemiology
JF - Journal of Clinical Epidemiology
ER -