Prognostic Models of Mortality from Accidental Severe Traumatic Brain Injury in Children and Adolescents: A Systematic Review

Available http://www.archpedneurosurg.com.br/ Introduction: Prognostic models are statistical models that combine two or more items of patient data to predict clinical outcomes. Objective : Identify prognostic models of mortality developed and published in the medical literature for possible applicability in children and adolescents victims of severe traumatic brain injury (TBI). Methods: Systematic review in the Medline electronic database (PubMed platform) of scientific articles published from 2006 (year of publication of the last systematic review on prognostic models for TBI before 2017) until July 29, 2017. Results: Ten studies on prognostic models of mortality in children and adolescents victims of severe TBI were identified for final inclusion in the review. There were eight development and two validation studies conducted in different countries. Conclusion: The analysis of this systematic review makes it possible to conclude that the ten prognostic models included in the final sample provide health professionals with a scientific evidence-based understanding of the severity of pediatric victims of severe TBI. This systematic review is classified as presenting 2A and 1 level of evidence (systematic review of homogeneous cohorts), according to the 2009 and 2011 classifications, respectively, of the Oxford Center for Evidence-Based Medicine.


INTRODUCTION
Children and adolescents comprise an important group affected by accidental mechanical trauma. Victims with admission Glasgow Coma Scale (GCS) scores < 8 are defined as suffering from severe traumatic brain injury (TBI) and must be intubated, have a cervical collar placed, and have adequate venous access ensured. Proper (pre-hospital) management of these victims from the trauma set to prevent secondary brain injuries and the recognition of factors associated with a worse prognosis and greater risk of death are essential in these patients [1][2][3][4][5][6][7][8][9][10][11].
Prognostic models are statistical models that combine two or more items of patient data to predict clinical outcomes. The aim of this systematic review is to identify prognostic models of mortality (death outcome) developed and published in the medical literature for possible applicability in children and adolescents victims of severe TBI.

Study design, information sources and search
Systematic review in the Medline electronic database (PubMed platform) of scientific articles published from 2006 (year of publication of the last systematic review on prognostic models for TBI before 2017) until July 29, 2017. The electronic search strategy was designed to maximize sensitivity and specificity, following validated recommendations for search strategies aimed at publications of prognostic models [12] in the Medline database. The terms used in the search were related to the population analyzed (children and adolescents), the situation concerned (severe TBI), and the objectives of the studies that are of interest for the present review (prognostic models of mortality). The bibliographic references of included articles were searched for potential relevant publications not found in the initial electronic search.

Selection of studies for definition of prognostic models
In the first-level screening, both authors read the titles of all citations identified by the electronic search and eliminated those that were not related to clinical prognostic models of severe TBI in children and adolescents. The second-level screening involved reviewing the abstracts. In this instance, full articles were retrieved. Kappa statistics with 95% confidence intervals were used to test the degree of agreement on study eligibility.
The study was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [13], and the specificities of the scientific question were framed according to the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist [14]. The protocol of the current systematic review was registered in the International Prospective Register of Systematic Reviews (PROSPERO) [15] (registration number 76364). reports, case series, qualitative studies, studies that did not report methods, cadaveric studies; • Study population: adult humans, animals, studies that did not separate children and adolescents vs. adults in the analysis, and studies that did not perform analysis according to the different degrees of TBI (mild, moderate, and severe); • Case definitions: studies that primarily focused on nonaccidental brain injuries (abuse), firearm injury, or other penetrating brain injury; • Study outcome: any non-death outcome (e.g., outcome GCS score, post-traumatic neurological deficits).

Data collection, listing, and risk of bias
After data extraction, the risk of bias of each domain according to CHARMS [14] of all selected studies was scored as "low", "moderate" or "high". The design of each study was classified as to the level of evidence according to the classification proposed by the Oxford Center for Evidence-Based Medicine (OCEBM) [16,17].

RESULTS
Five hundred and thirteen articles were initially listed; 273 articles were excluded after reading the titles, 198 after reading the abstracts and application of eligibility criteria, and 38 after reading the full text, for not meeting the defined eligibility criteria. Ten studies on prognostic models of mortality in children and adolescents victims of severe TBI were identified for final inclusion in the systematic review ( Figure 1 Original prediction models were developed in 08 studies, while 02 studies validated previously developed models. Only one study used a prospective cohort and was considered to have a low risk of bias. Six studies described the guidelines used in the treatment of the included patients and were classified as having a low risk of bias; four studies did not report how patients were treated and were classified as having a high risk of bias in this item. Only one of the studies did not explain how the predictors were measured and was classified as having a high risk of bias for this domain (Table 1). One study showed that the measurement of the most important predictor of the analysis was blind to the other predictors and to the outcome, being rated as having a low risk of bias; none of the other studies described this method and were classified as having a moderate risk of bias. Only one study reported not having categorized or dichotomized the continuous predictors and was classified as having a low risk of bias; one study reported having done this procedure and was classified as presenting a moderate risk of bias; and the other eight studies did not report whether they did or not dichotomized continuous variables, also rated as presenting a moderate risk of bias.
Seven studies did not report whether there were missing data, and one study had a number of missing data greater than 10% of the final population included in its analyzed sample; these eight studies were classified as presenting a high risk of bias. One study reported missing data, which corresponded to 5 to 10% of the final population, thus presenting a moderate risk of bias, and only one study was classified as having a low risk of bias in this category (low sample loss, < 5%) ( Table 1).
As for the statistical analysis, all studies were rated as having a low risk of bias, considering that multivariate regression was the method adopted for the model (Table 1). Three studies did not report how predictors were chosen to be included in the model, being rated as having a high risk of bias; three studies reported the method, but had a high event-to-variable (ETV) ratio or dichotomized continuous variables, thus being rated as having a moderate risk of bias; three studies reported the method and had a high ETV ratio and were rated as having a low risk of bias. Two studies did not report how the variables were chosen during multivariate modeling and were assessed as having a high risk of bias (all other studies reported some method and were rated as having a low risk of bias). Six studies were rated as presenting a low risk of bias because they had a high ETV ratio; three studies had a high risk of bias because they did not have a high ETV ratio and did not perform or report weight adjustments of predictors.
Four studies reported the accuracy of the analyzed model, being rated as having a low risk of bias; the other six studies did not report the accuracy of the models studied and were rated as having a moderate risk of bias. In the evaluation axis, the method used to test the model's performance and whether the model was adjusted after validation were the aspects analyzed. Only one study presented a low risk of bias because it consisted of an external validation. One study divided its case series to create and validate the model, and another study reevaluated the dataset itself; both were rated as having a moderate risk of bias. The other studies that did not report validation methods were assessed as presenting a high risk of bias.
Age was the most common predictor in all models, appearing in eight of them. GCS score was the second most common predictor, mentioned in seven studies; five models used only the motor score in the GCS. All studies included Concerning the predictive performance, model discrimination was evaluated as the area under the receiver operating curve (AUROC) in three studies, with results ranging from 0.91 to 0.989, while five development studies did not report predictive performance using c-statistic (AUROC). None of these studies presented the confidence interval (CI) in the report of the discriminatory power. The external validation study that did not update the predictors evaluated five models concomitantly, which had AUROC values between 0.85 and 0.92. The study that externally validated a model and updated the variables in the final model presented an AUROC value of 0.91, with a CI between 0.84 and 0.98. Only three validation and development studies presented the accuracy of the studied models, doing it in different ways: through the standardized mortality ratio; through the Brier's score; and through the standard accuracy percentage.
No study included in this systematic review developed and externally validated a model in a single publication. One of the external validation studies had a lower performance than that obtained in the development dataset and was updated for the analyzed sample, incorporating new predictors into the final model. The calibration of the models was presented in five studies, and the Hosmer-Lemeshow X² test was the most frequently used to that end; other methods included Cox regression and adjusted R². All models presented good performance based on significance (p value) results in the Hosmer-Lemeshow test, high adjusted R², and the Cox regression with intercept touching zero and slope touching one.

DISCUSSION
Eight development studies [3,4,[6][7][8][9][10][11] and two validation studies [2,5] were found in the present systematic review of prognostic models of mortality from severe TBI in children and adolescents. They were conducted in different continents and countries, namely, England, Canada, the United States, Iran, France, and Tunisia.
The individual discussion of the studies showed that the study by Young et al. 2016 [2] was one of the two external validation studies found in the review. Its level of evidence can be classified as 2B according to OCEBM guidelines. It presented a low risk of bias in half of the domains analyzed, and moderate and high risk of bias in 25% of the domains. The study analyzed five models in the same population with varied difficulty of predictor measurement: the simplest predictor was age (present in all models) and the most complex was cranial tomography, requiring expert analysis. The study identified a good performance of the five models in predicting mortality in victims of severe TBI.
The second validation study was the one by Liesemer et al. 2014 [5], and it was the only one that updated and added other predictors. The study obtained the best score regarding bias, with low, moderate, and high risk of bias in 53%, 37%, and 10%, respectively, of the analyzed domains. The study provided two multivariate models, one simpler, with only two predictors (one clinical and one radiological), and another more complex, with four predictors (the same as those of the simpler model plus two trauma-related predictors). The study showed that the more complex model performed better than the simpler one (the complex model corrected some bias regarding mortality) and still performed well [5].
The study by Alali et al. 2015 [3] presented low, moderate, and high risk of bias in 53%, 21%, and 26%, respectively, of the analyzed domains. The authors used clinical criteria related to trauma to derive a model that showed high discriminatory capacity. The study did not present the calibration and accuracy of the model. The study by Hochstadter et al. 2014 [4], which presented low, moderate, and high risk of bias in 45%, 33%, and 22%, respectively, of the analyzed domains, used clinical and laboratory predictors and was the one that obtained the best performance among the models analyzed in this review.
As for the study by Rahimi et al. 2014 [6], the risk of bias assessment showed that 33% of the domains had a low risk, 33% moderate risk, and 33% high risk. The authors used two predictors (clinical and hemogasometric) to derive a model but did not present its discriminative capacity and accuracy. The study by Alharfi et al. 2013 [7] showed low, moderate, and high risk of bias in 32%, 47%, and 21%, respectively, of the considered domains. The authors used clinical and laboratory criteria to derive a model, not presenting, however, its discriminative capacity and accuracy.
Two studies by Melo et al., 2009Melo et al., , 2010 were included in this systematic review. The first [8] had a low, moderate, and high risk of bias in 50%, 39%, and 11%, respectively, of the analyzed domains. The authors used clinical and laboratory criteria to derive a model that showed high discriminatory capacity, however, calibration and accuracy were not presented. This study was the only one that presented the model in an alternative way, of the summed-score type. The second study [9] showed low, moderate, and high risk of bias in 28%, 44%, and 28%, respectively, of the analyzed domains. The authors used only three clinical and laboratory predictors (present in the full study published in 2010) to derive a model.
The study by Bahloul et al. 2009 [10] showed low, moderate, and high risk with bias in 50%, 33%, and 17%, respectively, of the analyzed domains. The authors used two predictors (a clinical predictor and a pre-existing mortality score) to derive a prognostic model, but did not present any value regarding performance. Finally, the last study included in this review was the one carried out by Ducrocq et al. 2006 [11], in which 42%, 42%, and 16% of the analyzed domains presented a low, moderate, and high risk of bias, respectively. The study included seven predictors (one demographic predictor, two pre-existing scores, one radiological predictor, two clinical predictors, and one predictor related to trauma). The study did not show any value as to performance.
The assessment of the risk of bias of the axes analyzed using CHARMS [14] showed some points that studies could have presented, planned, or calculated more rigorously. All of these observations lead to an increased risk of overfitting of the prognostic models. Overfitting happens when the developed model is too close to the dataset that was used to create it, causing it to have poor performance when applied to another population. In fact, reproducibility is the basis of a good quality model. The risk of overfitting also increases from the fact that the most common criterion for inclusion of predictors in the model creation process was univariate association. Although a strong association with the outcome sometimes exists, this method generally includes predictors with a spurious association with the outcome in the final model. Additionally, the management of missing (lost) data is an important source of bias in longitudinal studies, as is the case of those that propose to evaluate prognostic models.
Only one study used multiple imputations as method of choice for handling missing data [18]. Furthermore, some studies dichotomized the continuous variables, which is an inadequate practice that discards information and generally leads to loss of power [19]. For better applicability, prognostic models need external validation to prove useful outside the dataset that produced it [20]. Most often developed prognostic models that have not undergone external validation are likely to perform beyond reality, which tends to underestimation of small risks and overestimation of high risks [14]. Thus, all development studies in this review were rated as having a high risk of bias in the axis of evaluation of the model, as they generally show unadjusted performance measures. Systematic reviews have certain limitations inherent to their design; even when the included studies present primary data, there is no control over their quality of execution and elaboration [14]. This risk was reduced by ensuring homogeneity among the included studies through application of the eligibility criteria. Another limitation of the present review is the fact that the outcome analyzed is not clear as to the etiology (death directly due to trauma or due to secondary complications). Only one of the included studies [10] showed the frequency of the etiologies of death in the population analyzed. Another point worth noting is that the patients analyzed in the studies possibly did not receive the same management and treatment, considering the peculiarities and protocols of the sites where data collection took place. These differences in patient management, both at the pre-hospital and in-hospital level, can influence death rates in severe TBI victims [8].
Regarding the advantages, this systematic review included only studies in which mortality (death) was the outcome analyzed, excluding many other studies describing post-traumatic neurological sequelae and functional deficits.
To our knowledge, this is the first systematic review of prognostic models of mortality from severe TBI in children and adolescents. Another previous systematic review published by Perel et al. 2006 [21] did not exclusively analyze victims in the pediatric age group. The number of publications on prognostic models is increasing. Overviews of the quality and characteristics of the projected models are essential. Another important strong point of this review is that it was based on validated methodologies [22] for the search of prognostic modeling studies in databases, and the CHARMS guidelines [14] were used to perform the risk of bias analysis and data extraction. Finally, this systematic review is classified as presenting 2A and 1 level of evidence (systematic review of homogeneous cohorts), according to the 2009 and 2011 classifications, respectively, of the OCEBM [16,17].

CONCLUSION
Ten studies of prognostic models of mortality from severe TBI in children and adolescents were identified in the proposal of this systematic review. Most studies showed moderate to high methodological quality. It is important that development and validation prognostic studies follow a rigorous methodology in order to obtain higher quality findings and propositions. The analysis of this systematic review allows us to conclude that the ten prognostic models included in the final sample provide health professionals with a scientific evidence-based understanding of the severity of pediatric victims of severe TBI and of the need for surveillance of predictive factors associated with better or worse outcomes, considering that some of them are modifiable risk factors.