Introduction Clinical decision rules aid clinicians with the management of head injured patients. This study aimed to identify clinical decision rules for children with minor head injury and compare their diagnostic accuracy for detection of intracranial injury (ICI) and injury requiring neurosurgical intervention (NSI).
Methods Relevant studies were identified by an electronic search of key databases. Papers in English were included with a cohort of at least 20 children suffering minor head injury (GCS 13–15). Studies of a decision rule derived to identify patients at risk of ICI or NSI had to include a proportion of the cohort undergoing imaging. Study quality was assessed using the QUADAS checklist.
Results 16 publications, representing 14 cohorts, with 79 740 patients were included. Only four rules were tested in more than one cohort. Of the validated rules the paediatric emergency care applied research network (PECARN) rule was most consistent (sensitivity 98%; specificity 58%). For neurosurgical injury all had high sensitivity (98–100%) but the children's head injury algorithm for the prediction of important clinical events (CHALICE) rule had the highest specificity (86%) in its derivation cohort.
Conclusion Of the current decision rules for minor head injury the PECARN rule appears the best for children and infants, with the largest cohort, highest sensitivity and acceptable specificity for clinically significant ICI. Application of this rule in the UK would probably result in an unacceptably high rate of CT scans per injury, and continued use of the CHALICE-based NICE guidelines represents an appropriate alternative.
Statistics from Altmetric.com
Head injury accounts for approximately 700 000 emergency department (ED) attendances every year in the UK, with 40–50% of these being children.1 An estimated 20% of these are admitted to hospital. There is an increasing tendency to perform early diagnostic imaging as this is linked to improved outcomes and a reduction in admission rates. CT of the head is the diagnostic standard for identifying intracranial injury (ICI). Routine CT of all minor head injury patients would result in a large number of normal CT scans being performed with associated risks of radiation exposure and waste of healthcare resources. Researchers have therefore attempted to derive clinical decision rules to identify those at risk of ICI based on clinical characteristics at presentation in order to select them for imaging.
What is already known on this topic
▶ Head injuries account for approximately 5% of ED attendances with a large proportion of these being children.
▶ Clinical decision rules aid in identifying high-risk patients following minor head injury and in the UK, the current NICE guidelines for children are based on the children's head injury algorithm for the prediction of important clinical events study data.
What this study adds
▶ Comparing the diagnostic accuracy of existing decision rules has identified what appears to be the best for this population.
▶ Significant differences exist in criteria used to test these rules and their future development should focus on using standard definitions that would allow more direct comparisons between rules.
There are many clinical decision rules for adults with minor head injury,1,–,4 but few that have been validated for the paediatric population. It is also unclear how they compare in terms of diagnostic accuracy. This study aims systematically to identify clinical decision rules for children with minor head injury and compare these rules in terms of diagnostic accuracy for any ICI and injury requiring neurosurgery.
Clinical question: In the paediatric population presenting to the ED following a minor head injury, which of the published clinical decision rules is the most accurate at predicting which patients will have either an ICI on CT or require neurosurgical intervention?
As this project involved no human subjects and remained a work of evidence synthesis no formal ethical approval was necessary in line with standard practice.
Potentially relevant studies were identified through electronic searches of key databases (from inception) including MEDLINE, EMBASE, CINAHL and CENTRAL up to April 2009, supplemented with an update from MEDLINE to March 2010. Sensitive keyword strategies using free text and, where available, thesaurus terms using Boolean operators and database-specific syntax were developed to search the electronic databases. Synonyms relating to the condition (eg, head injury) were combined with a search filter aimed at restricting results to diagnostic accuracy studies (used in the searches of MEDLINE, CINAHL and EMBASE). Language restrictions were not used on any database. To identify additional published, unpublished and ongoing studies, the reference lists of all relevant studies (including existing systematic reviews) were checked and a citation search of relevant articles (using the Web of Knowledge's Science Citation Index and Social Science Citation Index) was undertaken to identify articles that cite the relevant articles. In addition, systematic keyword searches of the worldwide web (www) were undertaken using the Copernic Agent Basic (version 6.12) meta-search engine and key experts in the field were contacted. Further details on the search strategies are available from the authors or our full report to the Health Technology Assessment Group.5
Studies were included if they met the following criteria: (1) a cohort study of children with minimum 20 patients and at least half had a Glasgow coma scale score of 13–15 at presentation; (2) they evaluated a decision rule that used three or more clinical criteria (such as history, physical examination or a simple diagnostic test) to determine the risk of any ICI (defined as any intracranial abnormality detected on CT or MRI scan due to trauma) or injury requiring neurosurgery (defined as any ICI seen on CT or MRI that required neurosurgery); (3) some or all of the patients were assessed with CT scan or MRI for ICI, or followed up to determine the need for neurosurgery and (4) provided data that allowed true positive, true negative, false positive and false negative numbers to be extracted or calculated.
Articles were considered for inclusion in three stages (figure 1). First, titles were sifted to exclude obviously irrelevant articles (APa and SH). Second, abstracts of the remaining studies were split between two teams of reviewers (APa and APi, SH and SG) and assessed for relevance to produce a list of potentially relevant articles. Finally, all studies on this list were obtained and studied in detail to generate a final list of included studies (SH and APa, checked by SG and APi). When discrepancies between reviewers occurred, these were resolved through discussion.
Assessment of methodological quality
The methodological quality of each included study was assessed by one reviewer (SH) and checked by another (APa) using a modified version (appendix 1) of the quality assessment of diagnostic accuracy studies tool6 (a generic, validated, quality assessment instrument for diagnostic accuracy studies) (figure 2). In case of doubt, a third and fourth reviewer (SG and APi) were consulted.
Data were extracted by one reviewer (SH) and checked by a second (APa). Only articles written in English were data extracted. Variables relating to study design, patient characteristics, study quality and diagnostic accuracy were extracted. When discrepancies occurred, these were resolved through discussion. When differences were unresolved, a third reviewer's opinion was sought (SG or APi). When a study presented several different versions of a rule produced during derivation, all versions were extracted but only one version of the rule (either that endorsed by the authors, or that with the most appropriate outcome definition) was considered in further analysis.
A meta-analysis of any rule that had been independently validated in multiple relatively homogeneous cohorts was planned (figures 3–5). During data extraction it became clear that no rule had been evaluated to this extent. Instead, a narrative synthesis was performed, comparing diagnostic accuracy estimates across cohorts to determine which (if any) of the decision rules were supported by consistent estimates of sensitivity and specificity in multiple validation cohorts. The reported statistics from each study are visually represented in figures 3–5.
We identified over 8000 titles relevant to minor head injury but only included 222 for full text review. From these we selected 16 publications representing 14 cohorts and 79 740 patients.7,–,22 Eight cohorts were from the USA,8 10 12 15 17 18 20 22 one from the USA and Canada,7 and one from each of Italy,9 UK,11 Turkey,14 Finland16 and Canada.19 Seven studies used data from multiple centres,7 9 11 17,–,19 22 of which two were very large with cohorts over 20 000.11 17 The smallest cohort was 97.8
Inclusion criteria varied greatly between cohorts. For studies of children, the upper age limit ranged between 16 and 21 years, and the lower limit between 0 and 5 years. For infants, the upper age limit was usually 2 years, but in one case it was 3 years of age. Eight cohorts included all severity of head injury,9,–,12 16 18 20 21 six recruited those with minor head injury,7 8 14 15 17 19 and two reported on a minor head injury subset.13 20 Exclusion criteria also varied, with five studies excluding those with trivial head injury and/or recruiting only those with clinical characteristics consistent with head trauma.15 17 19,–,21
The selection of patients on the basis of having had a CT scan and exclusion on the basis of trivial injury or not presenting with clinical characteristics is likely to recruit a patient spectrum with a greater risk of ICI. Six studies only included those who had a CT scan7 10 14 15 18 21 and three reported a subset in which all had CT.12 13 20 This explains some of the heterogeneity in estimates of ICI and neurosurgical injury rates. The median value for the prevalence of neurosurgery was 1.2% (IQR 0.2–1.4%). The median value for the prevalence of ICI was 6.5% (IQR 1.0–9.8%).
Definitions of outcomes and the reference standards used varied across studies (table 1). The main difference in outcome definition for ICI was the perception of clinical significance, with five studies defining this and the remaining 11 just identifying any common acute lesion (listed in table 1). Definitions of surgical interventions also varied (when reported), but most included haematoma evacuation and intracranial pressure monitoring, with only one mentioning elevation of skull fracture explicitly.
Using the quality assessment of diagnostic accuracy studies tool we identified three questions for which more than 25% of studies received a poor rating (figure 2). These related to two main areas. The first area was patient selection and the representative nature of the subjects included. Nearly all studies received a poor rating because of a small degree of selection for inclusion and exclusion criteria. The studies in question often excluded trivial or minimal injury and included subjects with symptoms following their injury. While this represents clinical practice it can introduce a degree of selection bias for the diagnostic test in question.
The second area was the use and description of the reference standards employed. The reference standards used when CT was not possible for all, and was not an inclusion criterion, usually comprised telephone follow-up, hospital records checks, or both. This reference standard method is likely to identify clinically significant lesions, particularly those requiring neurosurgery, but would not be expected to identify all ICI accurately—potentially affecting the diagnostic accuracy for this outcome. Higher quality studies include Atabaki et al7 and Greenes and Schutzman,12 13 while the children's head injury algorithm for the prediction of important clinical events (CHALICE) rule11 scores poorly.
Only four rules were tested in more than one cohort (figure 3). The paediatric emergency care applied research network (PECARN)16 rule for children (≥2 to <18 years) and infants (<2 years) comprised derivation and validation cohorts, both of which were performed in the same setting, by the same group and reported in the same article. Sensitivity (97%) and specificity (58%) were consistent and within narrow CI as a result of the large patient cohort. Three other rules were re-tested in a small retrospective case note review, involving 485 patients, based on their discharge diagnosis.16 Only 242 (49.9%) of these received a CT scan, based on clinical examination, and it can be considered as a methodologically poor validation for these decision rules as the reference standard was not uniformly applied. A modified version of the University of California–Davis rule (in which ‘headache’ and ‘vomiting’ were redefined as ‘severe headache’ and ‘severe vomiting’) was tested in a separate cohort of data collected for the National Emergency X-Radiography Utilization Study II,22 with a significantly lower sensitivity (91%).
Nine other rules were identified (table 1), three of which7 9 19 show promising diagnostic accuracy with an associated high specificity that could translate to a significant reduction in CT use. Validation is required, however.
For infants, seven studies8 12 13 17 18 20 22 were identified with results for ICI (figure 4) but only two rules have been tested in more than one cohort,17 20 with the largest of these (PECARN) giving the best results. The CHALICE rule includes this age group but does not report them separately.11 All of these rules require further validation by application in other settings before conclusions can be drawn.
Six rules provide results for the prediction of the need for neurosurgery (figure 5).6 10 14 16 18 19 All have very good sensitivity (approximately 100%) with the chalice rule also reporting high specificity. Despite large cohorts (>20 000 subjects) for two of these studies, the prevalence of neurosurgical injury varies greatly ranging from 0.11% to 3.4%, highlighting the existing heterogeneity.
The positive predictive value shows what proportion of scans identify an ICI (or neurosurgical intervention) with each rule. Therfore 2.0% of patients receiving scans according to the PECARN rule will show ICI, compared with 5.4% of patients receiving scans to the CHALICE rule (see table 2). There is substantial variation in methodology that limits our ability to draw comparisons. However, if it is assumed that in practice patients excluded from the study would not undergo any CT scanning then the positive predictive values are likely to be reasonably comparable across the cohorts.
This review has identified 14 rules with derivation and validation data for both infants and children following minor head injury. The most validated appears to be the PECARN rule. Clinical decision rules for children following minor head injury have increased in number from the eight identified in the 2008 review by Maguire et al.23 Their conclusion that more work was needed has been accepted and more of the existing rules are being validated and applied in different patient cohorts around the world.
More recent work in both adults and children has used predefined clinically significant injury instead of the older approach of identifying any injury on CT.5 This provides a more pragmatic solution to the ethical issues around unnecessary radiation exposure in susceptible children for the purposes of research and was adopted by the two largest cohorts analysed for this review (listed in table 1).11 17 One argument put forward for this approach is that using patient-centred outcomes overcomes the imperfect sensitivity and specificity of CT scans and allows minor or incidental findings to be ignored.
Relevant to the UK, the CHALICE-derived National Institute for Health and Clinical Excellence (NICE) criteria for CT scan in children under 16 years have not been validated in any population but form the basis for management decisions in 85% of ED.24 Estimates of predicted CT rates among practitioners in the UK following the NICE guidelines (2003) for children are approximately 12%.25 26 This is considerably higher than quoted baseline CT rates, before NICE implementation, which were approximately 3%.11 25 27 A more recent estimate from Australia looking at predicted CT scan rates following the CHALICE rule (the basis for NICE 2007) was 46%, although this was a retrospective case notes review.28 The PECARN and CHALICE rules (with cohorts over 20 000) have quoted expected scan rates of approximately 42% and 14%, respectively. However, 28% of the PECARN population was classed as moderate risk (0.9% risk of clinically important traumatic brain injury) and the authors recommended practitioner discretion for scanning this group citing observation as the alternative course of action.
For identifying one clinically significant ICI (see table 1), using the positive predictive values from the data, PECARN would scan approximately 50 children. CHALICE would scan 18 (see table 2). For identifying one neurosurgical injury PECARN would scan over 200 children while CHALICE would scan 24 (see table 2). A formal cost-effectiveness analysis is reported elsewhere,5 but these figures demonstrate a more refined approach to risk management in the CHALICE group, more in keeping with UK NHS-based practice.
The heterogeneity of these rules prevented any meta-analysis of the data and restricted our summary to a narrative synthesis. This highlights the inconsistencies involved in head injury research and the difficulties faced by clinicians in interpreting any published results. There remains a lack of robust validation for these decision rules and the shift from identifying any lesion on CT to focusing on clinically significant lesions has made results more difficult to compare for researchers. This shift also does not appear to take into account longer-term sequelae, which are beyond the scope of our review.
Future research efforts in this field should concentrate on the universal application of definitions for patient populations, inclusion criteria, reference standards and outcome criteria. This would enable easier comparison and further validation of the existing decision rules. Work should also focus on prognostic studies for those children with non-surgical injury identifiable on CT, and cost-effectiveness analysis of the adoption of different decision rules specifically in the paediatric population.
Of the currently published decision rules, methodologically the PECARN rule appears to be the best validated rule for both children and infants, with the largest study cohort, highest sensitivity and acceptable specificity for clinically significant ICI. However, application of this rule in the UK would probably result in an unacceptably high rate of CT scans for every injury identified, and continued use of the CHALICE-based NICE guidelines represents an appropriate alternative. Further validation in new cohorts is required to confirm our analysis and compare the PECARN rule directly with other rules.
The authors would like to thank the following people for their help with this project: Angie Rees for her help with the search strategy, Professor Alex Sutton, Department of Health Sciences, University of Leicester, for providing statistical advice and Joanne Turner for clerical assistance.
Funding This project was funded by the NETSCC Health Technology Assessment programme (ref 07/37/08) as part of a larger review of management strategies for minor head injuries.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.