Derivation of the children’s head injury algorithm for the prediction of important clinical events decision rule for head injury in children
- J Dunning1,
- J Patrick Daly2,
- J-P Lomas1,
- F Lecky3,
- J Batchelor1,
- K Mackway-Jones1,
- on behalf of the children’s head injury algorithm for the prediction of important clinical events (CHALICE) study group
- 1Emergency Medicine Research Group (EMERGE), Emergency Department, Manchester Royal Infirmary, Manchester, UK
- 2Manchester Computing, University of Manchester, Manchester
- 3Emergency Department, Hope Hospital, Manchester
- Correspondence to:
Emergency Medicine Research Group, Emergency Department, Manchester Royal Infirmary, Oxford Road, Manchester M13 9WL, UK;
- Accepted 12 June 2006
Background: A quarter of all patients presenting to emergency departments are children. Although there are several large, well-conducted studies on adults enabling accurate selection of patients with head injury at high risk for computed tomography scanning, no such study has derived a rule for children.
Aim: To conduct a prospective multicentre diagnostic cohort study to provide a rule for selection of high-risk children with head injury for computed tomography scanning.
Design: All children presenting to the emergency departments of 10 hospitals in the northwest of England with any severity of head injury were recruited. A tailor-made proforma was used to collect data on around 40 clinical variables for each child. These variables were defined from a literature review, and a pilot study was conducted before the children’s head injury algorithm for the prediction of important clinical events (CHALICE) study. All children who had a clinically significant head injury (death, need for neurosurgical intervention or abnormality on a computed tomography scan) were identified. Recursive partitioning was used to create a highly sensitive rule for the prediction of significant intracranial pathology.
Results: 22 772 children were recruited over 2½ years. 65% of these were boys and 56% were <5 years old. 281 children showed an abnormality on the computed tomography scan, 137 had a neurosurgical operation and 15 died. The CHALICE rule was derived with a sensitivity of 98% (95% confidence interval (CI) 96% to 100%) and a specificity of 87% (95% CI 86% to 87%) for the prediction of clinically significant head injury, and requires a computed tomography scan rate of 14%.
Conclusion: A highly sensitive clinical decision rule is derived for the identification of children who should undergo computed tomography scanning after head injury. This rule has the potential to improve and standardise the care of children presenting with head injuries. Validation of this rule in new cohorts of patients should now be undertaken.
- CHALICE, children’s head injury algorithm for the prediction of important clinical events
- LOC, loss of consciousness
- RCS, Royal College of Surgeons
- SXR, skull radiograph
One million patients with head injuries attend emergency departments each year in the UK, of whom as many as 50% are children1–3; this proportion is similar in the US, where there are 95 000 hospital admissions from childhood head injuries, at a cost of over US$ 1 billion per year.4–6 In contrast with the high incidence of head injury, mortality is comparatively low (6–10 per 100 000), and as few as 1 in 500 of all people attending the emergency department have a fatal outcome.7,8 Thus, although emergency physicians see a large number of patients with head injury, they rarely see patients who have life-threatening intracranial complications after the injury.
Over the past decade, several decision rules have been derived and validated using high-quality methods to identify adults with a head injury who require computed tomography scanning.9–14 Although children account for as many as half of all head injuries, no such well-derived multicentre clinical decision rules exist for children. The American Academy of Pediatrics in 199914a concluded that they could not advocate an evidence-based computed tomography scanning strategy because of the poor quality of studies on children.15 In 2003, The National Institute of Clinical Excellence in the UK found that the quality of studies on childhood head injuries was so poor that they issued a clinical decision rule for children that was derived and validated only in adults.16
Our aim was to derive a sensitive clinical decision rule for the management of children presenting with an acute head injury, which would identify children at high risk so as to undergo computed tomography scanning and allow the remaining patients to be discharged with no investigation.
Study setting and population
A prospective diagnostic cohort study was undertaken from February 2000 to August 2002, which aimed to recruit all patients <16 years presenting with head injury, who attended the emergency departments of 10 hospitals in the northwest of England. Three of these hospitals were children’s hospitals, three were teaching hospitals and four were district general hospitals.
Clinical symptoms and signs in young children can be unreliable, and therefore there is no universally agreed category of “trivial” head injury for which there is no risk of a major intracranial complication. Our inclusion criteria for this study were therefore as wide as possible. Any patient with a history or signs of injury to the head was eligible for inclusion into the study. In particular, loss of consciousness (LOC) or amnesia was not a requirement for entry. We also wanted to reflect the whole population with head injuries attending emergency departments and thus did not exclude patients with a head injury that may have been defined as “moderate or severe”. The only exclusion criterion was refusal to consent to entry into the study.
Standardised patient assessment
A specifically designed proforma was created for data collection. This proforma collected data on around 40 clinical variables pertaining to head injury, including variables on the mechanism of injury, symptoms, signs and management of the patient. Every doctor who participated in the study was given a 1-h training session on the study and the use of this proforma for data collection. Primary assessment of all patients who were eligible for our study was conducted using this proforma. This proforma also functioned as the patient’s clinical record on admission.
Response rates and quality of completion were monitored in all centres on a monthly basis to ensure high compliance. All doctors were asked to follow the 1999 Royal College of Surgeons (RCS) guidelines for the management of head injuries,17 and the guidelines were printed clearly on the front of every proforma.
The primary outcome measure was a composite comprising death as a result of head injury, requirement for neurosurgical intervention or marked abnormalities on the computed tomography scan (together referred to as “clinically significant intracranial injury”). In contrast with studies on adults where consensus has been reached as to the nature of what constitutes a clinically important brain injury, no consensus has been reached regarding children in studies either in the US or in the UK.9,16 We defined the computed tomography outcome measure as any new, acute, traumatic intracranial pathology as reported by the consultant radiologist, including intracranial haematomas of any size, cerebral contusion, diffuse cerebral oedema and depressed skull fractures. Simple or non-depressed skull fractures alone were not considered to be significant. Secondary outcome measures were the presence of a skull fracture or admission to hospital.
All patients who were documented as having had a skull radiograph, admission to hospital, computed tomography scanning or neurosurgery were followed up. Radiology departments across the 10 hospitals in the study and also from two further tertiary neurosurgical referral centres in the northwest of England (The Walton Centre, Liverpool, and Pendlebury Hospital, Manchester) collated data separately on every child who had a skull radiograph or computed tomography scan of the brain. In addition, hospitals prospectively collated data on patients who were admitted, underwent neurosurgery, stay in the intensive care unit or neurorehabilitation from these 12 centres. These data were then cross-checked with those in the database of the children’s head injury alorithm for the prediction of important clinical events (CHALICE). The Office of National Statistics provided us with the details of children in the UK who died, in whom head injury was any part of the cause of death.
Multicentre ethical approval was obtained for this study. Verbal consent to participate in this study was obtained from all patients or guardians before entry into the study. The doctor who obtained consent indicated acceptance on the proforma.
Sample size that would allow us to derive, with an 80% power, a clinical decision rule with 100% sensitivity and a lower confidence limit >95% was initially calculated. This required 75 patients with a positive outcome; thus, at an incidence of 1% for clinically significant brain injury, we required 7500 patients and a 1-year study. At interim analysis, patients who were difficult to predict were identified and thus the study was extended to keep the lower confidence interval >95%.
Univariate analysis was carried out using Fisher’s exact test for binomial categorical data or the χ2 test for unranked categorical data. Non-parametric rank data or continuous data were analysed using the Mann–Whitney U test, and unpaired Student’s t test was used for continuous data with a normal distribution. Interobserver agreement for each variable was calculated using the κ coefficient—that is, the proportion of potential agreement beyond chance—along with its 95% confidence interval (CI) in a subset of patients. In this subset from two centres, patients were seen twice by clinicians in the study and their results were compared. A weighted κ was calculated for rank variables.18,19
Variables that were both reproducible (κ>0.6) and associated with the outcome measure (p<0.1) were assessed by multivariate analysis using recursive partitioning. The primary objective of the multivariate analysis was to find the best combinations of predictor variables that were highly sensitive for detecting the presence of a clinically significant intracranial injury, while achieving the maximum possible specificity.
Our experience and that of other groups9,20 suggest that recursive partitioning may be more suitable than logistic regression when the objective is to correctly classify one outcome at the expense of the other—that is, when high sensitivity is more important than specificity.
Univariate analysis was carried out using SPSS V.11.5 and recursive partitioning using CART V.4.0 (Salford Systems, San Diego, California, USA), using the GINI splitting rule with a 3 to 1 weighting against misclassification of positive intracranial pathology.
In all, 22 772 patients were enrolled into the study, of whom 12 471 were from children’s hospitals, 3241 from teaching hospitals and 7060 from district general hospitals. In all, 65% of these were boys and 56% were <5 years old (table 1). The mean age was 5.7 years. Out of 744 computed tomography scans, scans of 281 (1.2%) patients showed an abnormality (3.2% of the cohort, of which 37.7% showed abnormality). In all, 1461 (6.4%) children were admitted, 137 (0.6%) had a neurosurgical operation and 15 children died.
In all 5318 skull radiographs (SXRs) were taken, from which 259 skull fractures were diagnosed after radiologist reporting (table 2). However, 44 (17%) fractures were missed by emergency physicians and 59 (1% of normal radiographs) fractures diagnosed by emergency physicians were normal, giving a correlation between radiologists and emergency physicians of κ = 0.80. Ninety eight patients with abnormality on computed tomography scan also had an SXR. SXRs reported by emergency physicians had a sensitivity of 77% (95% CI 67% to 85%, 75 fractures in 98 patients) for the prediction of positive pathology on computed tomography scanning.
Doctors were asked to use the 1999 RCS guidelines for the management of head injuries. These guidelines recommend all patients at high risk to be admitted, with computed tomography scanning for those with the highest risk. This protocol resulted in 1461 admissions, but 10 patients were sent home before returning and proving to have an abnormality on the computed tomography scan. Also, another two patients were admitted for a brief time and sent home without being scanned, only to return with intracranial pathology shown on the subsequent computed tomography scan. Of these 12 patients, seven required neurosurgery. Two patients with intracranial pathology who were discharged presented to a different hospital on their second attendance. Of patients who were admitted for observation without immediate computed tomography scanning, 27 patients deteriorated on the ward and 24 of these patients required neurosurgery. Table 2 gives the full details of radiographic and admission demographics.
Univariate analysis was carried out using the primary outcome measure. Table 3 gives the univariate analysis of those variables selected in the final model. The full analysis of the association between all clinical variables and primary outcome is available online at http://www.archdischild.com/supplemental. All variables that showed a univariate relationship with p<0.1 were entered into multivariate analysis.
The common clinical variables with unknown reproducibility were analysed in a subset of the CHALICE study. In all, 412 patients had their clinical condition assessed in this way. Good agreement was found for LOC (κ = 1, 95% CI 0.84 to 1), amnesia (κ = 0.93, 95% CI 0.79 to 1) and vomiting (κ = 0.94, 95% CI 0.81 to 1), but headache showed poor correlation (κ = 0.39, 95% CI 0.25 to 0.54) and was rejected from multivariate modelling owing to its poor predictive ability.
Recursive partitioning analysis produced a highly sensitive model, (see box). As shown in the clinical utility analysis (table 4), this model has an overall sensitivity of 98% (95% CI 96% to 100%) and an overall specificity of 87% (95% CI 86% to 87%). Our model misses only four patients, of whom two had depressed skull fractures that the physician did not suspect on examination, but on reattendance subsequent physicians documented that the depressions were easily palpable. The third patient had an unwitnessed fall against a wall. Initial examination found no high-risk variables and the patient was discharged, but the patient returned 2 h later, vomiting. He was admitted overnight, and the scan the next morning showed an epidural haematoma that required neurosurgery. The fourth patient had a fall into a stream from a swing, and had brief amnesia and a moderate headache. He was discharged but returned 11 days later with a persisting headache. A linear skull fracture and a small subdural and epidural haematoma were found that required no treatment.
We have successfully derived a highly sensitive clinical decision rule for the prediction of clinically significant intracranial pathology in children according to strict methodological standards, in the world’s largest prospective cohort of children with head injuries. This rule, if validated, will enable clinicians to request computed tomography scans for their patients on the basis of strong evidence in children.
If our rule is subsequently validated, we believe that patients regarded as high risk should undergo computed tomography scanning to look for intracranial pathology, whereas those with a normal scan might be regarded as low risk. Children who are regarded as low risk by a validated rule should be carefully counselled so that they understand the high-risk symptoms for which they should return. Although the CHALICE rule will increase the rate of computed tomography scanning, we envisage that the admission rate could be markedly reduced and thus the cost implications of our rule could be neutral. This remains to be shown in further studies.
Any decision rule that is to be of value to clinicians seeing undifferentiated children with head injury must be applicable to all such children. At study design stage it was important to take a pragmatic approach to this requirement, as it was clear that the rate of significant events was low and that it would be unethical to expose large numbers of children to unnecessary major radiation exposure. A balance needed to be struck between inclusion bias (selecting only children who already fulfilled some existing rule, which indicated that they should undergo computed tomography scanning) and the ethical limits that would be placed on the study. Thus, a composite end point was agreed and ethically approved. This involved the identification of all children who died, had neurosurgical intervention or had abnormalities on a computed tomography scan. This composite end point was reinforced with a prospective, thorough follow-up strategy, which was designed to ensure that no children who died as a result of their head injury or who had late neurosurgical intervention were missed. Although this approach can be criticised because some children with abnormalities on the computed tomography scan may not have been identified as they did not undergo such scanning, it has considerable strength in that children with both immediate or late significant events are included. Thus, it can be assumed that any undiagnosed abnormalities on the computed tomography scan were clinically non-significant. We believe that this method is superior to the alternative one of telephoning each patient 2 weeks after admission, and previous studies have been criticised for their inability to contact all patients for follow-up.9,21
The children’s head injury algorithm for the prediction of important clinical events rule
A computed tomography scan is required if any of the following criteria are present.
Witnessed loss of consciousness of >5 min duration
History of amnesia (either antegrade or retrograde) of >5 min duration
Abnormal drowsiness (defined as drowsiness in excess of that expected by the examining doctor)
⩾3 vomits after head injury (a vomit is defined as a single discrete episode of vomiting)
Suspicion of non-accidental injury (NAI, defined as any suspicion of NAI by the examining doctor)
Seizure after head injury in a patient who has no history of epilepsy
Glasgow Coma Score (GCS)<14, or GCS<15 if <1 year old
Suspicion of penetrating or depressed skull injury or tense fontanelle
Signs of a basal skull fracture (defined as evidence of blood or cerebrospinal fluid from ear or nose, panda eyes, Battles sign, haemotympanum, facial crepitus or serious facial injury)
Positive focal neurology (defined as any focal neurology, including motor, sensory, coordination or reflex abnormality)
Presence of bruise, swelling or laceration >5 cm if <1 year old
High-speed road traffic accident either as pedestrian, cyclist or occupant (defined as accident with speed >40 m/h)
Fall of >3 m in height
High-speed injury from a projectile or an object
If none of the above variables are present, the patient is at low risk of intracranial pathology.
We do not believe that there is considerable circularity between the clinical care drivers during the study and the findings as stated in the CHALICE rule. In particular, the recommendations of the RCS17 differ from those of the CHALICE in that a major indication for computed tomography in a district general hospital as per the RCS guidance is fracture on the SXR. CHALICE has looked critically at both the indications for SXR and the computed tomography scan in detail, and distilled from a long list (table A available at http://www.archdischild.com/supplemental) those that truly indicate the likelihood of clinically significant intracranial injury (the composite outcome). It is not surprising that some of the CHALICE rules support previous expert opinion. Perhaps it would be more surprising if it did not.
Our study has other limitations. Clinicians were not always blinded to the outcome of the computed tomography scan before completing the proforma. Although clinicians completed most proformas on the first clinical examination, they were reminded at a later date if they did not do this. Our proforma was also the clinical record sheet and, thus, compliance was generally high. However, we do not have data on the number of missed patients that could have been eligible. Finally, our study is only a derivation study and it should now be prospectively validated, with its reproducibility, acceptability, usability and economic effect evaluated across multiple sites.
Few studies have successfully derived a clinical decision rule applicable to all children from a large cohort of patients. Palchak et al23 in 2003 derived a rule on examining 2043 patients from a single hospital, aged <18 years, who had had head trauma and showed positive findings on history or clinical examination such as LOC, amnesia, vomiting or headache. Of nine predictive variables assessed, abnormal mental status, clinical signs of skull fracture, history of vomiting, scalp haematoma (in children aged ⩽2 years) or headache identified 96 of 98 patients with positive intracranial pathology on computed tomography scanning (98% sensitivity, 95% CI 93% to 100%).
Greenes and Schutzman23 conducted a prospective study on 608 patients aged <2 years in a single hospital.23 Their results support our finding that children with suspected non-accidental injury, history of lethargy or a major scalp haematoma had an increased risk of significant intracranial injury. In addition, they found that LOC, seizures or vomiting alone was not adequate to predict intracranial injury, and that the absence of clinical symptoms or signs did not fully exclude the possibility of uncovering positive pathology. Together with a systematic review by the American Academy of Pediatrics and expert consensus, these authors formalised this study by producing guidelines for head injuries in children <2 years old.24 They allocated patients into four risk groups, with computed tomography scanning recommended in the highest risk group of children who vomited >3 times or had an LOC, a history of lethargy, a high-risk mechanism or considerable bruising. Although these guidelines agree with many of our recommendations, including the suggestion that patients are safe for discharge after a negative finding on a computed tomography scan, like ours, their guidelines have not yet been validated in other hospitals.
Haydel and Shembekar25 in 2003 assessed the New Orleans criteria13 in children aged >5 years. They assessed 175 children with GCS 15 from a single institution and concluded that the 14 positive computed tomography scans that they found could be identified by their rule, which was derived from and validated in scans of adults.
The National Institute of Clinical Excellence, UK, systematically reviewed all studies on head injury up to 2002. They concluded that no studies on childhood head injury could be used to construct a robust rule, and advocated extrapolating the Canadian CT head rule for children. We assessed the performance of this rule in children,26 extending our previous work to the full database. We found that the sensitivity was 94% (95% CI 91% to 97%) and the specificity was 89% (95% CI 89% to 90%), with a computed tomography ordering rate of 12%. Sixteen patients would have been missed if this rule had been strictly applied to our database. Thus, we conclude that the CHALICE rule is safer than extrapolating an adult head injury guideline to children without a considerably increased computed tomography ordering rate. We also found that the RCS guidelines were being poorly applied to children, with half the radiographs, admissions and computed tomography scans recommended by the RCS guidelines not being carried out.26
Many of the variables identified as significant in our study have also been identified in a meta-analysis of clinical variables identified from 16 papers in the paediatric literature.27 Focal neurology, seizures, LOC and abnormal GCS were all major predictors, but headache was found not to considerably predict significant intracranial pathology.
Although computed tomography scanning is a safe procedure for those who are able to comply with the investigation, young children may require sedation, which is not without complications.28 Occasionally, the risk of the investigation should be balanced with the possibility of delayed diagnosis with observation alone in children who have been identified as high risk by our rule.
Finally, we do not support the continued use of SXR for children with acute head injury, except for highly selected patients who may have had non-accidental injury. Although many studies have found that the evidence of a fracture on SXR markedly increases the incidence of intracranial pathology,29–31 we agree with other studies that the SXR has a poor sensitivity for identifying patients with intracranial pathology, that fractures are identified on only a small number of radiographs and that fractures are easily missed by those interpreting the radiographs.32 We thus advocate that a clear decision rule (such as the CHALICE rule) that uses clinical variables alone and that identifies children at high risk of significant intracranial injury for computed tomography scanning is the optimal rule for the management of head injuries in children.
The CHALICE rule, derived from 22 772 children attending the emergency departments of 10 hospitals in the UK, may provide a comprehensive clinical decision rule for the management of head injuries in children that identifies patients at risk of significant intracranial pathology. Validation studies are now needed.
What is already known on this topic
1 in 500 children with a minor head injury will have serious intracranial complications.
Clinical guidelines extrapolated from adult guidelines and expert consensus are currently available to identify these patients in the emergency department.
What this study adds
The CHALICE rule identifies children at high risk of serious intracranial complications following a head injury.
The CHALICE rule is the first head injury decision rule derived entirely from prospective multicentre data from head injuries in children. The CHALICE database is the world’s largest prospective cohort study in childhood head injuries with 22 772 children studied from 10 centres.
We thank the following clinicians for assistance: Rosemary Morton, Manchester Royal Infirmary; David Lloyd, Helen Carty and Barbara Phillips, Alder Hey Hospital; Patricia Brennan, Sheffield Children’s Hospital; Marion Waters, Countess of Chester Hospital, Chester; Steven Southworth, Stepping Hill Hospital; S Kumar, Royal Oldham Hospital; Kasem Ali, Bury General Hospital; Brendan Ryan, Wythenshawe Hospital; Lorcan Duane, Booth Hall Children’s Hospital. We also thank Steve Roberts, Medical Statistician, Manchester Royal Infirmary.
Funding: This study was funded by the Enid Linder Foundation Research Fellowship from the Royal College of Surgeons of England, The Child Brain Injury Trust and The Dickinson Trust. None of these funding bodies had any role in the conduct of the study or in the preparation of the manuscript.
Competing interests: None.
This paper conforms to the STARD guidelines for reporting of diagnostic cohort studies.
Contributors: JD was the lead researcher, and led the study and prepared the manuscript. KM-J conceived the project, obtained funding, supervised the research, and oversaw and supervised its conduct. JPD created the proforma and supervised data collation. J-PL carried out the analysis for interobserver agreement. FL and JB organised the multicentre participation and supervised its conduct. All authors had access to all study data and participated in manuscript preparation and checking.