Background Clinical decision rules (CDRs) could be helpful to safely distinguish between bacterial and aseptic meningitis (AM).
Objective To compare the performance of two of these CDRs for children: the Bacterial Meningitis Score (BMS) and the Meningitest.
Design Secondary analysis of retrospective multicentre hospital-based cohort study.
Setting Six paediatric emergency or intensive care units of tertiary care centres in five European countries.
Patients Consecutive children aged 29 days to 18 years presenting with acute meningitis and procalcitonin (PCT) measurement.
Main outcome measures The sensitivity and specificity of the BMS (start antibiotics in case of seizure, positive cerebrospinal fluid (CSF) Gram staining, blood neutrophil count ≥10 ×109/l, CSF protein level ≥80 mg/dl or CSF neutrophil count ≥1000 ×106/l) and the Meningitest (start antibiotics in case of seizure, purpura, toxic appearance, PCT level ≥0.5 ng/ml, positive CSF Gram staining or CSF protein level ≥50 mg/dl) were compared using a McNemar test.
Results 198 patients (mean age 4.8 years) from six centres in five European countries were included; 96 had bacterial meningitis. The BMS and Meningitest both showed 100% sensitivity (95% CI 96% to 100%). The BMS had a significantly higher specificity (52%, 95% CI 42% to 62% vs 36%, 95% CI 27% to 46%; p<10−8).
Conclusion The Meningitest and the BMS were both 100% sensitive. This result provides level II evidence for the sensitivity of both rules, which can be used cautiously. However, use of the BMS could safely avoid significantly more unnecessary antibiotic treatments for children with AM than can the Meningitest in this population.
Statistics from Altmetric.com
Because distinguishing between bacterial meningitis (BM) and aseptic meningitis (AM) is sometimes difficult in a paediatric emergency department (ED), various tools—including clinical decision rules (CDRs)—have been proposed to help clinicians to treat BM as early as possible and limit unnecessary antibiotic use and hospital admissions for patients with AM. Among the CDR published for this purpose, only the Bacterial Meningitis Score (BMS, table 1)1 was found to have high sensitivity,2,–,5 good specificity and ease in bedside application.4
What is already known on this topic
▶ Only one rule, the Bacterial Meningitis Score (BMS), has reached a high level of performance and validation to distinguish early between bacterial and aseptic meningitis (AM) in children, but rare patients with bacterial meningitis may be missed.
▶ The Meningitest is a new performing rule, which includes procalcitonin identified as the best biological marker to distinguish between bacterial and AM in children. But the Meningitest needs to be validated.
What this study adds
▶ This study confirms the 100% sensitivity of the Meningitest, with a narrow CI (95% CI 96% to 100%).
▶ Although the BMS and the Meningitest reached the same 100% sensitivity, the specificity of the BMS was significantly higher in this population (52% vs 36%; p<10−8).
However, in the derivation study and in studies performed in Belgium and France, a few cases of BM would have been missed with use of the BMS.1 3 6 The BMS was refined mainly by adding serum procalcitonin (PCT) level,7 previously identified and validated as the best biological marker for distinguishing early between BM and AM in paediatric patients.8 9 In a single-centre study, this new rule (table 1), called the Meningitest, showed 100% sensitivity in construction and validation sets and at least 50% specificity.10 Because the study included only a few patients with BM (n=21), these results needed validation, with comparison to the reference rule (the BMS), as recommended by the Evidence-Based Medicine Working Group.11
We aimed to validate the Meningitest for early distinction between BM and AM in children presenting to hospital EDs and to compare its sensitivity and specificity with the BMS, by using a large European multicentre dataset.
Study design and setting
We conducted a secondary analysis of hospital-based case-cohort studies. The strategy to identify potential investigating centres has been detailed previously.9 Six centres from five European countries agreed to participate in the study.12,–,16 Definitions and inclusion and exclusion criteria were provided to each investigator within the study protocol, before completed data files were examined. The reference methodology for deriving/validating prediction rules has been followed.11 17
Selection of participants
All consecutive patients aged 29 days to 18 years who presented to hospital EDs for BM or AM, with measurement of the main inflammatory markers (including PCT) in blood and cerebrospinal fluid (CSF), quickly available in the ED, were eligible for inclusion. All tests should be performed regularly at all location. BM was defined as the acute onset of meningitis (CSF white blood cell count ≥7×106/l)1 18 and bacterial infection documented in CSF (direct examination, culture, latex agglutination or PCR) or on blood culture. AM was defined as the acute onset of meningitis and the absence of any BM criteria. Virological tests on the CSF were not specially required. Patients were excluded if they had known neurosurgical disease, known immunosuppression, traumatic lumbar puncture (CSF red blood cell count ≥10 ×109/l), septic shock or pretreated meningitis or were referred from another hospital because of a diagnosis of meningitis. When the aetiology of the meningitis could not be ascertained, the patient was excluded. Although the primary investigators at each site were not blinded to the outcome at inclusion, the main investigator for secondary analysis of centralised data was blinded to the outcome.
Data collection and processing
For each patient, data needed for the application of the BMS and the Meningitest (table 1) were collected, as were results of blood and CSF culture (compulsory), CSF soluble antigen tests or bacterial or viral PCR if any. These data were retrospectively extracted from existing databases or medical files in each centre and sent to the coordinating centre. For patients recruited in paediatric departments and intensive care units, the values were those measured at the time of admission to the ED. In all centres, at that time, PCT level was determined by a standardised and validated immunoluminometric test (LUMItest PCT; Brahms Diagnostica, Berlin, Germany). Institutional review committee approval was obtained from the institutions of each participating investigator, except in Lille, France, where, at that time, the committee did not require approval of such retrospective studies.
Criteria for application of the BMS and the Meningitest are detailed in table 1. All but one exclusion criterion are similar for both rules: known neurosurgical history, known immunosuppression, pretreatment with antibiotics, traumatic lumbar puncture (red blood cell count ≥10 ×109/l) and septic shock. The predictive value of purpura is not used in the same way in both rules: it is one of the exclusion criteria (ie, treatment without any decision rule) for the BMS and is within the decision rule (indicating treatment) in the Meningitest (see table 1). In other words, the authors of the two rules consider that patients should be considered at high risk of BM if this sign is present. The analysis has taken into account this variation.
Primary data analysis
Each rule was retrospectively applied to each patient of the cohort to calculate the rules' sensitivities and specificities with their 95% CI. The sensitivities and specificities of the rules were compared by use of a McNemar test for matched pairs. Given the differences between the two rules in presence of purpura, comparisons were performed including and then excluding patients with purpura. When the BMS was applied to the whole population of patients, including those with purpura, we considered that the BMS would have classified these patients at high risk of BM. Analyses involved the Epi-Info 6.04fr software (Centers for Disease Control and Prevention, Atlanta, GA, USA).
The six centres that agreed to participate in the study provided complete data for 198 patients. Thirty-four patients (15%) were excluded because of missing microbiological data to ascertain the aetiology of the meningitis: 27 were presumed to have BM and 7 AM. Some included patients needed further admission in a paediatric intensive care unit (n=34, including 24 with BM). Inclusion dates varied from 1996 to 2005 between centres, with inclusions mainly after 2000.
A total of 96 patients (48%) had BM and 102 (52%) AM. The mean age was 4.8 years (median 4.0 years; range: 1 month to 15.9 years), and the ratio of males to females was 1.4. CSF Gram staining showed 75% sensitivity (95% CI 66% to 84%; range 36% to 100%) for BM. The main micro-organisms identified were Neisseria meningitidis (n=45), Streptococcus pneumoniae (n=32), Haemophilus influenzae (n=7) and Streptococcus agalactiae (n=4). We found no contaminant micro-organism (eg, Staphylococcus epidermidis). CSF culture was positive for 79% of patients. Soluble antigen test-alone results were positive for 12% of patients, blood culture-alone results were positive for 7% and bacterial PCR-alone results were positive for 1%. Purpura was observed in 21% of patients at the time of presentation (n=40), including 37% of patients with BM (n=33/90) and 7% of patients with AM (n=7/98). A viral pathogen was identified in 21% of patients with AM. The main characteristics of this set of patients and derivation sets for both rules are in table 2.
Among the whole population, the Meningitest showed 100% sensitivity (95% CI 96% to 100%) and 36% specificity (95% CI 27% to 46%). The BMS applied to the same population (including patients with purpura in the high-risk group of BM) showed 100% sensitivity (95% CI 96% to 100%) and 52% specificity (95% CI 42% to 62%). The performances of each specific component of the two clinical rules are detailed in the table 3. In comparing the performances of the rules, there was no difference in sensitivities. The difference in specificity between the two rules was statistically significant (p<10−8). Among the population of patients without purpura (n=149, including 57 patients with BM), the Meningitest had 39% specificity (95% CI 29% to 49%) and the BMS 57% specificity (95% CI 46% to 67%) (p<10−5).
In this large multicentre study conducted in various countries in Europe, we confirmed the high sensitivity of the Meningitest (100%; 95% CI 96% to 100%) for the early distinction between BM and AM in children presenting to a hospital ED. On comparing the Meningitest with the BMS, both rules reached the same 100% sensitivity, however with significant differences in specificities (36% and 52%, respectively).
Decision rules need to follow a complete validation process before they are used in routine practice. The validation process includes internal validation, with the data for ‘new’ patients coming from the same centre, that are comparable to those used for the derivation of the rule, and external validation, with data for patients included in other centres. The BMS and Meningitest have now been validated with more than 4755 (including 1161 with BM) and 255 cumulative patients (including 103 with BM), respectively.19
The Meningitest and BMS have four main differences. The first is the extent of validation between the two rules, which is explained by the use of the PCT in the Meningitest. Although this marker is the best single biological predictor distinguishing between BM and AM in children,8 9 it is new and perhaps not routinely available in all EDs compared with the criteria used in the BMS. The second difference concerns the 100% sensitivity of the rules. It is already known that BMS results were found to be negative for a few children with BM (about 0.5%).3 6 Although the Meningitest has been used for relatively few patients with BM (n=103), such false-negative results have never been reported for that rule. For this situation, 100% sensitivity of the rule is crucial to avoid the potential clinical adverse effects of a false-negative result. Such an event is less probable with the Meningitest by its use of PCT and a lower CSF protein level threshold (50 mg/dl) than with the BMS. Compared to the BMS, the Meningitest with its lower CSF protein level threshold shows the third difference between the rules: in the present study, the BMS showed higher specificity (52% vs 36%). The fourth difference between the two rules is the classification of purpura. Indeed, ‘purpura’ in the BMS was an exclusion variable initially only if extensive,1 which means that these patients do not require a decision rule to be considered at high risk of BM. However, in the validation study, any type of febrile purpura (extensive or not) was classified as exclusion criteria, because of the high risk of BM.2 The authors of the Meningitest included purpura in the rule (indicating high risk of BM).10 This inclusion was considered safe because busy clinicians may forget to check exclusion criteria in applying a rule.
The main limitation of this study was its secondary analysis of retrospective cohorts, based on the existence of routine PCT detection in the ED.12,–,16 Publication and classification bias of patients has been previously discussed, appears to be limited9 and was balanced by the large size of our population, particularly for BM cases, which strengthens our results. The high prevalence of BM, compared to construction sets, may be explained by the inclusion of patients admitted to two paediatric intensive care units (and therefore are more likely to have BM) and three paediatric departments and by the local routine strategy in two centres of not performing routine lumbar puncture in paediatric patients with suspected acute AM who appeared well (Firat, Rzeszow). This process resulted in an overestimation of the frequency of patients with BM (close to 50%). However, it did not influence the sensitivity and specificity because these results, contrary to positive and negative predictive values, are independent of disease prevalence. This high prevalence may also be explained by a bias of PCT measurement. Indeed, although in theory, PCT was routinely measured in every patient with acute meningitis during the study periods the investigators selected, it could have concerned only inpatients and might not have been measured in some patients with AM treated on an outpatient basis. Because the performance of lumbar puncture and PCT is based on clinical signs and level of suspicion of the diagnosis, more suspected cases could have been included, thus affecting the diagnostic value of each predictor of the rules. The exact impact of this potential bias on the performance of the rules cannot be estimated in the present study but could probably be evaluated in a prospective study.
Should we apply the BMS or Meningitest in day-to-day practice? At the Evidence-Based Medicine Working Group scale, validation with prospective studies and impact analysis are necessary before establishing a definitive answer.11 However, such studies will be long and need a lot of investigators, because the frequency of BM has decreased with the dissemination of conjugated vaccinations with Haemophilus influenzae b, Streptococcus pneumoniae and Meningococcus C.20 Moreover, studies reporting physician attitudes towards treatment for children with AM have found that at least 20% of children are not given antibiotics.2 Therefore, implicit criteria are used to decide when not to treat. These criteria are probably based on the experience of skilled clinicians but are difficult to generalise. Therefore, while waiting for a validation by a large prospective multicentre study and impact analysis, a cautious use of the BMS or Meningitest to replace not-validated implicit criteria could be proposed to physicians needing explicit criteria to decide management. Physicians should be reminded that these rules are not proposed to decide whether lumbar puncture is needed. Physicians should also remember that rules are developed to help in reaching a decision and should not replace the clinician's skill and perception.
The specificity of the BMS is higher than that of the Meningitest, with both showing 100% sensitivity. However, because use of the BMS has previously resulted in BM misdiagnoses,1 3 6 the Meningitest could probably be used by physicians needing a more secure rule,21 by knowing that this rule safely avoided only 36% of unnecessary hospitalisations and antibiotic treatments for patients with AM in this population. A large prospective study is now needed to more precisely define the place of the Meningitest in comparison with the BMS.
Presented at the European Society for Pediatric Infectious Diseases annual meeting, Porto, Portugal, May 2007.
Funding This study was supported by the Direction de la Recherche Clinique and the Unit of Clinical Research in Hospital Cochin–Saint-Vincent-de-Paul, grant CRC 03154 from the Assistance Publique–Hôpitaux de Paris, the Fond d'Etude et de Recherche du Corps Médical des Hôpitaux de Paris and the Fondation Bayer Santé.
Competing interests MC and DG have received unconditional research support (less than €30 000) from Brahms AG, which manufactures PCT, for other studies on other topics in 2005 and 2007. FD and MC had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Ethics approval This study was conducted with the approval obtained from each institution.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.