Article Text

Download PDFPDF

Clinical prediction models for young febrile infants at the emergency department: an international validation study
  1. Evelien de Vos-Kerkhof1,
  2. Borja Gomez2,3,
  3. Karen Milcent4,
  4. Ewout W Steyerberg5,
  5. Ruud Gerard Nijman6,
  6. Frank J Smit7,
  7. Santiago Mintegi2,3,
  8. Henriette A Moll1,
  9. Vincent Gajdos8,
  10. Rianne Oostenbrink1
  1. 1 Department of General Paediatrics, Erasmus MC-Sophia Children’s Hospital, Rotterdam, The Netherlands
  2. 2 Paediatric Emergency Department, Cruces University Hospital, Bilbao, Spain
  3. 3 University of the Basque Country, Bilbao, Spain
  4. 4 AP-HP Department of Paediatrics, Hôpitaux Universitaires Paris Sud–Antoine Béclère, Clamart, France
  5. 5 Department of Public Health and Clinical Decision Making, Erasmus MC–University Medical Centre Rotterdam, Rotterdam, The Netherlands
  6. 6 Department of Paediatric Accident and Emergency, St Mary’s Hospital, Imperial College–NHS Healthcare Trust, Rotterdam, The Netherlands
  7. 7 Department of General Paediatrics, Maasstad Hospital, Rotterdam, The Netherlands
  8. 8 Université Paris-Saclay, Université Paris-Sud, UVSQ, CESP, INSERM, Villejuif, France
  1. Correspondence to Dr Rianne Oostenbrink, Department of General Paediatrics, Erasmus MC-Sophia Children’s Hospital, Rotterdam 3015, The Netherlands; r.oostenbrink{at}erasmusmc.nl

Abstract

Objective To assess the diagnostic value of existing clinical prediction models (CPM; ie, statistically derived) in febrile young infants at risk for serious bacterial infections.

Methods A systematic literature review identified eight CPMs for predicting serious bacterial infections in febrile children. We validated these CPMs on four validation cohorts of febrile children in Spain (age <3 months), France (age <3 months) and two cohorts in the Netherlands (age 1–3 months and >3–12 months). We evaluated the performance of the CPMs by sensitivity/specificity, area under the receiver operating characteristic curve (AUC) and calibration studies.

Results The original cohorts in which the prediction rules were developed (derivation cohorts) ranged from 381 to 15 781 children, with a prevalence of serious bacterial infections varying from 0.8% to 27% and spanned an age range of 0–16 years. All CPMs originally performed moderately to very well (AUC 0.60–0.93). The four validation cohorts included 159–2204 febrile children, with a median age range of 1.8 (1.2–2.4) months for the three cohorts <3 months and 8.4 (6.0–9.6) months for the cohort >3–12 months of age. The prevalence of serious bacterial infections varied between 15.1% and 17.2% in the three cohorts <3 months and was 9.8% for the cohort >3–12 months of age. Although discriminative values varied greatly, best performance was observed for four CPMs including clinical signs and symptoms, urine dipstick analyses and laboratory markers with AUC ranging from 0.68 to 0.94 in the three cohorts <3 months (ranges sensitivity: 0.48–0.94 and specificity: 0.71–0.97). For the >3–12 months’ cohort AUC ranges from 0.80 to 0.89 (ranges sensitivity: 0.70–0.82 and specificity: 0.78–0.90). In general, the specificities exceeded sensitivities in our cohorts, in contrast to derivation cohorts with high sensitivities, although this effect was stronger in infants <3 months than in infants >3–12 months.

Conclusion We identified four CPMs, including clinical signs and symptoms, urine dipstick analysis and laboratory markers, which can aid clinicians in identifying serious bacterial infections. We suggest clinicians should use CPMs as an adjunctive clinical tool when assessing the risk of serious bacterial infections in febrile young infants.

  • epidemiology
  • evidence-based medicine
  • general paediatrics
  • infectious diseases
View Full Text

Statistics from Altmetric.com

What is already known on this topic?

  • Febrile children, and young febrile children in particular, are at risk of serious bacterial infections, such as pneumonia, sepsis or meningitis, which are among the leading causes of morbidity and mortality in children both in developed and developing countries.

  • The management of febrile young infants is hampered by the difficulty of discriminating between children with serious bacterial infections and those with self-limiting infectious illnesses, as specific ‘red flag’ features are often absent.

  • Existing clinical prediction models for febrile children often are derived from small numbers of children aged <1 year, and few focused specifically on febrile children aged <1 year.

What this study adds?

  • Validation using cohorts from three European emergency care settings found four out of eight clinical prediction models showed satisfactory performance in identifying serious bacterial infections in febrile infants, but favoured specificity above sensitivity.

  • Clinical prediction models including clinical signs and symptoms, urine dipstick analysis and laboratory markers had better performance characteristics in young febrile infants compared with clinical prediction models including only symptoms and signs.

Introduction

Fever is a major cause for seeking medical care in childhood.1 Febrile children are at risk of serious bacterial infections, such as pneumonia, sepsis or meningitis. Despite the low incidences of serious bacterial infections, their delayed diagnoses are among the leading causes of morbidity and mortality in children.2 3 Clinical decision-making in febrile infants is hampered by the inability to easily discriminate the minority of children with serious bacterial infections from those with self-limiting viral illnesses.4 5 This is particularly true for young (age <1 year) febrile children, as young infants have the physiological ability to retain cardiovascular and respiratory parameters within normal ranges for a prolonged period, even in the presence of a serious infection.6 Also specific ‘red flag’ signs of serious bacterial infections may not be present in young children (eg, meningeal signs) and we must rely on non-specific features (eg, lethargy).7 Moreover, young infants are more susceptible to serious bacterial infections as they will not have completed their routine childhood immunisation schedules.8

In paediatric emergency care, low-risk criteria (Rochester9 or variations of these10–13) are often used to assist predicting which children are more likely to have serious bacterial infections. These criteria generally use a mix of general appearance, medical history, focal infection and white cell count to rule out serious bacterial infections (SBI). These consensus-based low-risk criteria date from an era with a much higher prevalence of SBI as a result of changing epidemiology, due to extended vaccination strategies, group b streptococcu screening and replacing emerging or seasonal pathogens.14–16

Alongside these low-risk criteria, clinical prediction models (CPM) with a statistical rather than consensus basis have been introduced to support the diagnostic process.17 18 They provide physicians with estimates of the probability of a child having a serious infection by combining clinical signs and symptoms and additional diagnostic test results in mathematical models.

Studies in which CPMs for children at risk for serious bacterial infections were derived often included only limited numbers of young children aged <1 year, and only few of these CPMs specifically targeted young infants.7 Hence, broad validation and implementation studies of CPMs for febrile children at risk of serious bacterial infections are an essential, yet currently missing, step in the translation of CPMs into clinical practice.19 20 As a result, the clinical value of existing CPMs remains unclear in this cohort of young infants at risk for serious bacterial infections. The aim of this study was to systematically search for CPMs targeting febrile children at risk of serious bacterial infections in emergency or primary care and to subsequently validate these CPMs in four different cohorts of febrile young infants aged <1 year from European emergency care settings.

Methods

Inclusion criteria systematic review

We included studies that1: described a CPM with the purpose of predicting serious bacterial infections in febrile children2; included three or more predictors which were routinely determined on clinical examination, possibly extended by diagnostic test results3; advised on management strategies (ie, clinical decision model) or gave a risk score/classification (ie, clinical prediction rule)4; and were based on statistical modelling such as multivariable logistic regression analysis. We excluded studies that: (1) were derived using expert opinion or consensus-based approaches; (2) investigated one specific diagnosis only (eg, pneumonia, meningitis); (3) included biomarkers only; (4) included immune-compromised patients; (5) were performed in low-income countries; (6) were published before the year 2000; and (7) were not published in English.

Search strategy and risk of bias assessment systematic review

We systematically searched Medline OvidSP, Embase, CINAHL, Cochrane central register of controlled trials, Web of Science, Google Scholar and PubMed as publisher (searches updated in May 2016) (online supplemental information 1). We then checked the reference list of these papers for additional articles that were not found by the initial computerised search. Two authors (EVK/RO) independently assessed the potential assessed risk of bias of the studies included using the QUADAS list, a quality assessment tool of diagnostic accuracy studies.21 The tool is a list of 14 questions to be answered ‘yes’, ‘no’ or ‘unclear’ (online supplemental information 2). Studies selected for analysis were given an A, B, C or D rating according to a previous described scoring system.22 Consensus was reached by the two reviewers (EVK/RO) if there was disagreement on an item. If no consensus was achieved, the opinion of a third independent reviewer was decisive (HAM). Moreover, we added the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) checklist (online supplemental information 3), which aims to improve the reporting of studies developing, validating or updating a prediction model.23

Supplemental material

Supplemental material

Supplemental material

Data extraction and management validation study

In this validation study we used four different cohorts of febrile young infants, including two cohorts with children <3 months from Spain (Cruces University Hospital, Bilbao, Basque Country, Spain) and France (15 French paediatric emergency departments (ED)),24 as well as one cohort with children aged <1 year from two hospitals in Rotterdam, the Netherlands (Maasstad Hospital and Sophia Children’s Hospital). The Dutch cohort was divided into a cohort of children aged <3 months (the Netherlands <3 months’ cohort) and children aged between 3 and <12 months (the Netherlands >3–12 months’ cohort). In the Dutch cohort we prospectively enrolled all children (1 month to 1 year) presenting with fever at the EDs of the Erasmus MC-Sophia Children’s Hospital (inner city university hospital, 7000 annual paediatric ED visits)25 and the Maasstad Hospital (inner-city teaching hospital, 10 000 paediatric medical emergency consults yearly), Rotterdam (2009–2012), the Netherlands. In both hospitals febrile children were eligible to participate in the study if fever had been noted at home in the 24 hours prior to presentation, if body temperature measured at the ED was ≥38.5°C or ‘fever’ was used as a positive discriminator of the Manchester Triage System. Children with chronic comorbidity were excluded. Informed consent was required and obtained. Detailed description of this febrile child cohort was published earlier.17 26 The Paediatric Emergency Department of Cruces University Hospital in Bilbao is a tertiary teaching hospital and provides care for approximately 55 000 children per year of less than 14 years of age. During 9 consecutive years (2003–2012) data on infants younger than 3 months of age with fever without source (FWS) seen in the ED were prospectively collected.27–31 FWS was defined as temperature measured at home or at the ED ≥38°C and without a clear source of the origin of the fever. The study was approved by the local Research Committee and as the registry contained anonymous information, and as no intervention was performed, the need for informed consent was waived.27 Lastly, the prospective cohort study of France was conducted at 15 paediatric EDs in several regions in France. These hospitals were both teaching and tertiary centres seeing 15 000–50 000 children annually. From October 2008 through March 2011 febrile children (≥38°C at home or on admission) aged between 7 and 91 days without major comorbidities were included.24 The local ethics committee approved the study and parental written informed consent was obtained.24 Definitions of serious bacterial infections were comparable in all four cohorts and included pneumonia, and ‘other’ serious bacterial infections as meningitis, septicaemia, urinary tract infections, erysipelas, cellulitis, bacterial gastroenteritis, orbital cellulitis, bacterial upper respiratory infection, ethmoiditis, septic arthritis and osteomyelitis. All definitions were defined by positive bacteriological cultures of normally sterile sites and/or by nodular infiltrates or consolidation in the lung on chest radiographs.17 24 27 All cohorts included a follow-up period to ensure no serious infections were missed.

Statistical analyses

When the selected CPMs included variables that were not identical to the variables available in our four cohorts we identified proxies by (1) combining available features into one predictor variable (eg, ‘dyspnoea’ and ‘chest-wall retractions’ into ‘shortness of breath’) or (2) dichotomising categorical variables (eg, mild, moderate, severe) into ‘absent ‘or ‘present’ (online supplemental information 4). In the situation when an original variable contained more categories (eg, mild, moderate, severe) than were available for a proxy variable (eg, absent, present) beta coefficients were recalculated as a weighted mean on the basis of the originally published beta coefficients and the number of patients with the variable present in the derivation cohort. If a CPM predictor variable was completely absent we used the prevalence observed in the original derivation cohort. Missing data of other predictor variables were assumed to be missing at random, and were imputed 10 times using a multiple imputation process with the mice algorithm in R software (V.3.0).32 Different imputational models were developed for each individual CPM, including CPM predictor variables, the outcome variable and variables describing case-mix of the cohort. Imputation was done per validation cohort and results were analysed using Rubin’s rules.33

Supplemental material

We only validated the CPMs if  >2/3 of the required variables were available in the validation data sets.33 The performance of CPMs in allocating children to high-risk versus low-risk categories of serious infection was evaluated by calculating sensitivity, specificity and positive/negative likelihood ratios (LR+ and LR−). Area under the receiver operating characteristic curve (AUC) was used to determine discriminative ability. For CPMs that provided risk predictions on a continuous (0%–100%) scale we assessed calibration graphically and we compared the incidence of cases of serious bacterial infections with the average predicted risk in each validation cohort (calibration in the large). In addition, the calibration slope was calculated using a logistic regression model with the log odds of predicted risk as the only covariate (logistic calibration).34 35 Logistic calibration with a slope of 1 means perfect calibration with similar predictive effects of the model in both the derivation and validation cohorts. Analyses were performed with SPSS software (V.20.0, IBM).

Results

Identification and description of CPMs

The literature search identified 2549 references. Figure 1 presents the flow chart of papers screened for eligibility. Data extraction was performed for eight included articles, each describing a single CPM.17 18 36–41 Seven CPMs were derived in secondary care settings and one CPM41 was derived in primary care (table 1). The prevalence of serious bacterial infections in the original study cohorts varied between 0.8% and 27%, and included between 381 and 15 781 children who had median ages ranging from 0.1 to 0.6 years. Combined, the 8 CPMs included 35 unique clinical variables (range 3–28 variables), with age, temperature, general appearance, capillary refill time, respiratory rate, breathing difficulty and urine dipstick analyses being the most frequently reported ones (online supplemental information 5). Four of the CPMs expressed the risk for serious bacterial infections in a dichotomous way (high/low)36 38 39 41 whereas the other four CPM predicted risks were on a continuous scale (0%–100%) (table 1).17 18 37 40

Figure 1

Flow chart systematic review of clinical prediction models for determination of serious bacterial infections.

Table 1

Overview of selected clinical prediction models*

Risk of bias assessment

All studies failed to report data on at least one or more QUADAS criteria (online supplemental information 2). The CPM by Bachur and Harper (CPM 1)36 and Galetto-Lacour et al (CPM 3)38 had the lowest risk of bias. There was initial disagreement on 8 out of the 112 assessed QUADAS items (7%), consensus was reached for all these items by the two reviewers (EVK/RO) and no third opinion was required.

Description of validation cohorts

The Dutch cohort consisted of 925 febrile children under the age of 1 year17 of whom 159 (17.2%) were under the age of 3 months. The Spanish cohort included 2148 febrile children aged <3 months, the French cohort 2204 children.24 27 The inclusion flow charts of these validation cohorts were published earlier.17 24 27 The prevalence of serious bacterial infections varied between 15.1% and 17.2% in the three cohorts of febrile children aged <3 months and was 9.8% for the cohort >3–12 months of age (table 2). A number of original CPM predictor variables were absent in some or all validation cohorts. Most notably this was the case for vital signs (missing in the Spanish cohort) and specific respiratory symptoms (missing in all cohorts) (online supplemental information 5).

Table 2

Patient characteristics of the validation cohorts

Performance of included CPMs

We evaluated the performance of seven CPMs in the Dutch cohorts, four CPMs in the Spanish cohort and all eight CPMs in the French cohort. Two36 38 out of four CPMs36 38 39 41 with high/low-risk classifications performed reasonably well with the discriminative ability (AUC) ranging from 0.72 to 0.87 in all three validation cohorts <3 months. Corresponding sensitivities ranged from 0.48 to 0.94 and specificities from 0.80 to 0.97 (table 3). These two CPMs also showed moderate discriminative values in the cohort >3–12 months.36 38

Table 3

Performance of clinical prediction models with high/low-risk prediction

Of the four CPMs using continuous risk predictions17 18 37 40 CPM5 (37) was validated in all cohorts with AUC ranging from 0.78 to 0.94 in the three cohorts of children <3 months and AUC 0.82 in the cohort of children >3–12 months. CPM 8 (17) could not be validated in the Spanish cohort but achieved good performance in both the two Dutch cohorts and the French cohort to predict pneumonia (range AUC 0.72–0.89), and to predict other SBI (range AUC 0.68–0.82) (table 4).

Table 4

Performance of clinical prediction models with continuous risk prediction

All CPMs calculating continuous risk predictions (CPM 5–8)17 18 37 40 showed poor calibration in the large. Logistic calibration showed almost equal predictor effects as at development for CPM 5 (42) in the two Dutch and the French validation cohorts (slope 0.97–1.15), but underestimated predictor effects in the Spanish cohort (slope 2.95, SE 2.81–3.09). For CPM 8 (17) we had contradictory findings in both Dutch cohorts, with underestimated predictor effects for pneumonia in children <1 year and overestimated predictor effects for other SBI. In the French cohort (<3 months) we observed overestimated predictor effects in both outcomes.

Discussion

Main findings

Eight CPMs were successfully validated in more than 4500 febrile young infants from three European emergency care cohorts. Four out of eight validated CPMs classifying young febrile infants into high-risk versus low-risk categories achieved consistently satisfactory performance for infants <3 months. These four CPMs are therefore potentially suitable in guiding medical decision-making in young febrile infants at risk of serious bacterial infections. Although the specificities of the CPMs generally exceeded the sensitivities of the CPMs in young infants, the overall discriminative ability (as defined by the AUC) in these best performing CPMs was comparable. We observed that CPMs including a combination of clinical signs and symptoms (temperature, general appearance, capillary refill time and breathing difficulty), urine dipstick analyses and laboratory markers (white blood cell (WBC) and C-reactive protein (CRP)) performed better in young infants than CPMs with just clinical parameters. Performance of CPM in infants 3–12 months were quite similar to the performance of CPMs in the cohorts with children aged <3 months.

Comparison with literature

We found general appearance to be a consistent and important clinical predictor variable for identifying serious bacterial infections in febrile young infants. This is supported by, for example, the accuracy study of Pantell et al in which ill appearance was one of the strongest clinical predictors of bacteraemia or bacterial meningitis in infants less than 3 months old.42 Also, the importance of combining biomarkers and clinical signs and symptoms to ensure validity of CPMs was underlined by several other studies.17 22 43–45 Moreover, biomarkers outperformed the clinician’s risk assessment, as generated by visual analogue scores of having SBI.46 Furthermore, the important diagnostic role of urine dipstick analyses was previously demonstrated by De et al who, based on the study’s findings, showed improved diagnostic performance of the National Institute for Health and Care Excellence (NICE) traffic light system recommended adding urine dipstick analysis to the NICE traffic light system.47

To better interpret the observed diagnostic performance of the validated CPMs based on statistically modelling in this study, it might be worthwhile to compare them with consensus-based guidelines or expert opinion-based criteria such as the step-by-step approach of Mintegi et al or the well-known low-risk criteria of Rochester9 and its variations.10–13 For example, Huppler et al 48 concluded that the low-risk criteria performed well, supporting the decision to safely withhold antibiotic treatment with a watchful waiting approach in young febrile infants <3 months. Applying these low-risk criteria of Rochester to our validation cohorts (n=5277) allocated 2033 (39%) children into low risk, with a false-positive rate (risk criteria present, but no SBI diagnosed) of 47% (2479/5277), and 66 SBI cases were classified as low risk by Rochester criteria (1%). Defining the high-risk cohort by the step-by-step criteria (ill appearance; ≤21 days; leukocyturia; procalcitonine ≥0.5) resulted in sensitivities varying from 0.78 to 0.94 in our cohorts against specificities ranging from 0.63 to 0.80. We would like to advocate the preferred use of our identified four CPMs17 36–38 as they showed reasonable performance and much lower false-positive rates for serious bacterial infections in this external validation study.

Interpretation of results and clinical implications

As the cost of missing a serious bacterial infection in a young febrile infant is potentially severe, we are in need of CPMs with high rule-out value in our daily practice (high sensitivity and strong negative LR). The four CPMs (CPMs 1, 3, 5 and 8) achieving the best performance in this study were all developed in secondary care. These CPMs are potentially suitable for identifying serious bacterial infections in young febrile infants. However, their sensitivity is not consistently high in all validation settings, although not consistently related to age.

Logistic calibration studies showed that most predictor effects were underestimated, that is, the predictor effects are probably stronger in these younger cohorts compared with the original cohorts. However, not all differences in validation performance are likely to be the result of this, as calibration in the large showed differences in (unobserved) case-mix in derivation and validation cohorts. We observed that CPM 1 (36) and CPM 5 (37) performed particularly well in the Spanish cohort, potentially due to similarities between derivation and validation cohorts. Many diagnostic studies exclude children <1 month, but the derivation cohort of CPM 1 (36) only included young infants <3 months. Next, the derivation cohort of CPM 5 (37) also targeted febrile children without a source, although including a much wider age range than the cohort at interest in our validation study. One could argue that performance of the rules in our validation study merely relates to differences in cohort and case-mix in general, rather than to the age difference. CPM 2 (41), however, has been externally validated in hospital cohorts elsewhere showing consistent sensitivity but with lower specificity. This contrast with our results, showing a relatively large reduction in sensitivity but only a minor loss of specificity.49 It should be noted that our focus on young febrile children reduces age effects in all CPMs that were developed for children of a much wider age range anyway. For example, this may have affected CPM 2 in particular,41 as this CPM included two cut-off points of age >1 year and which was therefore not applicable to our young infants aged <1 year. Likewise, even though CPM 4 (39) was derived in a cohort of infants >3 months of age, we observed similar performances in the Dutch cohort between the infants <3 months and the infants 3–12 months. A closer look at the predictors of the best performing CPMs in at least three validation cohorts (CPMs 1, 3, 5 and 8) showed a more consistent role of the clinical predictors’ general appearance, temperature, capillary refill time, duration of fever and breathing difficulty. Moreover, these CPMs included laboratory markers as CRP, WBC and urine dipstick analyses. The performance of CPM 3 (38) and CPM 8 (17) including general appearance and/or laboratory markers is comparable in all our four cohorts with their performances in the derivation cohorts. In contrast, CPMs not including general appearance and/or laboratory markers (CPMs 2, 4 and 6) performed worse than in their derivation cohorts. This highlights the importance of laboratory markers as adjuncts to clinical signs and symptoms for the identification of serious bacterial infections in young febrile infants. However, no existing CPM is able to identify all relevant cases of serious bacterial infection in clinical practice, highlighting the position of CPM as tools in assisting clinical decision-making.

Unfortunately, we were able to validate CPM 7, a polytomous model consisting of 27 clinical variables in only one cohort because of the absence of too many predictor variables in the other cohorts. In the French cohort, this model performed particularly well for the outcome ‘pneumonia’. However, compliance with a complex model containing many predictor variables will be limited in clinical practice.50–53 Moreover, the value of CPM 7 (18) is limited for children with FWS, because many predictor variables include symptoms related to a focus of infection (eg, wheeze, stridor, abnormal ear/nose/throat signs) that will be absent in cohorts of very young aged children with non-specific symptoms.

In summary, it remains difficult to identify one optimal CPM for young infants. For example, CPM 3 (38) shows a consistent high specificity in all validation cohorts, but could also result in considerable overtreatment due to its low sensitivity. Application of CPM 8 seems to require setting specific recalibration studies prior to implementation. We suggest that the use of robust CPMs is important in clinical practice evaluating febrile infants. They appear to outperform currently used low-risk criteria. However, given all the models’ fairly low sensitivity, using CPMs in clinical practice should be considered with some caution and only as an adjunct to the diagnostic work-up of febrile infants. Choices for the optimal CPM may be centre specific depending on setting.

Strengths and limitations

A major strength of our study is a synthesis of four large cohorts of febrile children with variable backgrounds to assess the diagnostic performance of existing CPMs. In addition, we focused on young children in particular, whereas most studies on CPMs focused on broader age categories. Moreover, we evaluated all potentially relevant CPMs by systematically reviewing the literature.

This study also has limitations. We included CPMs derived in cohorts with low SBI prevalence (from primary care) to high prevalence settings of febrile children without a source.37 38 We performed calibration studies to study systematic errors in risk prediction and overall predictor effects, but did not study the diagnostic value of each individual CPM variable. However, we feel this has been done exhaustively in previous work, such as a systematic review by Van den Bruel et al.22 Second, despite the considerable size of the validation cohorts, there were only a limited number of children with pneumonia in the Spanish cohort (n=4) and bacteraemia (n=1 in the <3 months’ cohort and n=6 for the cohort >3–12 months) in the Dutch cohort, thus limiting the precision of validating CPM 7 (18) and CPM 8 (17). Furthermore, not all CPM predictor variables had been recorded in all validation cohorts, necessitating the use of proxy variables or using the original prevalence data for completely absent variables. These original prevalence data were mostly used in the CPM of Craig et al (CPM 7) due to the great number of variables used in this CPM. However, this CPM could only be validated in the French cohort. Next, given the small number of the Dutch <3 months’ cohort (n=159, 15.1% SBI) results of this cohort should be interpreted with care, although the results were comparable with the larger validation cohorts. Pooled analysis of cohorts was considered, but not performed for the observed substantial differences in diagnostic performances between cohorts. Apparently, although similar inclusion criteria were used and similar SBI prevalence observed in the validation cohorts, there was still significant heterogeneous case-mix, as illustrated by the calibration studies.54 This may in particular have affected the validation of CPM 7 with both a substantial difference in age distribution, prevalence and distribution of diagnoses.

Although this study did not aim to develop a new prediction rule, the results pose the question whether or not there is a need for other prediction rules with more age-specific predictors. Based on previous literature and current findings, there did not appear to be clinical predictors specifically for young febrile infants. Larger studies could look into this into more detail, but it might be that biomarker discovery studies, using novel biomarkers or proteomics platforms, might prove more fruitful.

Last, most of the included studies had a low QUADAS quality score. This was mostly inherent to the criteria of the verification of the outcome and performance of reference tests. This was also true for the validation cohorts. However, to minimise the risk of verification bias, all studies included a follow-up period to reduce the chance of missed serious infection as has been used in other diagnostic studies in this area.17 18 Two of the best performing CPMs (CPM 1 (36) and CPM 3 (38)) achieved the lowest risk of bias assessment in the QUADAS quality score.

Conclusion

This study provides timely evidence for the diagnostic role of CPMs in identifying serious bacterial infections in febrile young infants. Four CPMs, including the combination of clinical signs and symptoms, urine dipstick analysis and laboratory markers, are best suited to aid physicians in identifying serious bacterial infections, favouring specificity above sensitivity. We did not observe differences in performance of CPMs in children <3 months compared with those aged 3–12 months. However, limited sensitivities of the CPM observed in all validation cohorts could lead to missing serious bacterial infections in this cohort of young febrile infants. We advocate clinicians to use CPMs as an adjunct clinical tool when assessing the risk of SBI in febrile young infants.

Supplemental material

Supplemental material

References

View Abstract

Footnotes

  • Contributors EdVK: conceptualised and designed the study, was responsible for data collection at one of the three sites, carried out the initial analyses, drafted the initial manuscript and approved the final manuscript as submitted. BG, KM, RGN, FJS, SM, VG: coordinated and supervised data collection at one of the three sites, critically reviewed the manuscript and approved the final manuscript as submitted. EWS: supervised the analyses, reviewed and revised the manuscript, and approved the final manuscript as submitted. HAM: conceptualised and designed the study, supervised data collection at one of the three sites, reviewed and revised the manuscript, and approved the final manuscript as submitted. RO: conceptualised and designed the study, supervised data collection at one of the three sites, supervised the analyses, reviewed and revised the manuscript, and approved the final manuscript as submitted. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Disclaimer To our best knowledge this article is not accessible as full paper (pdf), only the reference (title) can be retrieved from https://repub.eur.nl/pub/93311, the online e-publication of the PhD thesis: Integrating Clinical Decision Making and Patient Care at the Paediatric Emergency Department–focusing on children with serious bacterial infections. Erasmus University Rotterdam. EdVK (2016 September 20).

  • Competing interests None declared.

  • Patient consent Parental/guardian consent obtained.

  • Ethics approval The study was approved by the local institutional medical ethics committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.