Objective: To elucidate physicians' ability to correctly diagnose infection in critically ill children in three different situations: 1) post hoc adjudication (e.g., ward rounds, outcome determination in surveillance studies or controlled trials), 2) when decisions must be made (e.g., sepsis workup in suspected infection), c) and applying suggested adult consensus conference definitions in children.
Method: Appraisal of two previously published studies and a data simulation model.
Setting: Data of the reviewed studies were obtained from a multidisciplinary neonatal and a pediatric intensive care unit in tertiary hospitals.
Interventions: None.
Main results: In the first study reviewed, the post hoc adjudication of 167 consecutive cases of suspected infection was carried out by a fifth-year medical student and three senior consultants. The agreement of the three experts beyond chance in the 119 episodes not classifiable unanimously by a priori defined criteria into proven sepsis or no infection was poor. In the second study reviewed, the physicians provided daily predictions of the likelihood of infection (pretest probability) in premature infants and critically ill children (2567 hospitalization days). Estimated pretest probabilities provided at the time of sepsis workup showed a remarkable predictive accuracy (area under the receiver operating characteristic curve, 0.85). In the simulation model, in which catheter-related sepsis was assumed, correct classification of patients from a central and a peripheral culture decreased to 56% when a sensitivity of 70%-80% was assumed for blood cultures and amounted to 15% only when a sensitivity of 30%-50% was imputed.
Conclusion: Misclassification is a serious threat in post hoc adjudication of episodes or when consensus definitions rely on the application of criteria with imperfect sensitivity (e.g., the positivity of blood cultures in premature infants or children). This underscores the need to use probability-based categorizations such as probable and possible infection.