Abstract

Aims: To assess the reliability of mortality risk assessment using the Paediatric Risk of Mortality (PRISM) score and the Paediatric Index of Mortality (PIM) in daily practice.

Methods: Twenty seven physicians from eight tertiary paediatric intensive care units (PICUs) were asked to assess the severity of illness of 10 representative patients using the PRISM and PIM scores. Physicians were divided into three levels of experience: intensivists (>3 years PICU experience, n = 12), PICU fellows (6–30 months of PICU experience, n = 6), and residents (<6 months PICU experience, n = 9). This represents all large PICUs and about half of the paediatric intensivists and PICU fellows working in the Netherlands.

Results: Individual scores and predicted mortality risks for each patient varied widely. For PRISM scores the average intraclass correlation (ICC) was 0.51 (range 0.32–0.78), and the average kappa score 0.6 (range 0.28–0.87). For PIM scores the average ICC was 0.18 (range 0.08–0.46) and the average kappa score 0.53 (range 0.32–0.88). This variability occurred in both experienced and inexperienced physicians. The percentage of exact agreement ranged from 30% to 82% for PRISM scores and from 28 to 84% for PIM scores.

Conclusion: In daily practice severity of illness scoring using the PRISM and PIM risk adjustment systems is associated with wide variability. These differences could not be explained by the physician’s level of experience. Reliable assessment of PRISM and PIM scores requires rigorous specific training and strict adherence to guidelines. Consequently, assessment should probably be performed by a limited number of well trained professionals.