Objective To test the predictability of the National Health Service Institute for Innovation and Improvement (NHSIII) Paediatric Early Warning System (PEWS) score to identify children at risk of developing critical illness.

Design Cohort study.

Setting Admissions to all paediatric wards at the University Hospital of Wales between 1 December 2005 and 30 November 2006.

Outcome measures Unscheduled paediatric high dependency unit (PHDU) admission, paediatric intensive care unit (PICU) admission and death.

Results There were 9075 clinical observations from 1000 children. An NHSIII PEWS score of 2 or more, which triggers review, has a sensitivity of 73.2% (95% CI 62.2% to 82.4%), specificity of 75.2% (95% CI 74.3% to 76.1%), positive predictive value (PPV) of 2.6% (95% CI 2.0% to 3.4%), negative predictive value of 99.7% (95% CI 99.5% to 99.8%) and positive likelihood ratio of 3.0 (95% CI 2.6 to 3.4) for predicting PHDU admission, PICU admission or death. Six (37.5%) of the 16 children with an adverse outcome did not have an abnormal NHSIII PEWS score. The area under the receiver operating characteristic curve for the NHSIII PEWS score was 0.83 (95% CI 0.77 to 0.88).

Conclusions The NHSIII PEWS has a low PPV and its full implementation would result in a large number of false positive triggers. The issue with PEWS scores or triggers is neither their sensitivity nor children with high scores which require clinical interventions who are not ‘false positives’; but their low specificity and low PPV arising from the large number of children with low but raised scores.