Performance at optimal score
| PEWS (AUROC rank) | Optimal score/maximum score | Case patients (n=297) | Control patients (n=311) | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | LR +ve (95% CI) | LR −ve (95% CI) | Median hours to event (inter-quartile range) | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score at or above threshold (TP) | Score below threshold (FN) | Score at or above threshold (FP) | Score below threshold (TN) | |||||||||
| Scoring systems | ||||||||||||
| Cardiff and Vale PEWS (1) | 3/8 | 238 | 59 | 44 | 267 | 0.80 (0.75 to 0.84) | 0.86 (0.81 to 0.89) | 0.84 (0.80 to 0.88) | 0.82 (0.77 to 0.86) | 5.66 (4.28 to 7.49) | 0.23 (0.18 to 0.29) | 26.60 (11.57 to 42.19) |
| Bedside PEWS (2) | 6/26 | 215 | 82 | 35 | 276 | 0.72 (0.67 to 0.77) | 0.89 (0.85 to 0.92) | 0.860 (0.81 to 0.90) | 0.77 (0.72 to 0.81) | 6.43 (4.67 to 8.86) | 0.311 (0.26 to 0.37) | 26.25 (13.94 to 43.29) |
| Modified PEWS III (3) | 7/28 | 204 | 93 | 28 | 283 | 0.69 (0.63 to 0.74) | 0.91 (0.87 to 0.94) | 0.879 (0.83 to 0.92) | 0.75 (0.71 to 0.80) | 7.63 (5.31 to 10.95) | 0.34 (0.29 to 0.41) | 21.61 (12.42 to 40.10) |
| Modified PEWS II (4) | 6/26 | 228 | 69 | 63 | 248 | 0.83 (0.78 to 0.87) | 0.71 (0.66 to 0.76) | 0.731 (0.68 to 0.78) | 0.81 (0.76 to 0.85) | 2.85 (2.38 to 3.42) | 0.25 (0.19 to 0.32) | 36.57 (16.57 to 46.00) |
| Children's Early Warning Tool (4) | 4/24 | 245 | 52 | 90 | 221 | 0.77 (0.72 to 0.81) | 0.80 (0.75 to 0.84) | 0.784 (0.73 to 0.83) | 0.78 (0.73 to 0.83) | 3.79 (3.01 to 4.77) | 0.29 (0.24 to 0.36) | 37.66 (22.39 to 44.74) |
| PEWS I (6) | 3/10 | 247 | 50 | 99 | 212 | 0.83 (0.78 to 0.87) | 0.68 (0.63 to 0.73) | 0.714 (0.66 to 0.76) | 0.81 (0.76 to 0.85) | 2.61 (2.20 to 3.10) | 0.25 (0.19 to 0.32) | 24.00 (11.23 –44.13) |
| NHSI PEWS (7) | 2/7 | 247 | 50 | 108 | 203 | 0.83 (0.78 to 0.87) | 0.65 (0.60 to 0.71) | 0.696 (0.65 to 0.74) | 0.80 (0.75 to 085) | 2.40 (2.04 to 2.81) | 0.26 (0.20 to 0.33) | 29.90 (14.57 to 43.63) |
| PEWS system score (7) | 9/32 | 207 | 90 | 78 | 233 | 0.70 (0.64 to 0.75) | 0.75 (0.70 to 0.80) | 0.726 (0.67 to 0.78) | 0.72 (0.70 to 0.77) | 2.78 (2.26 to 3.42) | 0.40 (0.34 to 0.48) | 39.50 (17.43 to 46.57) |
| PEWS II (9) | 4/13 | 181 | 116 | 50 | 261 | 0.61 (0.55 to 0.67) | 0.84 (0.79 to 0.88) | 0.784 (0.72 to 0.83) | 0.69 (0.64 to 0.74) | 3.79 (2.89 to 4.96) | 0.47 (0.40 to 0.54) | 26.00 (11.75 to 41.58) |
| CEWS (9) | 4/21 | 179 | 118 | 48 | 263 | 0.60 (0.54 to 0.66) | 0.85 (0.80 to 0.88) | 0.789 (0.73 to 0.84) | 0.69 (0.64 to 0.74) | 3.91 (2.96 to 5.15) | 0.47 (0.41 to 0.54) | 21.05 (10.38 to 40.12) |
| ITAT score (11) | 3/8 | 202 | 95 | 82 | 229 | 0.68 (0.62 to 0.73) | 0.74 (0.68 to 0.78) | 0.711 (0.65 to 0.76) | 0.71 (0.65 to 0.76) | 2.58 (2.11 to 3.16) | 0.43 (0.37 to 0.51) | 28.95 (14.70 to 43.96) |
| Modified PEWS I (12) | 4/9 | 135 | 162 | 31 | 280 | 0.46 (0.40 to 0.51) | 0.90 (0.86 to 0.93) | 0.813 (0.74 to 0.87) | 0.63 (0.59 to 0.68) | 4.56 (3.19 to 6.51) | 0.61 (0.55 to 0.67) | 17.00 (6.75 to 35.68) |
| Trigger systems | ||||||||||||
| THSC MET calling criteria (13) | 1 or more triggers | 267 | 30 | 138 | 173 | 0.90 (0.86 to 0.93) | 0.56 (0.50 to 0.61) | 0.66 (0.61 to 0.71) | 0.85 (0.79 to 0.90) | 2.03 (1.78 to 2.31) | 0.18 (0.13 to 0.26) | 27.90 (13.74 to 42.37) |
| MET activation criteria I (14) | 1 or more triggers | 276 | 21 | 158 | 153 | 0.93 (0.89 to 0.96) | 0.49 (0.44 to 0.55) | 0.64 (0.59 to 0.68) | 0.88 (0.82 to 0.92) | 1.83 (1.63 to 2.05) | 0.14 (0.10 to 0.22) | 33.87 (18.76 to 45.52) |
| MET activation criteria II (14) | 1 or more triggers | 276 | 21 | 158 | 153 | 0.923 (0.89 to 0.96) | 0.49 (0.44 to 0.55) | 0.64 (0.59 to 0.68) | 0.88 (0.82 to 0.92) | 1.83 (1.63 to 2.05) | 0.14 (0.10 to 0.22) | 33.92 (18.76 to 45.52) |
| PMET triggers I (14) | 1 or more triggers | 273 | 24 | 157 | 154 | 0.92 (0.88 to 0.95) | 0.50 (0.44 to 0.55) | 0.64 (0.59 to 0.68) | 0.87 (0.80 to 0.68) | 1.82 (1.62 to 2.04) | 0.16 (0.11 to 0.24) | 33.25 (16.90 to 45.42) |
| Modified Bristol PEWS (17) | 1 or more triggers | 286 | 11 | 223 | 88 | 0.96 (0.93 to 0.98) | 0.28 (0.23 to 0.34) | 0.56 (0.52 to 0.61) | 0.90 (0.81 to 0.94) | 1.34 (1.25 to 1.45) | 0.13 (0.07 to 0.24) | 39.83 (23.82 to 46.25) |
| Bristol PEWS (17) | 1 or more triggers | 285 | 12 | 223 | 88 | 0.96 (0.93 to 0.98) | 0.28 (0.23 to 0.34) | 0.56 (0.52 to 0.61) | 0.88 (0.80 to 0.93) | 1.34 (1.24 to 1.44) | 0.14 (0.08 to 0.25) | 39.73 (23.45 to 46.25) |
The optimal score for trigger systems was determined as 1. The optimal score for scoring systems was determined as the cut-point which demonstrated the maximum value for the sum of the sensitivity and specificity, as described by Edwards et al.18 As such, this differed between different scoring systems.
The hours to event was calculated as the number of hours between the case patient's first achieving the optimal score/trigger to the occurrence of the critical deterioration event.
AUROC, area under the receiver operator characteristic curve; CEWS, Children's Early Warning Score; FN, false negative; FP, false positive; ITAT, inpatient triage, assessment and treatment score; LR +ve, positive likelihood ratio; LR –ve, negative likelihood ratio; MET, Medical Emergency Team; NHSI, NHS Institute; NPV, negative predictive value; PEWS, paediatric early warning system; PMET, Paediatric Medical Emergency Team; PPV, positive predictive value; THSC, Toronto Hospital for Sick Children; TN, true negative; TP, true positive.