Background The Manchester Triage System (MTS) determines an inappropriately low level of urgency (undertriage) to a minority of children. The aim of the study was to assess the clinical severity of undertriaged patients in the MTS and to define the determinants of undertriage.
Methods Patients who had attended the emergency department (ED) were triaged according to the MTS. Undertriage was defined as a ‘low urgent’ classification (levels 3, 4 and 5) under the MTS; as a ‘high urgent’ classification (levels 1 and 2) under an independent reference standard based on abnormal vital signs (level 1), potentially life-threatening conditions (level 2), and a combination of resource use, hospitalisation, and follow-up for the three lowest urgency levels.
In an expert meeting, three experienced paediatricians used a standardised format to determine the clinical severity. The clinical severity had been expressed by possible consequences of treatment delay caused by undertriage, such as the use of more interventions and diagnostics, longer hospitalisation, complications, morbidity, and mortality.
In a prospective observational study we used logistic regression analysis to assess predictors for undertriage.
Results In total, 0.9% (119/13,408) of the patients were undertriaged. In 53% (63/119) of these patients, experts considered undertriage as clinically severe. In 89% (56/63) of these patients the high reference urgency was determined on the basis of abnormal vital signs. The prospective observational study showed undertriage was more likely in infants (especially those younger than three months), and in children assigned to the MTS ‘unwell child’ flowchart (adjusted OR<3 months 4.2, 95% CI 2.3 to 7.7 and adjusted ORunwell child 11.1, 95% CI 5.5 to 22.3).
Conclusion Undertriage is infrequent, but can have serious clinical consequences. To reduce significant undertriage, the authors recommend a systematic assessment of vital signs in all children.
Statistics from Altmetric.com
Emergency departments (EDs) need triage systems to prioritise patients.1 The Manchester triage system (MTS) is a five-level triage system that allocates a clinical priority to adult and paediatric patients.2 3 It was introduced in the UK in 1996 and its translated versions have been adapted and are currently used around the world.3 The MTS applies 52 flowcharts which represent presenting complaints such as ‘worried parent’, ‘limping child’ and ‘shortness of breath in children’. Each flowchart contains general as well as problem-specific signs and symptoms (discriminators) that discriminate between the different urgency categories. The selection of a discriminator allocates the patient to one of five urgency levels, each indicating the maximum time a patient should wait before seeing a physician.2 3 Patients allocated to urgency level ‘immediate’ demand immediate medical evaluation, ‘very urgent’ need evaluation within 10 min, ‘urgent’ within 60 min, ‘standard’ within 120 min and ‘non-urgent’ patients can wait for up to 240 min prior to clinical assessment.
What is already known on this topic
▶ In paediatric emergency care the Manchester triage system (MTS) showed moderate validity with sensitivity of 63% and specificity of 79%.
▶ The MTS errs on the safe side because the proportion of overtriage is much larger than undertriage.
▶ Undertriage was considered more severe for the individual patient.
What this study adds
▶ Severe undertriage showed low prevalence in triaged children. Eighty-nine per cent of these children had abnormal vital signs, which the MTS had not recognised.
▶ The experts believe that treatment delay caused by undertriage could have serious consequences.
▶ The authors recommend a systematic assessment of vital signs in all children.
In a previous study conducted in a paediatric setting, the authors expressed sensitivity and specificity of the MTS as the capacity to distinguish between high (levels 1 and 2) and low urgency (levels 3, 4 and 5) defined by an independent five-level reference standard.8 The sensitivity of the MTS to detect highly urgent children was 63% and the specificity 79%. The agreement with the reference standard was 34%, with overtriage in 54% and undertriage in 12% (mostly by one category).8 Although overtriage can cause increased waiting times for truly urgent patients because of overcrowding, there is no direct harm for the overtriaged patient.9 To date, no study has been performed examining the consequences of undertriage for patients who had attended EDs, but it is expected that undertriage might increase morbidity and mortality.10 11 Therefore this study focuses on the possible consequences of undertriage and how to reduce these in clinical practice. The aim is to assess clinical severity of undertriage by expert opinion and to define determinants for undertriage in paediatric patients at the ED in a large observational study.
This study contained two parts, an expert opinion (case study) and a prospective observational study.
In the fist part, experienced experts discussed the possible impact of treatment delay in undertriaged cases to determine the clinical severity.
In the second part, a logistic regression analysis was performed in a large prospective cohort to define determinants of undertriage. The study was approved by the institutional medical ethical committee; the requirement for informed consent was waived.
Setting and selection of participants
The Erasmus MC–Sophia Children's Hospital in Rotterdam, The Netherlands is a university, inner-city hospital with a specific paediatric ED that receives 9000 patient visits a year. The MTS was implemented in 2005. The Haga Hospital–Juliana Children's Hospital in The Hague is a general teaching hospital with approximately 30 000 patient visits a year of which 15 000 are paediatric visits. In the Haga Hospital, the MTS was implemented in 2003. Both hospitals are comparable when it comes to availability of diagnostic resources and specialties and they have the same opening hours (24 h).
We included children aged 0–16 who had attended the ED at Haga Hospital–Juliana Children's Hospital between 1 January 2006 and 1 August 2006 and the ED at Erasmus MC–Sophia Children's Hospital between 1 January 2006 and 1 February 2007. Trained nurses experienced in both paediatric nursing and ED nursing (median experience 10 years (IQR 7–14 years)) triaged the patients with the official Dutch translation of the first edition of the MTS.2
Prior to the study, a reference standard was defined on the basis of the literature and expert opinion.8 12 The reference standard consisted of five urgency levels, which estimate patients' true urgency. Patients were considered to be level 1 if their vital signs (heart rate, blood pressure, pulse oximetry and respiratory rate) were abnormal according to the Paediatric Risk of Mortality III (PRISM III) score (see table 1)13 or in cases of hyperpyrexia (temperature >41°C) or altered level of consciousness. If vital signs were not recorded, they were assumed to be normal.
Level 2 was allocated to patients with normal vital signs, but potentially life-threatening conditions diagnosed by the paediatrician at the end of the ED visit. Most of these conditions are described in the Advanced Paediatric Life Support workbook as emergent conditions.14 In a systematic review, it was suggested that children with an apparent life-threatening event (ALTE) be monitored for 24 h. An ALTE is defined as an episode characterised by the combination of apnoea, colour change, marked change in muscle tone, choking, or gagging. Therefore ALTE was added as a level 2 condition to the reference standard.15
Patients were allocated to levels 3 or 4 depending on the combination of diagnostic and therapeutic resource use, hospitalisation and if a follow-up visit was scheduled. Resource use was associated with the urgency levels of the Emergency Severity Index.16 Category 5 was assigned if no resources were required. A classification matrix of the reference standard and detailed definitions of the reference urgencies have been published before.8 12 The reference standard was allocated independently of the MTS by using a computerised application of the classification matrix.
Undertriage was defined as a ‘low urgent’ classification (levels 3, 4 and 5) under the MTS; as a ‘high urgent’ classification (levels 1 and 2) under the reference standard; or at least a difference of two urgency levels between the MTS level and the reference urgency level. In the event of undertriaged cases presenting with similar problems (MTS flowchart and discriminator, and reference urgency classification) and within similar age categories, one case was randomly selected as a representative case for the expert meeting.
Three paediatricians evaluated the undertriaged cases at a meeting using anonymous ED forms. The ED forms included information on the assigned MTS and reference urgency levels and clinical notes about presenting symptom(s), history, physical examination, working diagnosis, therapy, diagnostics and suggested follow-up.
First, the experts scored the clinical severity of undertriage on a scale from 0 to 10. Zero represented ‘the absolute minimum severity’ and 10 ‘the maximum severity of undertriage’. Second, the experts evaluated the possible clinical consequences of undertriage by using a six-item questionnaire. For each case the experts assessed the probability of more interventions or diagnostics, a longer duration of hospitalisation, complications, and long-term morbidity and mortality because of treatment delay when the MTS protocol was followed instead of maximum waiting times assigned by the reference standard. The questionnaire was designed in a ‘yes and no’ format.
The experts had been working as paediatricians for at least 15 years and had clinical experience in emergency medicine. They were working at the Erasmus MC–Sophia Children's Hospital and the Haga Hospital–Juliana Children's Hospital at the time of the study.
The study used a computerised version of the official Dutch translation of the MTS.8 Data were collected on patients' characteristics, the selected flowcharts/discriminators and the MTS urgency levels.
Nurses and physicians used structured electronic and paper ED forms and they recorded data on vital signs measured directly after triage, diagnosis, diagnostic resources, therapeutic intervention and follow-up. Trained medical students entered these data in a separate database independently of triage outcome by using SPSS Data entry version 4. The data were checked for inconsistencies and outliers. Data on laboratory tests were obtained from the hospital information system.
The study assumed that the experts would have scored similar cases in an equal manner. The results of the discussed cases were therefore multiplied by the number of similar cases.
Undertriage was defined as severe if the experts' severity score was high (≥7) or if the patient could experience at least one consequence of undertriage. A univariate and multivariable logistic regression analysis was subsequently performed to define determinants for undertriage.
Age, gender and frequently assigned MTS flowcharts specific to the patient's presenting problem were considered as possible determinants of undertriage. Because the relation between age and risk of undertriage was non-linear, age was categorised as younger than 3 months, 3–11 months, 1–4 years, 4–8 years, and older than 8 years. SPSS version 15.0 was used for statistical analysis.
In total, 17 600 children attended the EDs. Complete data of MTS triage and reference standard were available for 13 408 children. Two per cent (189/9582) of the children triaged as ‘urgent’, ‘standard’ or ‘non-urgent’ according to the MTS were assigned to reference urgency levels 1 or 2.
In 37% (70/189) the difference between the MTS level and the reference urgency level was only one level. In total, 0.9% (119) undertriaged cases remained for analysis (figure 1). If patients had similar medical problems, one was randomly selected for evaluation in the expert meeting. This resulted in 20 cases for discussion by the expert panel.
Clinical severity of undertriage
Table 2 shows the items discussed by the experts. Undertriage was considered severe in 53% (63/119) of the undertriaged patients, and 89% (56/63) of these severely undertriaged patients had a high reference urgency level because of abnormal vital signs (heart rate, blood pressure, pulse oximetry, respiratory rate).
According to the experts, 50% (60/119) of the undertriaged patients could potentially have experienced at least one consequence of undertriage; 45% (54/119) might have undergone more interventions; 40% (48/119) might have experienced more diagnostic investigations because of treatment delay; 34% (40/119) would have been likely to have complications; 6% (7/119) might have been hospitalised for longer; 11% (13/119) might have experienced long-term morbidity; and 3% (3/119) might have died because of treatment delay caused by undertriage. Fifty-one of the undertriaged patients had an ALTE. All these patients were considered non-severe by the experts (severity score 1 and no consequences caused by undertriage).
Determinants of undertriage
The patients assigned to the MTS ‘unwell child’ flowchart had an increased OR of 10.7 (95% CI 5.4 to 20.9) for undertriage when compared with patients assigned to the MTS ‘general’ flowchart (table 3). This ORunwell child was adjusted for age in a multivariate analysis. This raised the adjusted ORunwell child to 11.1 (95% CI 5.5 to 22.3).
Infants (≤12 months) were at higher risk of undertriage than children aged 8–16 years (OR<3 months 9.6, 95% CI 5.6 to 16.7 and OR3–11 months 2.6, 95% CI 1.4 to 4.9). When these ORs were adjusted for flowcharts that were frequently used in undertriaged cases, children under the age of 3 months still had a higher adjusted OR<3 months of 4.2 (95% CI 2.3 to 7.7) for undertriage.
This study aimed to assess the clinical severity of treatment delay caused by undertriage and to define determinants for undertriage in paediatric patients at the ED. Undertriage was assessed by comparing the MTS with an independent reference standard. In total, 0.9% (119) of patients were undertriaged. These undertriaged cases were discussed by experts who considered 53% (63/119) as clinically severe, and that 50% (60/119) might experience at least one consequence because of undertriage. Eighty-nine per cent (56/63) of these clinically severe undertriaged patients had abnormal vital signs.
Rather than measuring abnormal vital signs, the MTS uses the following discriminators to describe their symptoms: shock, inadequate breathing, compromised airway and unresponsiveness. Because not all patients with abnormal vital signs were assigned to one of these discriminators, they were not always recognised as highly urgent patients.8 This is consistent with Cooke and Jinks,5 who demonstrated that misclassification in the MTS was due to the presence of abnormal vital signs in adults. If pulse oximetry had been part of the triage assessment, they would have been able to assign three patients with chest pain to the correct urgency level. This suggests that the MTS should include vital signs to reduce undertriage.
One of the findings of this study was relevant to the definition of the reference standard. Reference level 2 is defined by the presence of potential life-threatening conditions. One of these conditions is an ALTE. In the expert opinion study the experts agreed that increased waiting time in patients with an ALTE does not influence patients' outcome. ALTE was skipped from the list of life-threatening conditions in the reference standard (urgency level 2).
In the observational part of the study, children younger than 3 months and those assigned to the MTS ‘unwell child’ flowchart were shown to be more likely to be undertriaged than other patients. Although ‘unwell appearance’ is an important predictor for serious infections in children,17 it is not a very sensitive and objective clinical feature.18 Children who had been assigned to the MTS ‘unwell child’ flowchart could have attended the ED with a variety of problems, while the flowchart mainly focuses on children with infectious diseases based on the discriminators fever, signs of meningitis, purpura and signs of dehydration. As a result children with conditions other than infectious problems are difficult to assign to an urgency level and therefore undertriage may occur more frequently in the MTS ‘unwell child’ flowchart than other flowcharts.
Infants are also more difficult to allocate to urgency levels because they have non-specific signs and symptoms at presentation for several diseases, for example, fever.19 In the revisions of the paediatric Canadian Triage and Acuity Scale, systematic assessment of level of consciousness, respiratory rate, heart rate and circulatory status were recommended for infants and young children to assist with the assessment of severity of illness.19
Systematic assessment of vital signs for selected groups of children (children younger than 3 months) could prevent 21% (13/63) of clinically severe undertriage and the workload would increase to 8.0% extra measurements (1072/13 408 of patients). The number needed to treat (NNT) is 82 (1072/13). If we measure vital signs in the MTS ‘unwell child’ flowchart as well, the NNT decreases to 60 (1200/20). Only systematic assessment of the vital signs in all children could prevent clinically severe undertriage in nearly all children (89%) (NNT 213).
Although the value of expert opinion as evidence in biomedical research has been criticised, it remains the best available method for evaluating the consequences of undertriage for individual patients. To improve the validity of the judgement of cases, standardised questionnaires were used and paediatricians experienced in emergency medicine were selected as experts.
To determine undertriage, a measure for patients' ‘true’ urgency was needed. The reference standard is based on patients' characteristics at ED presentation and at the end of their ED visit. Characteristics gathered at the end of consultation might be less suitable to define urgency because patients' conditions might change over time. However, assessment of true urgency requires more information than available at presentation (triage). The reference standard used in this study is the best available approximation of an ideal reference standard.20
Despite these limitations, the authors believe a reference standard is a reasonable best approach to determine the urgency with which particular patients should be seen and assessed.20
Although serious undertriage in the MTS occurs in very small numbers of patients (approximately 1%), the experts believed that it could have serious consequences. To reduce significant undertriage, the authors recommend a systematic assessment of vital signs in all children.
Funding Netherlands Organization for Health Research and Development (ZonMw), and Erasmus University Medical Centre, Rotterdam, The Netherlands.
Role of sponsor None of the granting agencies listed had any role in the design and conduct of the interpretation of the data; or preparation, review, or approval of the manuscript. Researchers are independent of the sponsor.
Competing interests None.
Ethics approval This study was conducted with the approval of the Erasmus Medical Centre, Rotterdam, The Netherlands.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.