Article Text

Is the NICE traffic light system fit-for-purpose for children presenting with undifferentiated acute illness in primary care?
  1. Megan Hedd Blyth1,
  2. Rebecca Cannings-John2,
  3. Alastair D Hay3,
  4. Christopher C Butler4,
  5. Kathryn Hughes1
  1. 1 Division of Population Medicine, Cardiff University, Cardiff, UK
  2. 2 Centre for Trials Research, Cardiff University, Cardiff, UK
  3. 3 Division of Primary Health Care, University of Bristol, Bristol, UK
  4. 4 Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
  1. Correspondence to Dr Megan Hedd Blyth, Cardiff University, Cardiff CF10 3AT, UK; BlythM1{at}cardiff.ac.uk

Abstract

Background The National Institute of Clinical Excellence (NICE) traffic light system uses children’s symptoms and signs to categorise acute infections into red, amber and green. To our knowledge, no study has described the proportion of children with acute undifferentiated illness who fall into these categories in primary care, which is important since red and amber children are considered at higher risk of serious illness requiring urgent secondary care assessment.

Aim To estimate the proportion of acutely unwell children presenting to primary care classified by the NICE traffic light system as red, amber or green, and to describe their initial management.

Design and setting Secondary analysis of the Diagnosis of Urinary Tract infection in Young children prospective cohort study.

Method 6797 children under 5 years presenting to 225 general practices with acute undifferentiated illness were retrospectively mapped to the NICE traffic light system by a panel of general practitioners.

Results 6406 (94%) children were classified as NICE red (32%) or amber (62%) with 1.6% red and 0.3%, respectively, referred the same day for hospital assessment; and 46% and 31%, respectively, treated with antibiotics. The remaining 385 (6%) were classified green, with none referred and 27% treated with antibiotics. Results were robust to sensitivity analyses.

Conclusion The majority of children presenting to UK primary care with acute undifferentiated illness meet red or amber NICE traffic light criteria,with only 6% classified as low risk, making it unfit for use in general practice. Research is urgently needed to establish as triage system suitable for general practice.

  • paediatrics
  • primary health care

Data availability statement

No data are available. .

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

What is already known on this topic?

  • The National Institute of Clinical Excellence (NICE) traffic light system is promoted for use in primary and secondary care.

  • The traffic light system has been validated in secondary care where serious illness is more prevalent. There have been no studies in UK primary care.

What this study adds?

  • This secondary analysis of a large representative study shows the vast majority of children presenting to UK general practices with acute undifferentiated illnesses are classified as red or amber. We conclude that the NICE traffic light system is unfit for use in general practice, and that further research is urgently needed to develop and validate paediatric triage tools for use in this setting.

Background

Children account for up to 25% of all consultations in primary care.1 Identifying unwell children can be difficult as symptoms are often non-specific; there is an element of uncertainty due to communication difficulties with young children and the incidence of serious illness is low in primary care.2

It is important to be able to identify children who are at risk of a serious infection to avoid delay in treatment. The traffic light system was initially developed in the National Institute of Clinical Excellence (NICE) (2007) guideline ‘Fever in under 5s: assessment and initial management’ and updated in 2019.3 The system categorises signs and symptoms so that clinicians can determine whether children are at ‘low risk’ (green); ‘moderate risk’ (amber) or ‘high risk’ (red) of serious illness. It was based on the Yale Observation Scale4 and evidence-based reviews to identify symptoms and signs and the likely presence of any serious illness. Understanding the incidence of the risk categories in primary care is important in determining the burden of disease in this population.

It is important to understand how children are being managed, and what proportion of children are referred for hospital assessment. The NICE guidelines3 say that children categorised as ‘red’ should be referred for clinical assessment by a paediatric specialist. For children categorised as ‘amber’, general practitioners (GPs) should use clinical judgement to decide whether children should be referred to hospital. Often, it is difficult for GPs to decide which cases can be safely managed at home. Unpicking the cases to see the features that are most associated with admission and identification of serious illness is important.

Several studies have examined the NICE traffic light system in emergency departments (EDs).5 6 However, we could find no studies describing the incidence and management of children presenting to UK general practice categorised with NICE traffic light system. In the NICE guideline,3 a panel member conducted a survey of 157 consultations with children under 5 years in a GP practice in England to explore the incidence of red or amber features for children presenting with fever. Out of 157 consultations, 31 had amber features. No red features were recorded. Six of these children were referred to the paediatric assessment unit for specialist advice.

The use of the NICE traffic light system in general practice has been criticised as the research was based on hospital populations with a higher proportion of serious bacterial infections.7 The predictive value of the system in a UK general practice population with low risk of serious bacterial illness remains unknown.

The aim of this study was to describe the proportion of children presenting with acute illness to UK general practice who would be classified as red, amber or green by the NICE traffic light system and determine the initial management of children.

Methods

Study design and population

We analysed data from a cohort of children from ‘the DUTY (the Diagnosis of Urinary Tract infection in Young children) Study’.8 The DUTY Study was a multicentre, prospective observational study that recruited children under the age of 5 years who were being assessed in primary care for an acute, undifferentiated, non-traumatic illness of less than 28-day duration. Children were recruited from 233 primary care sites (GP surgeries, walk-in centres and EDs) in England and Wales. Full details of the study have been published elsewhere.8 9 The majority (6797, 94.9%) of children were recruited from general practices with 4.0% from children’s EDs and 1.1% from walk-in-centres. The study was designed to investigate the diagnostic features of urinary tract infection (UTI) in young children in primary care but recruited children who were generally unwell as well as those with a possible UTI. The children were recruited for the study between 2010 and 2012.

Children were included in DUTY if they were under the age of 5 years; had presented with an acute (≤28 days) illness and with at least one ‘constitutional’ symptoms or sign as a potential marker for UTI (lethargy/malaise/vomiting/irritability/poor feeding and failure to thrive) or urinary symptom. We included all children with acute illness in this study as we felt it represented what GPs would see in usual practice. Children were excluded if consent was not obtained; illness was longer than 28-day duration; they had taken any antibiotics in the previous 7 days; had a history of known bladder problems; were on immunosuppressant medication or had already been recruited into the study. For this study, we excluded the children who had presented to ED or walk-in centres.

Data collection

One hundred seven data points were recorded on a standardised report form. Information recorded included parent-reported symptoms and signs gathered from a full clinical examination. This included the treating clinician’s working diagnosis, initial management and whether the child was referred for immediate assessment in hospital.

Mapping exercise

Prior to analysis of the data, a panel of four clinicians studied the variables from the DUTY data together with the variables in the NICE traffic light system to map which DUTY variables could be used to determine NICE traffic light status. Both sets of variables were predetermined. If all four clinicians could not reach an agreement that a DUTY variable could be used to accurately represent a NICE traffic light variable, this variable was removed from the analysis (see table 1).

Table 1

Mapping of the NICE traffic light system to DUTY variables

Children were categorised into the traffic light system as follows:

  • Red=high risk if the child had at least one red criteria.

  • Amber=intermediate risk if the child had at least one amber criteria and no red criteria.

  • Green =low risk if the child had no amber or red criteria

  • Child not examined/missing.

Twenty-nine (64%) of the 45 NICE traffic light signs and symptoms were mapped to DUTY study variables. Fourteen (31%) NICE traffic light variables could not be directly mapped to DUTY Study variables (seven red, six amber and one green). These were mainly related to neurological or orthopaedic pathology. Two (4%) variables could not be matched because the panel of clinicians could not agree that the mapping was an accurate reflection of the NICE variable.

In order to ensure that mapping was appropriate, as a sensitivity analysis we used stricter categorisation of children:

  • Red=high risk if the child had two or more red criteria.

  • Amber=intermediate risk if the child had one red, or at least two amber criteria.

  • Green=low risk if the child had one amber or no red or amber criteria.

Analysis

Descriptive statistics were used to characterise the children (numbers (proportion), mean (SD) or median (25th–75th centiles)). Logistic regression models were run to examine associations between traffic light category categorisations and both treatment with antibiotics and referral to hospital for same day assessment. These models were rerun using the stricter categorisation of children for the sensitivity analysis. Parameter estimates are shown as ORs and 95% CIs.

IBM Statistics (SPSS V.25) was used to analyse the data.

Results

Within the study period (April 2010–April 2012), 6797 (94.9%) children under the age of 5 years presented to general practice and were included in the analysis. Table 2 shows the number of children meeting the criteria of the NICE traffic light system.

Table 2

Number of children meeting criteria of the NICE traffic light system

From the 6797 children, 2149 (32%) had at least one red flag symptom or sign, 4257 (62%) had at least one amber (and no red), the remaining 385 (6%) were green and 6 (0.1%) were missing the mapped DUTY fields to enable a traffic light category. The over-riding reason for a red child being classed as such is based on ‘response to social cues’ (n=1007), followed by ‘pallor’ (n=389). An additional 310 (14%) children had ‘no response to social cues’ in combination with another red symptom or sign. The vital sign variables (temperature, heart rate, respiratory rate, capillary refill time (CRT) and O2 saturation) contributed to 35 (0.5%) children being classified as ‘red’ (see table 3 and online supplemental appendix 1).

Table 3

Child characteristics, clinical observations on examination and management

The initial management of children according to traffic light system classification

The most common working diagnosis was upper respiratory tract infection followed by viral infection and then otitis media. A higher proportion of the red category were treated with an antibiotic compared with amber and green categories (46% vs 31% vs 27%, respectively), and a similar pattern observed in the proportion of children referred to hospital for same day assessment (1.6% vs 0.3% vs 0%, respectively). Antibiotic prescriptions were statistically significantly higher in the children classified as red when compared with those classified as green (OR=2.33 (95% CI=1.81 to 3.01), p<0.001; table 3). Forty-one children (1.9%) were referred to hospital for same day assessment: 30 red and 11 amber. Sensitivity and specificity for a referral for same day assessment were 73% and 80%, respectively. The most common symptoms that these children had were decreased activity (n=23), low oxygen saturation (n=11), tachycardia (n=9) and prolonged CRT (n=9).

Using a stricter definition of red high-risk symptoms, we found that 6% (n=372) of children were categorised as high risk and 50% (n=3380) as amber. Similarly, a higher proportion of the red category were treated with an antibiotic and referred to hospital compared with amber and green categories (table 3). Again, antibiotic prescriptions were significantly higher in the children classified as red and amber when compared with those classified as green (OR=3.38 (95% CI=2.67 to 4.27), p<0.001 and OR=1.65 (95% CI=1.48 to 1.85), p<0.001, respectively; table 3). This stricter definition resulted in a higher proportion of those categorised as red or amber referred for same day hospital assessment (6.2% and 0.5% compared with 1.6% and 0.3%, respectively). The proportion of children referred for same day hospital assessment was significantly higher in the children classified as red when compared with those classified as green (OR=25.71 (95% CI=10.79 to 61.31), p<0.001; table 3). Sensitivity for referred for same day assessment was much reduced using this stricter definition (49%) but specificity was higher (95%).

Discussion

Summary of main findings

This study describes a large cohort of 6797 acutely ill children less than 5 years old presenting to UK general practices. There were 2175 (32%) with at least one red flag symptom according to the NICE traffic light system; 4231 (62%) with at least one amber (and no red), and only 6% were green or uncategorised. Of the children with at least one red symptom or sign, only 1.6% were referred for same day assessment. Of the children with no red symptoms or signs, 0.3% were referred for same day hospital assessment. The sensitivity analysis increased the threshold for diagnosis of high risk (two or more red criteria) however still identified 372 (6%) children as red (high risk) and 3380 (50%) of children as amber.

Strengths and weaknesses of the study

This is a large cohort with well-documented symptoms and signs, including heart rate, respiratory rate, temperature and CRT which are important objective variables in the traffic light system but are infrequently measured in routine general practice.10

A further strength of this study is that DUTY recruited children with suspected acute infection. The NICE guideline is regarding the assessment of ‘febrile children’; however, we believe that including all acutely ill children is more representative of normal practice and gives a better insight into general practice.

A limitation of this study was that the NICE variables were not prospectively assessed and the DUTY variables did not perfectly match. We may have underestimated the numbers of amber and red as some of the variables could not be mapped to DUTY Study variables (eg, non-blanching rash). The NICE traffic light system was designed for children with no obvious focus of infection, and children considered to have specific infections (eg, tonsillitis) by GPs were not excluded from our study. However, other studies validating the traffic light system in other settings have not excluded children on this basis.5 6

Although recruiting acutely ill children, the DUTY Study was primarily focused on diagnosis of UTIs and this could have led to the GP working diagnosis of UTI being higher than usual. Some acutely ill children may have been excluded if they did not fit the DUTY inclusion criteria. Very ill children may have been less likely to participate in the study. Immunisation schedules have also changed since the DUTY data were obtained.

Without hospital admission data, we could not tell whether the paediatric team admitted the child to hospital or whether there was a diagnosis of serious bacterial infection.

Results in relation to other studies, discussing important differences in results

To our knowledge, this is the only UK study describing the incidence of low, intermediate or high risk of serious infection using the NICE traffic light system in general practice, and much larger than the only other European study which we are aware of.

An Australian retrospective study5 conducted in an ED (N=15 781) found that 25.8% of children under the age of 5 years met the red category of the traffic light system which is similar to our finding of 32%; 46.7% met amber criteria (compared with our 62%) and 27.5% met green criteria (compared with our 6%). We had a similar proportion of children categorised as red in our study but lower numbers categorised as green. This is surprising as the ED population has a higher prevalence of serious illness.

A prospective validation study in UK ED (n=700) found that 72.6% of children diagnosed with a mild infection had at least one amber or red symptom, whereas 7.4% of children with a serious infection had no amber or red features.6

A systematic review11 concluded that the NICE guideline traffic light system had excellent sensitivity (100%) in a low prevalence setting though specificity was 1.03%. This was based on only one study of 506 children less than 6 years old in Belgian GP out-of-hours care.12 In this study, nearly all the children (99%) had a red or amber sign. Our much larger study of 6797 children also found a high percentage (94%) of children meeting red or amber criteria in primary care.

Implications for clinicians, policymakers and future research

This study demonstrates high numbers of children who would be classified as red (32%). If guidelines were followed, this would lead to a high number of children referred for same day hospital assessment.

There is a very high proportion of children who meet the amber criteria of the NICE traffic light system (62%). The current guideline advice is that referral for specialist assessment is down to the clinical judgement of the assessing clinician. This makes it a less helpful guideline for GPs with no direct guidance.

Our study demonstrated a low number of patients (6%) classified as low risk (green) for serious illness in primary care. Primary care however is considered to have a low prevalence of serious illness. Clinical prediction rules should have high sensitivity to identify those who are at risk however there are costs in terms of referral or admission rates to additionally be considered.

Reclassifying the high-risk, intermediate-risk and low-risk categories resulted in less children classified as high risk but only identified half of those who GPs sent for same day hospital assessment.

The guideline is complex and many of the symptoms and signs are subjective. We found the most common reason for a child to be classified as high risk (red) was their response to social cues. There may be a lot of variation with how doctors or parents might define this. Similarly, with conscious levels, definitions may vary significantly between parents and clinicians. As clinicians, we felt uncomfortable that waking only with prolonged stimulation would only meet amber criteria. This view has been shared.7

Our research needs to be taken further, linking primary care data with hospital data, to identify which children were admitted to hospital or who re-presented to primary care with serious bacterial illness and relation to traffic light system categories.

Other scoring systems developed for the identification of serious illness in children in other settings should also be validated to see if they would be suitable for use in general practice. If these are not suitable, a new triage system fit for use in UK general practice needs to be urgently developed. Ideally, this would be a simple scoring system based on objective measurements, standardised with those used in EDs and hospital. However, it is essential that any scoring system is validated for use in general practice.

Conclusion

The vast majority of children presenting to UK primary care with acute undifferentiated illness meet amber or red NICE traffic light criteria. The NICE traffic light system is not fit for use as it will overcall the need for hospital assessment.

Data availability statement

No data are available. .

Ethics statements

Patient consent for publication

Ethics approval

Ethical approval was granted by the South West Southmead Research Ethics Committee for the DUTY Study (ref #09/H0102/64).

Acknowledgments

We would like to thank the clinicians, research team and participants of the DUTY Study.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors MHB is the lead author and contributed to the conception and design of the work; data analysis and interpretation; drafting of the article; critical revision of the article and final approval of the version to be published. RC-J contributed to the conception and design of the work, data analysis, drafting of the article and final approval. KH contributed to the conception and design of the work, data analysis and interpretation, drafting, supervision and final approval. ADH contributed to data collection, critical revision of the article and final approval of the version to be published. CCB contributed to data collection and final approval of the version to be published.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.