The use of appropriate pain scoring tools has been recommended in recent guidelines published by both the Royal College of Nursing² and the Royal College of Paediatrics and Child Health.³ The British Association of Emergency Medicine standard is for patients in severe pain (as determined by a pain score of 7–10) to have the prescription of analgesia within 20 minutes of arrival.⁴ However, studies have suggested that children’s pain management in A&E departments is suboptimal.⁵ A large variety of paediatric pain scoring tools have been developed using different methods and designed for children with different cognitive ability, with no single score appropriate for all situations and age groups.⁶ Self-assessment tools are most commonly used. However, such tools were often originally developed for use in children experiencing postoperative or recurrent pain, for example, sickle cell crises.⁷ In these situations, there is an opportunity to explain how to use the tool prior to the painful experience. This clearly is not possible in an acute injury presenting to an A&E department which demands urgent assessment to allow prompt treatment. In addition, the stresses and anxiety that occur with an unexpected painful experience disadvantage accurate self-assessment.⁸ The use of complex composite tools is confusing in this situation.

Formal pain tools and protocols have been shown to improve pain management in A&E,⁹ although levels of pain scoring still remain relatively low. Several visual analogue scale tools, many using a reference face scale, have been developed. Of these the Wong-Baker Faces Pain Rating Scale¹ is probably best known and most widely used. The Toddler Pre-schooler Postoperative Pain Score (TPPPS)¹⁰ is a behavioural tool which identifies and quantifies pain behaviours.

This tool has been tested in the A&E setting, but it is inappropriate because, as the name implies, it is validated only for preschool children in a different setting.¹¹

Pain scoring commenced in the A&E department at Alder Hey Children’s Hospital in 1998 using a pain ladder, a composite tool including self-assessment, the Manchester Triage Pain Scale.¹² This scale has never been formally validated and had a low level of uptake. Senior nursing staff felt that it was an unreliable method of assessing the level of a child’s pain at triage and in addition that it was unhelpful in determining the level of analgesia required. Our experience was that children either quickly picked a value at random which often appeared to be out of proportion to the nature of the injury, appeared confused, or deferred to their parents for an answer. Self-assessment scores appeared to be more relevant after the initial presentation and are widely and successfully used in the inpatient setting in our hospital. For these reasons, we felt that this type of tool was not the most appropriate in A&E.

Development of tool

The Alder Hey Paediatric Triage Score(AHTPS), was developed specifically for use with children at triage in A&E departments (table 1). It is scored using observational and behavioural characteristics for assessment. Inquiry on pain related web discussion groups and personal correspondence to the Cochrane Pain, Palliative and Support Care Collaborative Group¹³ and other paediatric A&E departments, confirmed that there was no pain score validated in this field.

The departmental Multidisciplinary Pain Group discussed and identified the principle factors that experienced nurses had traditionally used to assess pain in children. In addition, the items in other behavioural/observational tools were studied, for example, CHEOPS¹⁴ and TPPPS.¹⁰ Key factors were agreed by the group, the departmental lead nurse, senior nursing staff, and two paediatric A&E consultants. The items and description were then clarified by the group and the format agreed. Detailed explanation of the various categories was documented in unambiguous language to ensure uniform interpretation of the parameters. It was also decided that the tool must be in a format that was quick and easy to apply appropriate to the triage setting.

An initial assessment of the tool was performed by two groups of medical students working in the A&E department. The results confirmed face and content validity and a high level of inter-rater reliability and acted as pilot projects prior to the formal assessment of validity.^15,16 A change was made to the “cry/complaint” category to improve consistency of interpretation.

The aim of this study is to validate the AHTPS and to show it to be reliable in the paediatric A&E setting. A validated tool that is appropriate for this purpose would be an important aid to rapid and accurate triage and management of painful conditions which are so commonly a challenge to health professionals in A&E departments.

METHODS

A sample of 575 children, aged 0–15 years, presenting to the A&E department between January and July 2002 with a potentially painful condition, identified by the child, the carer, or the nature of the condition, were included in the study. Written consent for inclusion in the study was obtained from the child, parent, or carer after initial assessment. Children with immediate threat to life (triage category red) were excluded. Local Research and Ethics Committee approval was obtained for the study.

All triage nursing staff, 27 different nurses, were trained in the use of the AHTPS. Each member of the departmental pain group undertook a one to one discussion and education session with several of their colleagues until all staff were familiarised with the tool and had an opportunity to have any questions answered. These nurses then developed experience in its use at triage for about six months before the study commenced. During the study the AHTPS was not directly used to determine analgesia required. A senior research (G grade) nurse (JL) assessed the children’s AHTPS at triage at the same time as the departmental triage nursing staff performed their standard assessment using the AHTPS. They were blinded to each other’s scores. The scoring was repeated by the research nurse after 45 minutes on 310 of these children who had an intervention (for example, sling, splint etc), and/or analgesia to relieve their pain. The research nurse also documented a self-assessment Wong-Baker Face Score¹ for those children aged 3 and upwards with appropriate cognitive ability, both immediately after triage, and repeated the score after intervention and/or analgesia (292 children).

Measures

The Alder Hey Triage Pain Score (AHTPS)

Five categories of observations are identified: voice/cry, facial expression, posture, movement, and colour. Each of these has a possible score of 0, 1, or 2, resulting in a total score ranging between 0 and 10. The reference scoring chart is shown in table 1.

There is also a detailed description of each item provided on an additional sheet (see Appendix). Copyright for this tool was obtained in June 2002.

Wong-Baker Faces Pain Rating Scale¹

This self-assessment tool shows a selection of six faces with different degrees of distress. After explanation the child is asked to pick the one that best represents their hurt. This scale ranges numerically from 0 to 10 in steps of 2. The tool was used in the standard form as described by the authors.¹

Data analysis

Methods of validation of pain scoring tools for children have been described by Beyer.¹⁷ We chose to look at inter-rater reliability and three aspects of validity. The analyses were performed using the statistical package Stats Direct.¹⁸

Construct validity

This is demonstrated if the tool is shown to conform to theoretical expectations, for example, that the pain score would decrease after therapeutic doses of analgesics. We therefore compared the AHTPS score taken by the research nurse before and after intervention and/or analgesia (310 children) using the Wilcoxon signed ranks test.

Known group construct validity

This is a particular type of construct validity, which compares the new score in different settings where the pain level should differ, for example, children experiencing a painful condition with those with a less painful condition. Each child was grouped into one of the following categories: no pain, mild pain, moderate pain, and severe pain. The groups were chosen as they represent groups that have a practical application to trigger the need to prescribe the appropriate level of analgesia. This was done by one author (BS) by examining the presenting complaint and discharge diagnosis of each individual record, based on her experience of those conditions having worked in paediatric A&E medicine for nine years. A minor head injury might be expected to cause mild pain, whereas a compound displaced forearm fracture would be expected to cause severe pain. The AHTPS scores obtained by the research nurse at triage were then compared within each of these groups for all 575 children using Cuzick’s trend test,¹⁹ and the strength of the association with diagnostic group assessed using Spearman’s rank correlation coefficient.

Predictive validity

This is a type of validity that is demonstrated by using some external criteria (in our case the level of analgesia prescribed) to assess the level of pain experienced. The level of analgesia that the child was given was determined from the A&E documentation. The children were sorted into four groups: no pain (none), mild (paracetamol), moderate (anti-inflammatory medication with or without paracetamol), strong (opiates). Differences in investigator AHTPS between the groups was assessed using Cuzick’s trend test, and for strength of association using Spearman’s rank correlation coefficient. Results were available on 476 patients. Ninety nine patients who refused analgesia were omitted.

Reliability

We assessed inter-rater reliability by comparing the AHTPS judged by the research nurse with that obtained by the triage nurse working concurrently but blinded from each other’s score. Agreement was measured by a kappa statistic²⁰ for all 575 patients, using the criteria defined by Landis and Koch²¹ for agreement: kappa values <0.2 indicate poor agreement, >0.2–0.4 fair agreement, >0.4–0.6 moderate agreement, >0.6–0.8 good agreement, and >0.8 very good agreement.

AHTPS versus Wong-Baker Faces Pain Scale

As there is no validated gold standard pain tool in this area, demonstration of concurrent validity was not possible. We chose to compare the tool with the Wong-Baker Faces Pain Scale.¹ The research nurse collected the scores both at triage and after intervention using the appropriate method described.¹ We wanted to illustrate and understand the relation between the two tools and determine whether our previous reservations about self-assessment could be demonstrated. Spearman’s rank correlation was used to compare the two test scores, because a kappa statistic could not be calculated for scores using different scales. Reservations about comparing a five point scale with a ten point scale were acknowledged, although reference to such comparisons is found on the web site¹ and stated to be acceptable. The AHTPS of only the 292 children who had the cognitive ability to use the self-assessment score (3 years of age or over) were included in the comparison.

Sample size

It was determined that in order to calculate an estimated kappa of 0.7 to an accuracy of ±5%, using a 95% confidence interval, a sample size of 500 children would be necessary²²

RESULTS

Five hundred and seventy five children (male 59%, female 41%) were recruited into the study. No parent approached refused entry into the study. The median age of the children was 10 years (IQR 5–12) and included 10 children (2%) less than 1 year of age.

The diagnoses reflected the spectrum of clinical conditions presenting to the department. Soft tissue injuries represented 33%, lacerations 20%, fractures 18%, and abdominal pain 5%. One hundred and eighty five children had some form of intervention: sling 11%, wheelchair 11%, dressing 5%. Three hundred and fifty four children received analgesia; an additional 99 who were offered analgesia, refused. Analgesic medication and pain relieving intervention were used either in combination or alone as clinical need dictated by the staff caring for the patient at the time.

The median AHTPS at triage in all patients was 2 (IQR 1–3).

Construct validity

The comparison of the pain score before and after analgesia

There was a significant difference in the pain scores taken by the investigator before and after intervention (median (IQR) 3 (2–4) v 2 (1–2), p < 0.001), showing the ability of the pain tool to distinguish between differing levels of pain before and after treatment.

Known group construct validity

The comparison of pain score with pain expected by diagnostic group (based on experience)

The correlation of AHTPS with diagnostic group was not well demonstrated (r = 0.57, 95% CI 0.51 to 0.62). However, the box and whisker plot in fig 1 indicates rising pain levels for each diagnostic group of children, grouped according to the severity of their discharge diagnoses. This trend was significant using Cuzick’s trend test (p < 0.001).¹⁹

Predictive validity

The comparison of the pain score with the level of analgesia prescribed

Rising pain levels were again apparent when the children were grouped by level of analgesia required, as shown in fig 2. There was a high association with analgesic group (r = 0.80, 95% CI 0.76 to 0.83), which produced a significant trend in pain levels (p < 0.001) using Cuzick’s trend test.¹⁹

Reliability

The blinded comparison of two simultaneous pain scores by different triage nurses

There was a very high level of agreement in pain scores between the investigator and triage nurse, with a kappa value of 0.84 (95% CI 0.80 to 0.88), indicating that the tool was reliable for use at triage.

Comparison of AHTPS with Wong-Baker Faces Pain Score

The association between the investigator AHTPS (a behavioural/observational score) and the Wong-Baker Faces Pain Score (self-assessment) scores was lower at triage (r = 0.46, 95% CI 0.37 to 0.55) than after intervention (r = 0.53, 95% CI 0.45 to 0.61). This is illustrated in table 2 which gives the raw scores, and is summarised in fig 3, the investigator indicating a lower pain score than the child.

DISCUSSION

Evaluating a child’s pain is a complex subject and the optimum measure may vary according to the child’s age, cognitive development, type of pain, and presenting situation.⁵ Our tool was developed specifically for the A&E setting, choosing an observational model that we applied in the first instance to all ages. We recognise that this tool is open to interpreter bias, but this can be minimised by instigating a good staff training programme to promote uniformity of interpretation. The nature of pain is complex and multifactorial, and aspects of the observations in our tool could be influenced by factors other than pain itself. It is important therefore to use the tool accurately and in context.

The Alder Hey Triage tool has a high degree of inter-rater reliability. A kappa statistic of >0.8 represents a high degree of agreement²¹ and this is specifically considered to be adequate in assessing inter-rater reliability of behavioural tools.¹⁷ In our study this high degree of agreement was shown despite the involvement of 27 different members of nursing staff triaging patients. This is important since in reality, in the A&E setting, the tool is likely to be used by a large number of nursing staff when triaging patients. The tool was found to be quick and easy to use and staff training was easily achieved by individual discussions of about 10 minutes and then supervised use at triage for part of a shift. Members of the pain team were also available to discuss individual concerns with interpretation as they arose.

The study demonstrated construct validity for conditions presenting during the normal day to day activity of a paediatric A&E department. Predictive validity was shown with the reservation that although the pain score was not directly used to determine the analgesia prescribed but as part of a multi-factorial assessment, it cannot be assumed this was an entirely independent measure. However, in cases other than mild pain the level of analgesia needed was not assessed during this study by the triage (scoring) nurse.

Known group construct validity was less well demonstrated, probably reflecting the difficulty of allocating pain levels from the underlying diagnosis of the condition itself, which had a subjective element from the author assigning the “known groups”. In addition individuals’ pain perception to a certain injury varies. Both these factors would confound precise categorisation. It was however felt to be worth considering for conditions at either end of the spectrum and it did contribute to the overall assessment of the tool.

These results together support the relevance of using the tool in this setting.

Comparison of the AHTPS with the self-assessment tool supported our perception that often children select a relatively high value at first. The relatively poor correlation is a reflection of this but also illustrates the problem of self assessment at triage which was the reason for developing our tool. If the correlation had been very good the need for a new tool would be questionable. We anticipated that the values of AHTPS and the Wong-Baker Faces Pain Scale scores would be closer after intervention when the child has had opportunity to feel more at ease in the department and to become familiar with the self-assessment score,⁷ and this was confirmed. Although there is no validated gold standard pain scoring tool in our setting and differences between self-assessment parental, nursing, or doctor assessment of a child’s pain can be quite marked,²³ the ideal tool is one which can be used with confidence by the staff to enable prompt pain management.

We would still advocate the early introduction to self-assessment tools for children after triage in the A&E department, especially for those likely to be admitted and/or who require repeated assessment of their pain, where there is the opportunity to explain the scoring system and become familiar with it.

Other A&E departments²⁴ have reported difficulties with self-assessment tools for pain at triage in adult patients. For these patients there may be other factors that bias self-assessment values such as the perception that they will be seen and assessed more quickly or that they are more likely to be treated with strong opiate based analgesia. We suggest that the tool may also be validated for use in these departments.

We propose to continue to study and refine the tool further by looking at the internal reliability of the individual categories. In addition the tool will be studied for a variety of age groups (in particular its relevance to children <1 year). We recommend further validity studies, but we believe that it can be used and will benefit children, allowing more prompt and appropriate management of their pain at triage in A&E departments.

APPENDIX: EXPLANATION SHEET FOR THE AHTPS

Explanatory notes of observational pain scores are presented in table A1.