METHODS

Setting

The research was conducted in the paediatric accident and emergency department at Queen’s Medical Centre, Nottingham which is the only A&E department serving the whole of the population of the city of Nottingham and the surrounding area (approximately 745 000, of which children under 15 years make up approximately 136 000).

All acute child attenders aged 0–15 years, whether self or GP referred, are seen within the paediatric A&E department (40 000 new attenders per year). On average, half of the senior house officers (SHOs) working in the department have had previous paediatric experience. The majority of nursing staff are registered children’s nurses. Children were admitted from the A&E department to a short stay admissions ward or a general paediatric ward.

Care pathway development

Care pathways incorporating the recommendations of the guidelines were developed and piloted in consultation with A&E and ward nursing staff and medical staff (between February 1999 and April 1999). The pathways contained boxed areas for the initial recording of observations and historical and examination findings, so that important symptoms and signs could be documented. The clinician is prompted for a diagnosis and directed to management sections depending on the findings. For example, in the diarrhoea care pathway, once acute gastroenteritis has been diagnosed, there are different sections for the management of children with “severe”, “mild-moderate” or “no” signs of dehydration, each incorporating the relevant part of the guideline algorithm. The “severe” section includes spaces for the recording of the resuscitation fluid volume, subsequent vital signs, further fluids given, investigations done, and admission instructions. A section for recording any management that differed from that recommended (variance) was included. The documentation is available from the authors.

Implementation

Medical and nursing staff were taught how to use the care pathways in small groups by the first author over a period of a week. The nursing staff at triage identified children presenting with diarrhoea or seizure and care pathways were allocated. Laminated copies of the guideline algorithms were placed with the patient documentation. Copies of the full guidelines were available.

Power

The preimplementation admission rate for children presenting with seizure was 68%. We determined that a 20% fall in admissions would be of clinical significance. In order to show a 20% fall in admissions at a power of 80% with a 0.05% significance level, the total number required was 400¹³ (200 before and 200 after guideline implementation). It was known that 200 children were likely to attend with seizure in a 4 month period from previous data. Similar calculations were made for diarrhoea.

Data collection

Children with a medical problem presenting to A&E for a 4 month period preimplementation (April to July 1997) and postimplementation (May to August 1999) were identified prospectively. Data was collected from two sources, the patient administration system and a form completed by the attending doctor, and included: date and time of attendance; presenting problem category; length of stay in the department, and destination on leaving A&E. Two hundred seizure attendees and 300 diarrhoea attendees (divided equally between pre- and postimplementation) were selected for more detailed data collection using a stratified random sample method, based on destination at discharge from A&E. The validity of the complete data set was checked using these case notes and an error rate of 0.3% was found. Data were analysed using the χ² and Mann–Whitney U tests as appropriate, with the help of SPSS® version 8.0 for windows.

RESULTS

Medical attendance data 1997 compared with 1999 (excluding diarrhoea or seizure presentations)

Changes that occurred between 1997 and 1999 for all medical attenders (excluding those with diarrhoea or seizure) were analysed. The total number of attenders rose by 13% (table 1) and there was a rise in the proportion of all medical attenders admitted to hospital between 1997 (26%) and 1999 (30.1%) (χ² = 8, 1 df, p = 0.005) with relative risk for admission of 1.1 (95% CI 1.0 to 1.24). There was no difference in the total time spent in the department.

Attenders with diarrhoea or seizure presentation pre and post implementation

Numbers

Over the two study periods there were 531 attendances in 502 children with diarrhoea, 292 before guideline implementation in 1997 and 239 after implementation in 1999. For seizure there were 411 attendances in 398 children, 212 in 1997 and 199 in 1999 (table 2). There were no differences in the age profiles and sex of seizure or diarrhoea attenders before and after guideline implementation.

Time of attendance

There were no differences in the times of attendance, examination and departure nor in the proportion with severe dehydration (for diarrhoea attenders). However the time taken for treatment, and time spent in the department, for both presentations, were significantly reduced in the group after guideline implementation (for example the median treatment time for seizure was reduced by 25 minutes) (table 3).

Admission

Data on destination was not available for nine diarrhoea attenders and six seizure attenders. For children attending with seizure there was no change in the proportion of children admitted to hospital following implementation of the guideline (table 4). This was in contrast to the rise in the proportion admitted of other medical attenders, including those presenting with diarrhoea. The relative risk of admission with diarrhoea after guideline implementation was 1.3 (95% CI 1.25 to 1.35), marginally greater than the increased risk of admission for other medical attenders (see above).

Reattendance (returns)

Any reattendance within 1 week with the same problem was classified as a return.

• Diarrhoea. Of the 271 children who attended with diarrhoea before guideline implementation, 19 returned once and one returned twice (hence total attendances in 271 children = 292). Of the 231 children seen with diarrhoea after implementation, eight returned once (hence total = 239). This fall in children reattending with diarrhoea from 20 of 271 (7%) to eight of 231 (3%) was not statistically significant (p = 0.07).

• Seizure. There was no difference in returns for seizure following implementation, five of 207 (2.4%) returned once before guideline implementation and six of 191 (3%) returned once and one returned twice after guideline implementation. The returns included older children with known afebrile seizures (two in 1997, four in 1999) and children with febrile seizure (three in 1997 and 1999). The children reattending after implementation had all been managed appropriately according to guideline recommendations.

Additional data on selected cases compared to all cases

The selected cases were representative of all attendances with the same presenting problem with respect to age, sex, referral source, time of arrival, and times spent in the department.

Selected cases before and after guideline implementation

Recording of clinical assessment

There was a significant improvement in completion of clinical details in the record following the implementation of the guideline (see table 5 for diarrhoea and table 6 for seizure). The proportion of children attending with seizure and fever was the same, but the presumed source was documented in significantly more after guideline implementation (27% v 48%). Whilst the estimated level of dehydration was more frequently recorded in those presenting with diarrhoea after guideline implementation, there were similar proportions in each level (table 7), indicating similar severity.

Investigations and management

Diarrhoea

Intravenous infusions were recommended in the guideline only in severe dehydration, and measurement of blood urea and electrolyte only if an intravenous infusion (IVI) was to be used. After guideline implementation significantly fewer children had blood tests for full blood count, urea, and electrolytes, and fewer children had a cannula inserted and IVI commenced (table 5). The guideline states that oral rehydration solution (ORS) should only be used in a dehydrated child, and more appropriate fluids after guideline implementation were given for those with no dehydration (p<0.001) (table 8).

The guideline suggests that those with mild/moderate dehydration should be admitted for rehydration. When destination was stratified for level of dehydration, 18 of 19 (95%) of those deemed mildly/moderately dehydrated were admitted for observation after guideline implementation compared with 14 of 24 (58%) before guideline implementation.

Seizure

Fewer children were seizing on arrival before guideline implementation (14 v 3) and fewer had a prolonged seizure (23 v 10). Of 200 seizure presentations, 122 (61%) were with a fever at the time. The guideline states that investigations should be directed at determining the source of fever or cause of seizure and not be performed routinely without indication. There was a significant reduction in numbers of children having measurement of blood calcium, urea, and electrolytes (table 6). None of the children investigated for calcium, magnesium, urea, and electrolytes had abnormal results. Of the 122 children presenting with seizure and fever, five underwent a lumbar-puncture (LP), two before and three after guideline implementation. All LPs yielded negative results. All five were in infants less than 18 months old, of whom two had complex febrile seizures.

Deviations from the guideline

In four cases of children presenting with diarrhoea after guideline implementation, variances to the recommended course of action were noted and justified by the clinician. While extracting the data from the clinical records three further variances were noted which had not been recorded by the clinician—all involved treatment with ORS for children without documented dehydration. For seizure, two children had documented variances and three were found at data extraction: one for whom no outpatient appointment had been made following a first afebrile seizure; two were admitted who did not fit the criteria for admission.

DISCUSSION

The impact of guideline recommendations are rarely assessed. This study group aimed to assess the guidelines it developed with a prospective observational study with an intervention. While a randomised controlled trial on guideline implementation might produce stronger, more robust conclusions, a valid control group is difficult to accomplish. The nature of an algorithmic guideline is to teach clinicians a “map” for management which will be recalled for subsequent patients and thus influence care regardless of whether they are randomised to the guideline with care pathway documentation or not. Within the context of evaluating primary care interventions, when one practice can be compared with another, well controlled studies appear to show less of an impact than poorly controlled or uncontrolled studies.¹⁴ We have attempted to assess the impact of an intervention using before and after data. Such data will be influenced by other changes that inevitably take place in a clinical setting and which are outside the control of the researchers. This is exemplified by the increase in medical attenders of 13% between 1997 and 1999, in keeping with national trends.¹⁵ For two of the outcome measures the “control group” in this study were the medical children who attended with other conditions. Their rate of admission rose and time in A&E remained the same, unlike the intervention group. We speculate that the process for decision making may be streamlined by the availability of a structured guideline, thus reducing the time in the department.

Readers will note the increase in self referrals for all attenders (table 1, 62% to 71%) and diarrhoea and seizure attenders (table 2, 47% to 66% and 42% to 69%, respectively). Changes in health service delivery between the two time periods included the introduction of NHS Direct and a GP cooperative “out of hours” service. More children arrived seizing in 1997 (14) compared with 1999 (three), and there were more prolonged convulsions in 1997 (23 v 10). This may reflect a change in the use of rectal diazepam, which is increasingly given by the ambulance service and by parents. The numbers who had been given diazepam before attending hospital increased in 1999 (4/14, 29% pre- and 6/9, 67% post-guideline). Although the numbers involved were small, data were reanalysed after removing those with prolonged seizures, but there were no changes in the findings.

The observed rise in proportion of diarrhoea attenders admitted following implementation of the guideline may have several explanations. It is possible that some children prior to implementation were being sent home inappropriately. The frequency of returns fell after implementation, though not significantly, and a short admission may have influenced this. The guideline may also have helped doctors to recognise signs or risks of dehydration. Children admitted with diarrhoea after implementation went to a recently opened short stay ward, where the length of stay was significantly reduced compared to previous practice (the percentage of children staying more than 24 hours with seizure, gastroenteritis, asthma, and ingestion fell from 45% to 21%). For all children presenting with a medical problem, a 14% rise in hospital admissions was observed between 1997 and 1999, in keeping with reported national trends.¹⁶ There was no such increase in admissions for those children presenting with seizure, and returns remained static.

Both care pathways led to marked improvements in documentation. Important features of the history and the examination were more frequently recorded (for example, recording of stool frequency rose from 54% before guideline implementation to 78%, noting skin turgor rose from 25% to 45%, and documenting the focus of infection in febrile seizure rose from 27% to 48%). This is a consequence of a more systematic approach to record keeping, and an increased awareness of key features of the history and examination. These are especially important in view of the more frequent handover between juniors and in the interests of patient care.

There was a trend for all investigations to fall following guideline implementation, and this was statistically significant for several invasive tests. Only two children with diarrhoea after guideline implementation were given an IVI, one of whom was classified as severely dehydrated. This is in stark contrast to data before implementation where 11 children had an IVI, none of whom were classified as severely dehydrated. Whilst the level of dehydration was poorly recorded (particularly before implementation), it had been documented in these 11 children, suggesting that these data were missing in those with milder disease. It is notable that this rate of IVI for gastroenteritis was already low compared to other hospitals (for example, Pinderfields District General Hospital had a 34% IVI rate for gastroenteritis admissions in 2001). IVIs carry higher risk than oral rehydration^17–19 and therefore quality of care was improved both by reducing potentially distressing venepuncture and by decreased risk exposure.

Marked improvements in appropriate fluid management for the level of dehydration were observed for diarrhoea presenters. Fluid management recommendations were based on level 1 evidence, where clear benefits for patients had been demonstrated. By following this recommendation a quality improvement has been achieved, even though our outcome measures may not be subtle enough to demonstrate this.

Demonstrating improvements in outcome of care following implementation of guidelines is difficult. Some measure of the quality of care can be obtained by studying the process. In our study the process of care improved in many areas: fewer children had invasive and potentially distressing investigations and treatment; management was more appropriate for the clinical findings; better records were kept; children spent less time in the A&E department; there were fewer return visits and though a higher proportion of children were admitted in the case of diarrhoea, this was to a ward for observation and education where duration of stay was shorter. We recommend to other hospitals the use of these guidelines, following appropriate tailoring for local practice.