Background: Clinical decisions which impact directly on patient safety and quality of care are made during acute asthma attacks by individual doctors based on their knowledge and experience. Decisions include administration of systemic corticosteroids (CS) and oral antibiotics, and admission to hospital. Clinical judgement analysis provides a methodology for comparing decisions between practitioners with different training and experience, and improving decision making.
Methods: Stepwise linear regression was used to select clinical cues based on visual analogue scale assessments of the propensity of 62 clinicians to prescribe a short course of oral CS (decision 1), a course of antibiotics (decision 2), and/or admit to hospital (decision 3) for 60 “paper” patients.
Results: When compared by specialty, paediatricians’ models for decision 1 were more likely to include level of alertness as a cue (54% vs 16%); for decision 2 they were more likely to include presence of crepitations (49% vs 16%) and less likely to include inhaled CS (8% vs 40%), respiratory rate (0% vs 24%) and air entry (70% vs 100%). When compared to other grades, the models derived for decision 3 by consultants/general practitioners were more likely to include wheeze severity as a cue (39% vs 6%).
Conclusions: Clinicians differed in their use of individual cues and the number included in their models. Patient safety and quality of care will benefit from clarification of decision-making strategies as general learning points during medical training, in the development of guidelines and care pathways, and by clinicians developing self-awareness of their own preferences.
- A&E, accident and emergency
- CS, corticosteroids
- GPs, general practitioners
- VAS, visual analogue scale
- clinical judgement analysis
- decision making
- systemic corticosteroids
- hospital admission
Statistics from Altmetric.com
Doctors make decisions based on their education, training and personal experience. It is important that educational programmes identify their (often subconscious) use of cues, so that decisions can become more evidence based. Clinical judgement analysis provides a method of assessing factors affecting clinical judgement.1 It can help doctors understand the differing emphases they place on various items of clinical information, and so improve these decisions. It also has the potential to help doctors compare their decision making with that of colleagues, both between specialties and between those with different levels of training and experience. There is evidence that this enables novice practitioners to learn appropriate decision making more quickly than through simple feedback on the correctness of their diagnosis.1,2 Changing the behaviour of doctors depends on not only improving their knowledge and understanding of the evidence, but also increasing their understanding of what influences their own decision making.
Acute asthma remains a common cause for a child’s admission to hospital. The decision whether or not to admit remains one of the most important issues in children’s healthcare,3 and may be the responsibility of a relatively junior doctor in training. Failure to appropriately admit has the potential to place the individual child at risk of poor outcome, but inappropriate admission not only impacts adversely on that child and their family but also reduces the availability of resources for others.4
A second important decision is whether or not to administer systemic corticosteroids (CS). Despite evidence suggesting that CS should be considered part of the initial early treatment of all but mild exacerbations, some doctors may still be uncomfortable with their early use.5–7 An adequate assessment of the severity of the asthma attack, ideally after initial bronchodilator therapy, may help ensure early consideration of the need for systemic CS and/or hospital admission.8 The national asthma guidelines clearly advise doctors not to “give antibiotics routinely in the management of acute childhood asthma”.9 The vast majority of acute asthma attacks in childhood are triggered by viral respiratory infections. However, as the signs and symptoms in both upper and lower respiratory tract infections and in viral and bacterial infections overlap in children, many doctors will have worries about concomitant bacterial infections and may prescribe antibiotics inappropriately.
It has been shown that there are considerable differences between treatment decisions in the care of adults with asthma across five European countries.10 We have therefore undertaken a study to assess the clinical decision making of a group of clinicians who would be involved in the assessment and management of children presenting to an accident and emergency (A&E) department with an asthmatic attack. We have analysed the decisions to administer a short course of oral CS, to prescribe antibiotics and to admit to hospital, and have compared the models of trained doctors with those in training, and of paediatricians with those working in other areas.
Practitioners participating in this study comprised paediatricians, A&E specialists and general practitioners (GPs). Included in the sample of 62 were clinicians from every acute hospital with an A&E department in Northern Ireland. Information was recorded regarding their seniority (fully trained or in training) and their area of practice (paediatrics, A&E or general practice). The final sample of practitioners comprised 17 consultant paediatricians and A&E specialists, and 32 trainees in those specialties, together with 11 GPs, one nurse practitioner, and one staff grade doctor. Forty nine practitioners completed the appraisals of all 60 cases, and the remaining 13 completed all but one (11 practitioners) or two (two practitioners).
Prior to the meetings, two paediatricians (JJ and MS) prepared a folder of 60 fictional patient vignettes (with an additional 10 duplicates), describing the symptoms and signs (after initial nebulised bronchodilator therapy) of a child presenting to the A&E department with presumed asthma. Table 1 shows an example vignette and the visual analogue scale (VAS), as presented to the participants. Table 1 also shows the distribution of values of the 12 cues in the entire set of vignettes (this was not shown to participants). In devising the series, JJ and MS examined the profiles to ensure that there were no nonsense or incompatible cue combinations and that they were clinically credible. Cases were presented in the same random order for all participants. Each was asked to indicate their view, based on the information provided, on each of the three decisions by drawing a short vertical line somewhere on the horizontal lines (which were scaled 0 to 100 for analysis), indicating their strength of preference for the particular management option. These judgements were used as the dependent variable in the regression analyses.
The cues covered the following areas which, according to Advanced Paediatric Life Support recommendations, require assessment: (1) the “effectiveness of breathing” (eg, degree of air entry, cyanosis and alertness) and (2) the “work of breathing” (eg, respiratory rate, degree of dyspnoea, indrawing and wheeze).11 Categories for wheeze were mild, moderate, severe and “little heard” which was used in the context of severe dyspnoea and poor air entry to describe silent chest. Peak expiratory flow rate was not included because many of the vignettes described children too young to perform this manoeuvre. Decisions regarding prescription of medication and (in particular) admission to hospital depend not only on the assessment of severity of the illness but also on the assessment of parental ability (including their perceived competence and level of anxiety), and on the age of the child. Current inhaled CS usage was also included because children requiring this therapy (particularly if high-dose) may be at risk of more severe asthma attacks. Many acute asthma episodes in childhood are triggered by respiratory infections. The signs of lower airways obstruction are thus variably mixed with those of infection. Therefore, we also included cues which would be associated with infection, including fever and sounds on auscultation other than wheeze (coarse crepitations).
Multiple regression analysis was used to assess all cues simultaneously to discover which were predictive of the VAS score for oral CS prescription, antibiotic prescription and admission. Examination of the 60 vignettes revealed that altered levels of alertness and slight/moderate cyanosis were present in the same 21 vignettes. Change in alertness can provide warning of developing hypoxia before cyanosis is clinically evident, and consequently cyanosis was omitted from the remainder of the analysis in favour of alertness. Some of the cues were categorised into groups prior to analysis as indicated in table 2.
In light of the knowledge that doctors tend to use a limited number of clinical cues, further analysis was performed using stepwise selection of variables to obtain a more parsimonious model.12 Given the relatively small sample size (60 vignettes), we set a conservative p value of 0.10 for entry and elimination in the stepwise selection procedure to minimise the risk of rejecting cues inappropriately through lack of power. R2 values were used to indicate the proportion of the variation in VAS scores that could be explained by the model. Intraclass correlation coefficients were calculated to measure the degree of agreement between the 10 pairs of repeated VAS assessments made by each participant.
Given that there was some imbalance in the study design between level of training (consultant/GP vs other) and specialty (paediatrician vs other), analysis of variance was used to compare R2 values, numbers of cues used in the model, and intraclass correlation coefficients between grades whilst adjusting for speciality, and between specialties whilst adjusting for grade. Similarly, Mantel-Haenszel stratified χ2 tests were used to investigate the use of individual cues in the stepwise model between the clinician subgroups. Given the large numbers of comparisons of cue usage performed between subgroups of clinicians, tests were conducted using the 0.01 significance level rather than the more conventional 0.05 level. Statistical analyses were performed using SPSS release 12.0 (SPSS, Chicago, IL) and STATA release 8.0 (Stata, College Station, TX).
Table 2 summarises the regression coefficients for the three VAS scores averaged over the 62 participants for the remaining 11 cues. These represent the average increase in VAS score (on a 0–100 scale) associated with each cue after adjusting for all other cues. One sample t tests were used to assess if the mean coefficient calculated across all participants differed significantly from zero. These show, for example, that young age was associated mainly with an increased admission score, and that a history of use of inhaled CS was associated with an increase in the oral CS score and, for high doses only, an increase in admission score.
Table 3 shows information about the repeatability of VAS scores. The intraclass correlation derived from the 10 repeated scores for the decision to prescribe oral steroids yielded a coefficient of 0.70 when averaged across all participants. Paediatricians were significantly more reproducible in their scores on this decision than other specialties (mean 0.77 vs 0.59; p<0.05). The admission and antibiotic decisions were more repeatable with average intraclass correlations of 0.78, and with no statistically significant differences between subgroups of the participants.
Table 3 also summarises information about characteristics of the participants’ models. For the oral steroid decision, the regression models based on paediatricians’ scores had significantly larger R2 values both when all variables were included in the model (full model: mean 0.82 vs 0.74; p<0.05) and after elimination of cues that did not contribute significantly (stepwise model: mean 0.80 vs 0.70; p<0.01). The stepwise models for paediatricians contained a significantly greater number of cues (mean 5.0 vs 3.9; p<0.05). For the admission decision, the same pattern of differences was observed, although only the paediatrician comparison of the value of R2 in the full model attained significance (mean 0.86 vs 0.81; p<0.05). No significant differences were found for the antibiotic decision.
Figure 1 demonstrates differences observed between the clinical judgement models of two participants for the oral CS decision. The consultant paediatrician’s model included eight cues and achieved an R2 value of 0.87, compared with the trainee paediatrician’s which included four cues and achieved an R2 value of 0.74.
The relevant columns of table 2 indicate the individual cues that were significantly associated with increases in the participants’ scores for each of the three decisions. Analysis of the inclusion of cues in the clinical judgement models of our clinicians showed which cues were most commonly included for each of the three decisions. These results are detailed in tables 4, 5 and 6.
For the decision to prescribe a short course of oral CS (table 4), paediatricians were more likely, compared to other specialties, to have altered level of alertness (54% vs 16%; p<0.01) in their models.
For the decision to prescribe oral antibiotics (table 5), paediatricians’ models were significantly more likely to include presence of crepitations (49% vs 16%; p<0.01) and significantly less likely to include inhaled CS (8% vs 40%; p<0.01), respiratory rate (0% vs 24%; p<0.01) and air entry (70% vs 100%; p<0.01).
For the decision to admit to hospital (table 6), compared to other grades, consultants’/GPs’ models were significantly more likely to include wheeze severity (39% vs 6%; p<0.01).
The techniques of clinical judgement analysis have been applied to a number of areas within medicine, for example, judging treatment efficacy in rheumatoid arthritis,1 prioritisation decisions within a dialysis programme,13 the diagnosis of chronic heart failure,14 and urgency and priority for cardiac surgery.15 Although one study has examined diagnostic and treatment decisions for acute otitis media,16 there has been relatively little work done in relation to conditions affecting children. A recent study has however concluded that there is significant variability in decision making for children presenting to hospital during acute illness, and that this was related to the presence of doctors in training.4
The children described in the study vignettes had varying degrees of respiratory symptoms and signs following initial bronchodilator therapy. We have identified differences in the inclusion of various clinical cues in the judgement models of clinicians from different grades and different specialties which are consistent with their differing levels of paediatric experience and reflect actual clinical decision making. As the relatively small group of participants limits the generalisability of our results, we hope that others will find our methodology helpful in undertaking further studies in this area. This type of analysis can provide an insight for an individual into their decision making, and thus form the basis of informed discussion regarding the number of key cues used in reaching a clinical decision, and the effectiveness with which they are used. This has the potential to contribute to the education of doctors at all stages in their careers, and so to improve the safety and quality of the care provided for children presenting to hospital during acute illness. Guidelines are playing an increasingly important part in patient management and are often incorporated into care pathways. Preparation and implementation of these also need to take into account issues in decision making such as those identified in this study.17 Any algorithmic approach to patient management, to be successful, should incorporate the smallest number of cues which have been shown to be effective in enabling doctors to reach meaningful and informed decisions regarding issues of clinical management such as prescribing and admission to hospital.
In a review of medical decision making in paediatrics, Bauchner et al concluded that decision making is largely influenced by factors from a set of three overlapping domains.17 These are (i) physician knowledge and characteristics, (ii) patient characteristics and values, and (iii) external clinical evidence. All three are set within the broad context of societal norms, which may of course differ significantly between different countries or even regions. The balance between the three overlapping domains will also change depending on the type of decision being made. The process of decision making used by doctors has been described as “a subcortical algorithm, built up over years of experience in the diagnosis, management and outcomes of the common, the common variations, and the rare”.18 Although this is often regarded as a “black box” activity (of which clinical judgement analysis can provide no more than a paramorphic representation),19 there is increasing recognition of the importance of improving our understanding of the processes that affect clinical decisions in everyday practice. In so doing we may not only improve the quality of individual patient care, but also, through education and training, enable practitioners to gain better insights into their own decision making, and hence design better tailored continuing professional development throughout their careers.16,20 This has particular relevance to current discussions regarding the length and content of specialist training. A good practitioner not only has the right knowledge and an ability to interpret the relevant literature, but can also place their decision making in a wider context and accommodate personal, patient and societal values and pressures.
What is already known on this topic
Medical decision making is a key factor in quality of care.
Decision making can be analysed through clinical judgement analysis.
What this study adds
Clinicians differ in their use of individual cues and the number included in their models.
Simulated patient vignettes can help doctors understand and improve their clinical management decisions.
Clinicians differed in their use of individual cues and the number included in their models. Patient safety and quality of care will benefit from clarification of decision-making strategies as general learning points during medical training, in the development of guidelines and care pathways, and by clinicians developing self-awareness of their own preferences.
The authors are indebted to Professor John Watson for encouragement to undertake the study, to Janice McConnell for assistance with analysis, and to the clinicians whose willing participation made it possible.
Published Online First 11 April 2007
Competing interests: None.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.