Article Text
PDF
Abstract
Background In 2007, ESAC (http://www.esac.ua.ac.be) published a set of 12 valid drug-specific quality indicators for outpatient antibiotic use in Europe. In this study, the authors aimed to develop evidence-based disease-specific quality indicators for outpatient antibiotic prescribing in Europe.
Methods Two meetings were convened to produce a list of disease-specific quality indicators for outpatient antibiotic prescribing which conform to internationally agreed recommendations, building on a similar development of drug-specific quality indicators, and in collaboration with CHAMP and HAPPY AUDIT. 62 experts were asked to complete two scoring rounds of the proposed indicators on seven dimensions: their relevance to (1) reducing antimicrobial resistance, (2) patient health benefit, (3) cost-effectiveness, (4) policy makers, (5) individual prescribers, (6) their evidence base and (7) their range of acceptable use, using a scale ranging from 1 (=completely disagree) to 9 (=completely agree). Scores were judged according to the UCLA-RAND appropriateness method.
Results For the six main indications for antibiotic prescribing (acute otitis media, acute upper-respiratory infection, acute/chronic sinusitis, acute tonsillitis, acute bronchitis/bronchiolitis, cystitis/other urinary infection) and for pneumonia, three quality indicators were proposed, the percentage prescribed (a) antibiotics; (b) recommended antibiotics; (c) quinolones. This set was scored by 40 experts from 25 countries. After one scoring round, all indicators were already rated as relevant on all dimensions, except one.
Conclusion All proposed disease-specific quality indicators for outpatient antibiotic prescribing have face validity and are potentially applicable. They could be used to better describe antibiotic use and assess the quality of antibiotic prescribing patterns in ambulatory care.
- General practice
- drug therapy
- quality of care
- evidence-based medicine
- ambulatory care
Statistics from Altmetric.com
Introduction
In 2007, the European Surveillance of Antimicrobial Consumption project (ESAC, http://www.esac.ua.ac.be) published a set of 12 valid quality indicators for outpatient antibiotic use in Europe.1 Antibiotic use is increasingly recognised as the main driver for antimicrobial resistance,2–4 and, if we want to improve antibiotic use, we have to be able to measure its quality. Meanwhile, in the USA, total antibiotic consumption is included as a quality indicator by the National Committee for Quality Assurance (NACQ; http://www.ncqa.org/Portals/0/HEDISQM/HEDIS2009/2009_Measures.pdf), and in Scotland, the Scottish government and the Scottish Antimicrobial Prescribing Group have agreed that seasonal variation of quinolone use should be ≤5% (http://www.sehd.scot.nhs.uk/mels/CEL2009_11.pdf). These quality indicators, however, might not be as relevant for individual prescribers as they are for policy makers, since clinicians need disease-specific indicators rather than drug-specific indicators.1
The largest volumes of antibiotic prescriptions for systemic use are prescribed in primary care,2 with respiratory-tract infections (RTI) and urinary-tract infections being the most common indications.5 The effect of antibiotic prescribing in primary care for these conditions on antimicrobial resistance can be observed at individual patient-3 4 and practice levels.6 Reducing antibiotic dispensing at general-practice level is associated with reduced local antibiotic resistance.6
Since resistance is not limited by the borders of a practice, county or country, we aimed to develop a set of relevant evidence-based disease-specific quality indicators for outpatient antibiotic prescribing that is measurable on each of these levels in Europe, using a similar methodology to that we used before to develop the drug-specific quality indicators.
Methods
Developing a set of quality indicators
Quality indicators are defined as explicitly defined measurable items of antibiotic use giving a possible indication of the level of quality,7 8 focusing on different aspects of quality (effectiveness, safety, appropriateness and costs9; compliance and persistence) and relevant for clinical practice.10 A prescribing quality indicator is a measurable element of prescribing performance, for which there is evidence or consensus that it can be used to assess quality and, hence, be used in changing the quality of care provided.11 See box 1, for more information about quality indicators.
Prescribing quality indicators: definition and requirements
A prescribing quality indicator (PQI) is a measurable element of prescribing performance for which there is evidence or consensus that it can be used to assess quality and, hence, be used in changing the quality of care provided.11 Usually, a PQI is defined as a percentage of patients who received the recommended drug treatment, with the numerator comprising the number actually receiving the treatment and denominator comprising the number of all patients for whom the treatment is appropriate. PQIs have explicitly defined criteria regarding what constitutes good quality of care, and the values of the PQI can be compared over time and across different providers. Usually, PQIs are developed based on scientific evidence and/or acceptance by professionals in the field.7 Therefore, they can be used to measure the compliance with drug recommendations as given in clinical guidelines. When implementing PQIs in daily practice, unintended consequences of publishing performance data as described by Smith et al should be taken into account.12 That is why the requirements regarding validity and reliability of PQIs depend on the aim of their use. For internal purposes, PQIs need to be relevant for healthcare providers: they have to be specific and sufficiently detailed to show potential problems and capture pertinent changes in prescribing. However, for a fair comparison between healthcare providers for external use, for example by third-party payers, there are additional requirements, such as adjustment for patient case-mixes and having an adequate number of patients per provider.13
Measurability of PQIs provides a great opportunity for research—for example, for comparison of indicator scores to assess their concurrent validity or by linking the scores of prescribing indicators to different patients' outcomes for predictive validity assessment.14
To produce a draft list of evidence-based disease-specific antibiotic prescribing quality indicators in Europe, two meetings were convened in Antwerp by the ESAC Ambulatory Care Subproject Group, funded by the European Centre for Disease prevention and Control (ECDC) (figure 1). During the first meeting in June 2008, the ESAC drug-specific quality indicators were presented, as well as results from two European projects, Health Alliance for Prudent Prescribing, Yield and Use of Antimicrobial Drugs in the Treatment of Respiratory Tract Infections (HAPPY AUDIT; http://www.happyaudit.org)15 and Changing behaviour of Healthcare professionals And the general public towards a More Prudent use of antimicrobial agents (CHAMP; http://www.champ-antibiotics.org),16 with closely related objectives. HAPPY-AUDIT shared its useful experience in developing quality indicators for diagnosis and treatment of RTI in general practice using a modified Delphi methodology, and CHAMP shared information on its collection and comparison of national guidelines for RTI (including acute lower-RTIs, acute sore throat, acute otitis media and acute sinusitis).
Development of outpatient disease-specific antibiotic prescribing quality indicators. APRES, The Appropriateness of prescribing antibiotics in primary healthcare in Europe with respect to antibiotic resistance (http://www.nivel.eu/apres); BAPCOC, Belgian Antibiotic Policy Coordination Committee (http://www.bapcoc.be/); CHAMP, Changing behaviour of Healthcare professionals And the general public towards a More Prudent use of antimicrobial agents (http://www.champ-antibiotics.org); DURQUIM, Drug Utilisation Research Quality Indicators Meeting11; ESAC AC NR, European Surveillance of Antimicrobial Consumption Ambulatory Care subproject National Representative (http://www.esac.ua.ac.be/); GRACE, Genomics to combat Resistance against Antibiotics in Community-acquired LRTI in Europe (http://www.grace-lrti.org/); GRIN, General Practice Respiratory Infections Network; HAPPY AUDIT, Health Alliance for Prudent Prescribing; Yield And Use of antimicrobial Drugs In the Treatment of Respiratory Tract Infections (http://www.happyaudit.org/); WHO, World Health Organization (http://www.who.int/).
A second ESAC meeting was held 1 year later to present and discuss the DURQUIM framework for quality indicators,11 the HAPPY AUDIT quality indicators for diagnosis and treatment of RTI,15 17 as well as a draft list of ESAC disease-specific quality indicators using guidelines (collected in CHAMP) as the evidence base. This draft list consisted of 12 indicators, two for each of the six indications for which most antibiotics are being prescribed. These indications were selected based on IMS Health data linking primary care antibiotic prescribing data with indications labelled with codes from the revised second edition of International Classification of Primary Care (ICPC-2-R)18 for France, Germany, Italy, Spain and the UK. After the second meeting, NA and SC developed a proposed list of ESAC disease-specific quality indicators taking into account the discussions and using a similar outline as for the ESAC drug-specific quality indicators that is each indicator description contains an indicator number, a title (Label), definition, public health objective, calculation formula, range of acceptable use, recommended actions, limitations and references (to (inter)national guidelines) (table S1).
During both meetings, the presence of clinicians and scientists with expertise in general practice, microbiology, infectious diseases, pharmaco-epidemiology, pharmacy and/or drug utilisation allowed the development of quality indicators from the perspective of professionals to be discussed, and a proposed set of disease specific antibiotic prescribing quality indicators as well as a roadmap describing the next steps with these indicators to be produced. More details on the method of indicator development have been described elsewhere (table S1).
Assessing a set of quality indicators
To assess the relevance of the proposed disease-specific quality indicators, two consecutive cycles of scoring were performed. For this purpose, we aimed to select experts with known expertise primarily in general practice and secondarily in infectious diseases, microbiology, pharmaco-epidemiology, pharmacy, quality indicators development and/or drug utilisation. Therefore, the ESAC lead National Representatives were asked to identify an expert meeting these criteria. In addition, experts were selected among the participants of several EU projects relevant to this topic—for example, GRACE, APRES, etc (figure 1). Sixty-two experts from 33 countries were sent an email containing the proposed set of quality indicators and a scoring sheet. They were invited to score the proposed set of disease-specific antibiotic prescribing quality indicators on seven dimensions, that is their relevance to (1) reducing antimicrobial resistance, (2) patient health benefit, (3) cost-effectiveness, (4) policy makers, (5) individual prescribers, (6) their evidence base and (7) their range of acceptable use, using a scale ranging from 1 (=completely disagree), over 5 (=uncertain) to 9 (=completely agree). If participants did not agree with the range of acceptable use (score: 1–5), they were asked to suggest a new acceptable lower and upper limit. Experts were welcome to make any suggestion to improve the description of the proposed indicators, ideally providing supporting evidence. These were to be taken into account in the second round of scoring.
In the second round of scoring, experts were given the opportunity to reconsider their score. They were sent an email now containing the same proposed set of quality indicators and a scoring sheet with their original score as well as the median score of all respondents per indicator and per dimension.
The scores were processed according to the UCLA-RAND appropriateness method.19 20 Proposed indicators were judged relevant if the median score was not within the 1–6 interval, and if there was consensus, that is if the number of scores within the 1–3 interval was less than one-third of the panel.
Defining a final set of quality indicators
To define the final set, only relevant indicators were selected.
Results
As a result of two ESAC Ambulatory Care Subproject Group meetings, it was concluded that a proposed list of quality indicators will be developed for the six most frequent indications for antibiotic prescribing (in descending order: acute bronchitis/bronchiolitis, acute upper RTI, cystitis/other urinary infection, acute tonsillitis, acute/chronic sinusitis, and acute otitis media; data not shown) and for pneumonia (labelled with ICPC-2-R codes R78, R74, U71, R76, R75, H71 and R81, respectively). In addition, it was decided not to adopt the HAPPY AUDIT quality indicators, mainly because it is difficult to assess their evidence base, based on their description, and to produce indicator values using data routinely collected in electronic medical health records in primary care. For each of these seven indications, three indicators were proposed:
the percentage of patients with age and/or gender limitation prescribed an antibiotic;
the percentage of patients with age and/or gender limitation prescribed an antibiotic, and receiving the guideline recommended antibiotic;
the percentage of patients with age and/or gender limitation prescribed an antibiotic, and receiving quinolones (tables 1, and S1).
List of proposed disease specific antibiotic prescribing quality indicators in Europe
The indication ‘pneumonia’ as well as the third indicator (c) resulted from the discussion during the second ESAC meeting.
All 21 indicators were described in a way that allowed them to be read and scored separately. Therefore, parts of their description were very similar.
We received scores from 40 participants (12 women; 25 countries). Eight experts declined (six no time, two no longer active in the research field), three experts forwarded their invitation to another expert within their network, and 14 did not respond. After the first round of scoring, all indicators were rated as relevant (ie, score not within the 1–6 interval) quality indicators on all seven dimensions except 3a, that is the percentage of female patients older than 18 years with cystitis/other urinary infection (ICPC-2-R: U71) prescribed antibacterials for systemic use (ATC: J01), which was scored 6 for cost-effectiveness.
Similar scores were received after the second scoring round (table 2.)
Relevance of the proposed disease-specific quality indicators for outpatient antibiotic prescribing in Europe: the scores* for and consensus† on seven dimensions: their relevance to (1) reducing antimicrobial resistance, (2) patient health benefit, (3) cost-effectiveness, (4) policy makers and (5) individual prescribers, (6) their evidence base and (7) their range of acceptable use
The median score per dimension across all 21 indicators was about 8, with only minor differences in median score per dimension between individual indicators. Indicators 1–6 had a high median score for relevance to reducing antimicrobial resistance and a low median score for relevance to patient health benefit, while it was the other way around for indicator 7. For relevance to individual prescribers, indicator 6c had the highest median score, followed by 2a, 2c, 3c and 6b, while 3a had the lowest median score.
Discussion
Taking into account the scores from a relevant group of experts—professionals rather than policy makers—from a set of 21 (7×3) proposed guideline based disease-specific quality indicators for outpatient antibiotic use in Europe, all seem to be relevant, that is, they have face validity and are potentially applicable. Only one proposed indicator, the percentage of female patients older than 18 years with cystitis/other urinary infection (ICPC-2-R: U71) prescribed antibacterials for systemic use (ATC: J01), was not scored as relevant for one out of seven dimensions, that is for cost-effectiveness. The fact that not all countries, for example the UK, recommend antibacterials in the treatment of urinary-tract infection could be a possible explanation.21
These results are remarkable, since they were already achieved after the first of two rounds of scoring. And what is more, the expert panel consisted of experts from 24 different countries all over Europe and Israel (figure 2). Because all quality indicators are well defined, the risk of misinterpretation is limited. In addition, for all indicators, an evidence base was provided, based on current national guidelines, and this was scored as relevant.
Map of Europe (+Israel): the number in each country represents the number of experts participating in the expert panel.
Some limitations have to be taken into account specifically for these quality indicators. When interpreting the indicators expressing the percentage of patients prescribed an antibiotic (ie, table 1, 1a–7a), where lower scores represent better quality of care except for cystitis and pneumonia, different thresholds for consulting a GP because of differences in healthcare organisation in primary care settings22 can influence the acceptable range. Delayed prescribing is another reason for potential bias in the interpretation of values for these quality indicators. The percentage of patients prescribed the recommended antibiotics (ie, table 1, 1b–7b), where higher scores represent better quality of care, can be biased by country-specific guidelines recommending other antibiotic classes as first-line therapy—for example, small spectrum penicillins for acute otitis media (H71) in Scandinavian countries.23 24 The percentage of patients prescribed quinolones (ie, table 1, 1c–7c), where lower scores represent better quality of care, which has also been suggested as a quality indicator by Altiner et al,25 can be biased by specific resistance patterns.
Differentiating between acute pharyngitis (=ICPC-2-R: R74, acute upper RTI) and acute tonsillitis (=ICPC-2-R: R76, acute tonsillitis) can still be challenging or even impossible if coinciding. Since essentially the quality indicators for these diagnoses, 2a–c and 4a–c, respectively, do not differ, it could be tempting to merge them into one indicator, for example, for 2a and 4a: the percentage of patients older than 1 year with acute upper RTI (ICPC-2-R: R74) or with acute tonsillitis (ICPC-2-R: R76) prescribed antibacterials for systemic use (ATC: J01). At this stage, however, we prefer to consider them as separate indicators, since for acute upper RTI (including pharyngitis) there is less evidence to support antibiotic prescribing than for acute tonsillitis. There are certain subgroups of patients with acute tonsillitis, for example, those with a history of rheumatic fever, who might benefit from antibiotics.
Our quality indicators are based on national guidelines. Looking for consensus, we choose to take a conservative position for example for age limits. Since national guidelines are composed of available best evidence and expert opinions which can evolve over time, a regular review of these indicators will be preferable.
Other general limitations regarding consensus methods, for example possible language barriers that apply to our work as well, have been described in detail elsewhere.17
Indicator 7 is the only indicator for which antibiotic prescribing is generally advised. Therefore, relevance to patient health benefit is probably scored higher than relevance to reducing antimicrobial resistance, while for the other indicators, it is the other way around. Compared with the ESAC drug-specific indicators, which were all scored as relevant to reducing antimicrobial resistance and to policymakers, but not always to patient health benefit and cost-effectiveness, the disease-specific quality indicators scored relevant on all these and three extra dimensions, except 3a (cf supra). The disease-specific indicators therefore indeed seem to be more useful to individual prescribers. In addition, for all indicators, relevance to individual prescribers was scored higher than or as high as relevance to policymakers.
The percentage of patients older than 2 years with acute otitis media/myringitis (ICPC-2-R: H71) prescribed antibacterials for systemic use (ATC: J01), receiving quinolones (indicator 6c), obtained the maximum score in the dimension relevance to individual prescribers. This is probably explained by the high prevalence of otitis media in children for which quinolones are not advised.
We believe the most relevant indicator probably is that which identifies actual problems with the quality of outpatient antibiotic prescribing rather than that which is most relevant on one specific or all dimensions. The indications for which the highest amount of inappropriate antibiotics are prescribed can be different at practice-, county- or country level. In addition, and similar to our advice regarding the use of the drug-specific quality indicators,1 we do not recommend assessing the quality of outpatient antibiotic prescribing by looking at a single quality indicator. After all, acceptable antibiotic prescribing proportions for one indication might coincide with unacceptable antibiotic prescribing proportions for other indications or with unacceptable proportions of antibiotics that are not recommended, and vice versa. Hence, we talk about a set of indicators.
Unlike the ESAC drug-specific quality indicators for which values can be derived from ESAC data, for these disease-specific quality indicators data linking antibiotic prescriptions with patients' age/gender and diagnosis are not readily available.
A Dutch study in which clinical information from individual patients was linked to antibiotic use, and guidelines were used as a quality benchmark,26 assessed the quality of antibiotic prescribing related to indication. This was even done on a national scale in The Netherlands,27 Denmark28 and Finland,29 but in many European countries this is not feasible on a routine base at present.
Our quality indicators are linked to disease using the revised second edition of International Classification of Primary Care (ICPC-2-R) codes. While we are aware that in daily general practice, the link between a diagnostic label or code and the actual disease might not always be reliable, we believe that linking to an international classification is the most pragmatic solution. Since publication by WONCA in 1987,18 ICPC has received increasing world recognition as an appropriate and comprehensive classification for general/family practice and primary care, and has been used extensively in some parts of the world, notably in Europe and Australia. The WHO has accepted ICPC as a WHO-related classification to be used for health-information registration in primary care. Additionally, ICPC-2 has been mapped to the 10th revision of the WHO International Classification of Diseases (ICD-10).30
In the ESAC Ambulatory Care Subproject, we aim to gather routinely collected general practitioners' antibiotic prescribing data linked to indication expressed in ICPC-2-R or ICD-10 to substantially broaden our interpretation of the striking variation in antibiotic use between European countries, and to produce values for the final set of 21 evidence-based disease-specific quality indicators. However, the collection of these data has been challenging. Other projects have shown that such routinely collected data exist in most European countries—for example, Electronic Health Indicator Data.31 These data are, however, not easily accessible or have not been processed. The publication of these quality indicators will hopefully result in increased accessibility of these routinely collected data sources.
Once this hurdle is overcome, our quality indicators will be very easy to implement, even at the level of the individual primary care prescriber using electronic medical records with ICPC coding for diagnosis, and ATC coding for antibiotic prescriptions. This is in contrast to other primary-care quality indicators that require information like CRP values or Anthonisen criteria as in the HAPPY AUDIT quality indicators.17
Comparisons between peers has been considered an important stimulus to quality improvement, for example, to antibiotic consumption as well.32 If our proposal is considered acceptable by general practitioners, our final set of quality indicators could allow GPs, practices, networks or even individual countries to assess their position in relation to others. This could trigger actions to improve antimicrobial prescribing. It could also allow the identification of temporal trends and regional differences, and therefore trigger evaluation and action. Finally, this set of indicators will be available to inform the process of development, implementation and evaluation of national and regional guidelines, and thus be useful for policy makers as well.
Although the acceptable range was scored relevant, the use of this range as a real benchmark should be considered carefully. In our opinion, one benchmark value on a European level cannot be given, but rather a range of acceptable indicator values should be defined.33 Several contextual factors, such as local guidelines, and different thresholds for consulting a GP because of differences in healthcare organisation in primary care settings22 have to be taken into account. Hence, we believe these disease-specific quality indicators for outpatient antibiotic prescribing can be the same in all countries, but the interpretation could vary from country to country.
Thus, some of the variations revealed by routine data may reflect real and important variations in actual healthcare quality, that is inappropriate antibiotic use, that merit further investigation and action, but some apparent variation may also arise because of other misleading factors such as unadjusted case-mix differences.13
Using information from recorded diagnoses can introduce bias owing to incomplete registration when some patients with a condition do not have a corresponding diagnostic code registered in the data or owing to incorrectly registered diagnostic codes.34 When there is information bias in the documentation of diagnoses in relation to the treatment status, the use of diagnostic codes alone can mislead both policy makers and healthcare providers about the performance scores of quality indicators. In such cases, a combination of diagnostic codes and clinical information is recommended for prescribing quality assessment.35 If register-based proxies for diagnoses, disease severity or risk factors are employed, validation is essential.36 Assessment of the concurrent validity of these indicators by comparing with a ‘gold standard’ quality assessment at the patient level using all available clinical information will be an important next step. This assessment will also help to decide whether or not indicators on acute upper RTI and acute tonsillitis can be merged.
In conclusion, our work could be considered as a solid, next step in the development of a set of valid and feasible evidence-based disease-specific quality indicators for antibiotic prescribing in Europe. Consequently, the challenge is to further validate these indicators for individual prescribers. However, until then, even without correction for resistance patterns and other contextual factors, the reported final set of ESAC disease-specific quality indicators can be used to describe antibiotic use in ambulatory care in order to assess the quality of antibiotic prescribing.
Acknowledgments
We thank all participants of the ESAC Outpatient Disease-specific Antibiotic Prescribing Quality Indicators expert panel for their valuable contribution to this work: M Maier, H Mittermayer (deceased 6 July 2010) (Austria), S Bartholomeeusen, A De Sutter, R Van der Stichele (Belgium), A Tambic Andrasevic, M Katic (Croatia), J Vlcek (Czech Republic), L Bjerrum, M Plejdrup Hansen, J Sahl Andersen (Denmark), I Lutsar (Estonia), U-M Rautakorpi (Finland), A Poppelier (France), A Altiner (Germany), G Ternak (Hungary), C Bradley, R Cunney (Ireland), H Edelstein (Israel), A Rossi (Italy), U Dumpis (Latvia), F Schellevis, T Verheij (The Netherlands), H Melbye, H Salvesen Blix (Norway), S Schlabicz (Poland), M Ribeirinho (Portugal), G Popescu (Romania), H Hupkova (Slovakia), M Cizman, J Kersnik (Slovenia), A Bada Valmaseda (Spain), S Mölstad (Sweden), G Zanetti (Switzerland), C Butler, J Cooke, M Heginbothom, C Hughes, P Little, W Malcolm (UK). We would also like to thank O Laius (Estonia), G Hebbrecht, D Didier (France), B Chazan (Israel), A Băicuş (Romania), C McNulty, P Davey (UK). The other ESAC Project Group members are S Vaerenberg (Belgium), B Markova (Bulgaria), A Kontemeniotis (Cyprus), N Frimodt-Møller (Denmark), L Rootslane (Estonia), J Vuopio-Varkila (Finland), P Cavalie (France), W Kern (Germany), H Giamarellou (Greece), H Briem (Iceland), R Raz (Israel), P Folino (Italy), R Valinteliene (Lithuania), M Bruch (Luxembourg), M Borg (Malta), S Natsch (The Netherlands), W Hryniewicz (Poland), A Băicuş (Romania), S Ratchina (Russia), V Foltán (Slovakia), J Campos (Spain), G Skoog (Sweden), S Ünal (Turkey). More information on the ESAC Project, including the present members of the ESAC Project Group, is available at http://www.esac.ua.ac.be.
References
Supplementary materials
Web Only Data
Files in this Data Supplement:
Footnotes
NA and SC contributed equally.
Funding The ESAC project was funded by the European Centre for Disease Prevention and Control (ECDC; Grant Agreement 2007/001).
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.