AIM Controversy exists regarding the evidence base of medicine. Estimates range from 20% to 80% in various specialties, but there have been no studies in paediatrics. The aim of this study was to ascertain the evidence base for community paediatrics.
METHODS Twelve community paediatricians working in clinics and schools in Yorkshire, Manchester, Teesside, and Cheshire carried out a prospective review of consecutive clinical contacts. Evidence for diagnostic processes, prescribing, referrals, counselling/advice, and child health promotion was found by searching electronic databases. This information was critically appraised and a consensus was obtained regarding quality and whether it supported actions taken.
RESULTS Two hundred and forty seven consultations and 1149 clinical actions were performed. Good evidence was found from a randomised controlled trial or other appropriate study for 39.9% of the 629 actions studied; convincing non-experimental evidence for 7%; inconclusive evidence for 25.4%; evidence of ineffectiveness for 0.2%; and no evidence for 27.5%. Prescribing and child health promotion activities had the highest levels of quality evidence, and counselling/advice had the lowest.
CONCLUSIONS An encouraging amount of evidence was found to support much of community paediatric practice. This study improved on previous research in other specialties because actions other than medications and surgery were included.
- evidence based medicine
- community practice
- randomised controlled trials
Statistics from Altmetric.com
The office of technology assessment of the US Congress published a report in 1978 stating that: “only 10–20% of all procedures currently used in medical practice have been shown to be efficacious by controlled trial”.1 Since then, this figure has been challenged by a number of disciplines.2-4Claims have been made that 82% of interventions in inpatient general medicine are evidence based2 and 81% in general practice.3
On a recent evidence based paediatric course at the University of Leeds, these figures generated much discussion. “Guestimates” were made as to how clinically effective paediatric practice might be. The hospital paediatricians estimated that 49% of their work was evidence based, whereas the community paediatricians felt that only 20% of their work was likely to be so.5
The reason behind the community paediatricians’ lack of confidence was evident. Community paediatrics has evolved only over the last 20 years to become a consultant led specialty in its own right. As a relative newcomer, it was not likely to have developed the body of research and resources that underpin other specialties.
Therefore, with some apprehension that community paediatrics might prove to be at the bottom of the evidence based medicine league table, the community paediatricians decided to set out to ascertain the evidence base for this specialty.
In planning our study, the limitations of the previous studies were taken into account.2 3 In these studies, only the primary diagnosis and intervention (determined from retrospective review of notes) had been searched—a limitation of particular importance in the community setting. Moreover, it was not clear that evidence found had been rigorously critically appraised for quality. Finally, an overly broad definition of convincing non-experimental evidence had been used by the general practice paper.2
Therefore, our study design was structured so that all aspects of clinical activity undertaken by community paediatricians were studied, and efforts were made to ensure that all evidence was appraised for both quality and relevance to community paediatric patients.
In May 1997, a pilot study was undertaken to highlight any difficulties with data gathering and to develop the format of the clinical diary used. Nine paediatricians recorded every clinical action they undertook on patients over a two day period. From this pilot study, it was determined that there were 10 categories of clinical activity: clinical evaluation, investigations, prescriptions, referrals, monitoring, liaison, counselling/advice (including reassurance), child health promotion, statutory items, and multidisciplinary work.
The clinical group for the full study consisted of two consultants, two senior clinical medical officers/associate specialists, one senior registrar, and seven staff grade doctors/clinical medical officers. The children were seen in a variety of settings: schools/nurseries, special schools, hospital clinics, secondary referral clinics, health centres, child development centres, and at home.
During a two week period in June 1997, the 12 community paediatricians recorded data on every patient they saw during the first four days that they had patient contact. This time scale for recording allowed work done by part-time doctors to be represented as equally as the full timers. Details of all the child’s problems and clinical actions taken were recorded and classified according to the categories determined in the pilot study.
Questions were then developed—designed to test the efficacy of the clinical actions. Of the 10 categories, only six were considered to contain clinical actions suitable for posing questions of efficacy and literature searching. An attempt was made to investigate as many of the clinical actions as feasible, focusing on the more frequent and important actions. Evidence for efficacy was then sought by carrying out computerised data searches using medline, and in some instances cinahl, psychology, and the Cochrane databases as well. Further evidence was found by hand searching recent articles, obtaining references from relevant articles, consulting experts in the field, and personal communication with colleagues.
Relevant articles identified were critically appraised, using theJAMA guidelines,6 by the individual doing the search. The group then discussed each article both for the quality of the research and its relevance to the patients concerned. The article was graded by consensus according to quality criteria developed by the group (table 1). In all cases where evidence from a good systematic review, a randomised controlled trial or other appropriate design was found (A1 evidence) the article was independently critically appraised by three members of the group. A1 status was only applied if all three agreed that the study was of good design and directly relevant to community paediatrics. Clinical actions were only classified as having convincing non-experimental evidence (A2) if : (1) in the case of prescriptions, a randomised controlled trial with a placebo arm would be unethical; or (2) in the case of clinical evaluation, a study would essentially be impossible, either because the clinical evaluation itself is the gold standard for diagnosis (for example, obesity) or because the definitive study would involve children undergoing hazardous, expensive investigations to ascertain the accuracy of diagnosis, which would also be unethical.
During the study period, 247 children were seen with 446 problems and a total of 1149 clinical actions were carried out. One hundred and forty questions were posed, which addressed 629 clinical actions. Table2 shows the numbers of questions by category and the corresponding numbers of clinical actions. Table 3 shows the quality of evidence determined for each category.
There was good evidence from a randomised controlled trial, systematic review, or another appropriate study for 39.9% of the clinical actions studied, and convincing non-experimental evidence for 7%. A further 9.7% of evidence was of good quality but was classed as inconclusive until the research findings could be confirmed in the community paediatric setting. Table 4 details the A1 evidence.
Figure 1 shows the good quality evidence by category. A substantial amount of good quality evidence was found for child health promotion and prescriptions, whereas least evidence was found to support counselling and advice. Even when immunisations were excluded from the analysis of child health promotion, 40% of actions were graded A1.
Most of the interventions evaluated in general practice and inpatient general medicine were prescribed medications. Good experimental evidence was found for 53% of interventions in the general medicine paper2 and 31% of interventions in the general practice paper.3 Our study has shown that there is good experimental evidence for 53% of the prescriptions issued during the study period in community paediatrics. On a very simplistic basis it would be easy to state that there is as much good experimental evidence for community paediatric practice as the other specialties who have studied their evidence base.
However, our study was a refinement on the previously mentioned studies. In our study every clinical action carried out on patients was recorded, not just the action which on review seemed to be the principal one. Second, the quality of the evidence accepted was carefully assessed. Experimental evidence of effectiveness was only accepted if the study was methodologically sound and where the results were directly translatable to patients in a community paediatric setting.
Interestingly, because all clinical actions were recorded in our study, it was apparent that prescriptions accounted for only 3.7% of clinical actions performed during the study period. Clearly, stating that a specialty is evidence based when only the evidence of the effectiveness of treatments has been considered does not give the whole picture. There should also be clear evidence that diagnostic processes are effective (both clinical evaluation and relevant investigations), and for other components of the consultation, such as advice or reassurance given and referral to other professionals.
Community paediatrics involves more than seeing and treating acutely unwell children and making them well. Child health promotion, counselling and advice, and referral to other professionals are as important. They are included in the calculation that there is good evidence of effectiveness for 47%, and some evidence of effectiveness for a further 25% of the clinical actions in this community paediatric study.
In some areas disappointingly little evidence was found. Only 4% of the clinical evaluations made and 19% of the investigations carried out had clear evidence of accuracy or effectiveness in the community paediatric setting. Furthermore, the study highlighted the difficulties involved in assessing the accuracy of these diagnostic processes where studies would essentially be impossible to design. Further areas lacking in evidence were counselling/advice and referrals to other professionals.
It must be emphasised, however, that absence of evidence is not evidence of ineffectiveness,38 and the conclusion should therefore be that further research is required, rather than that these aspects of community paediatric practice are not worthwhile. Monitoring, liaison, multidisciplinary work, and statutory work are also an important part of community paediatrics. They were not evaluated in our study, and it is likely that the effectiveness or value of this type of work may best be answered by qualitative rather than quantitative research methods.
It was clear that overall there was a paucity of evidence from studies that had been carried out in the community or ambulatory paediatric setting. This meant that some good evidence was not given A1 status because we could not deem that the results were applicable to our patients. Examples included studies where counselling had been provided by a psychologist, or clinical evaluations had been carried out by tertiary specialists and where it could not be assumed that a community paediatrician would be similarly effective. Much research from hospital settings must be substantiated in the community setting before it can be considered applicable to community paediatricians and our broad spectrum of patients.
It is worthy of mention that so much of the clinical work of community paediatricians (identified by the process of keeping diaries) was found to be very similar to the work conventionally thought to be the province of hospital paediatricians. This suggests that the results of our study might well be extrapolated to the specialty of paediatrics as a whole.
The design of our study, which followed that of the studies in inpatient general medicine and general practice has both strengths and limitations. Using clinical encounters as the basis for searching for evidence ensures that common and important questions are answered. Reassuringly higher levels of evidence were found in all studies based on this design than those figures based on assessments of interventions that are high tech, high profile, and expensive.1 2
However, limitations were identified. Community child health tends to be seasonal and, because all the data collection for our study was carried out towards the end of the academic year, statutory work in school was underrepresented. Also underrepresented, by chance, were child protection and multidisciplinary work, both important aspects of community paediatrics. Our study also highlighted the need to ensure that clinical questions are well thought out and appropriate before searching databases.
Our study emphasises the need to ascertain the evidence underpinning all clinical actions, not simply “principal” interventions, which are often the easiest to search or indeed research. It should also be noted that searching for evidence can be time consuming. In preparing this paper 151 hours were spent doing electronic literature searches alone. It was estimated that these hours would be tripled when the time taken to find the journal articles and appraise them was included. Less than 50% of the group had access to a database at work or at home, and a number of journeys had to be made to the British Library to retrieve articles from some journals. If evidence based medicine is to become a reality, clinicians need easy access to electronic databases and good library support.
The authors are community paediatricians working in Yorkshire, Manchester, Teesside, and Cheshire. They are all students or staff in the Academic Unit of Paediatrics and Child Health, University of Leeds. This paper was a collaborative effort with all authors contributing to conception, design, analysis, and interpretation of the data; drafting and/or revising the article; and approval of the final version.