AIM To determine the diagnostic accuracy of physical examination by office based (general practice) paediatricians in the evaluation of heart murmurs.
DESIGN Each of 30 office based paediatricians blindly examined a random sample of children with murmurs (43% of which were pathological). Sensitivity and specificity were calculated and were related to paediatricians’ characteristics.
RESULTS Mean (SD) sensitivity was 82 (24)% with a mean specificity of 72 (24)% in differentiating pathological from innocent murmurs, with further investigations requested for 54% of assessments. The addition of a referral strategy would have increased mean sensitivity to 87 (20)% and specificity to 98 (8)%. Diagnostic accuracy was not significantly related to the paediatricians’ age, education or practice characteristics, but was related to referral practices and confidence in assessment.
CONCLUSIONS Diagnostic accuracy of clinical assessment of heart murmurs by office based paediatricians is suboptimal, and educational strategies are needed to improve accuracy and reduce unnecessary referrals and misdiagnosis.
- heart murmurs
- diagnostic accuracy
- clinical assessment
Statistics from Altmetric.com
It is estimated that up to 80% of children have heart murmurs1 2 but only 0.35% have confirmed organic heart disease.3 Previous studies have shown that paediatric cardiologists can differentiate innocent from pathological murmurs, based on clinical examination, with high sensitivity and specificity.2 4-6 The accuracy of clinical assessment by office based paediatricians involved mainly in primary care practice is not known. This information is very important, as office based paediatricians provide primary and consultative care to children, and often “gatekeep” access to echocardiography or referral to paediatric cardiologists. Low sensitivity leads to missed disease, and may expose children with undiagnosed minor and major cardiac lesions to dangerous complications such as bacterial endocarditis, heart failure, and pulmonary vascular obstructive disease. Low specificity leads to overdiagnosis and inappropriate use of diagnostic testing and referrals, which reduce the cost effectiveness of clinical practice. Inaccurate diagnosis may result in undue patient and parental anxiety and psychological labelling when incorrectly identifying a child with an innocent murmur as having heart disease.7 8 Bergman studied this phenomenon and concluded that the degree of disability from “cardiac non-disease” in children was greater than that caused by actual heart disease.9 We sought to determine the diagnostic accuracy of physical examination of office based paediatricians in the differentiation of heart murmurs, and to relate this to physician characteristics and practices.
Thirty office based paediatricians involved mainly in community based primary care practice were recruited by telephone or mail, with the assistance of the Ontario Association of Pediatricians. They were asked to participate in a research study with continuing medical education, and were reimbursed for travel expenses. Ethics approval was obtained from the research ethics board of The Hospital for Sick Children, and all paediatricians, parents and children gave informed consent or assent as appropriate.
The study was conducted in three weekend half day sessions with 10 paediatricians per session. Upon arrival for their scheduled session, they completed a questionnaire regarding physician and practice characteristics.
Patients with asymptomatic heart murmurs were recruited for each session from the cardiac outpatient clinics of The Hospital for Sick Children, and the practices of two office based paediatric cardiologists and two office based paediatricians not participating in the study. Patients were deemed acceptable for inclusion if they had either an innocent murmur or a simple asymptomatic cardiac lesion as confirmed at consultation by a paediatric cardiologist with complete echocardiographic assessment. Patients were reimbursed for their expenses related to participation, and attended only one of the three weekend sessions.
Upon arrival for their scheduled session, both paediatricians and patients were randomly assigned to three groups. A tertiary centre based paediatric cardiologist (BWM) then met each patient and their family members and confirmed and noted physical findings. Patients and parents were instructed not to disclose medical information. Pediatricians were instructed that all patients were asymptomatic with normal vital signs, and that clinical assessment was to be limited to precordial palpation and auscultation. Each paediatrician independently assessed the five to nine patients randomly assigned to their subgroup, and recorded their assessments on a standardised data collection form. The paediatricians were requested to record their physical findings, state whether or not they felt the child had a cardiac lesion, define the specific lesion type, and estimate how confident they were in their assessment. After completing the assessments, they were given feedback as a group by the paediatric cardiologist (BWM) who reviewed the physical findings and diagnoses of the patients assessed. A didactic teaching session followed related to clinical skills and strategies related to the outpatient evaluation of heart murmurs in children.
Data are described as frequencies, medians with ranges, and means with standard deviations as appropriate. Sensitivity and specificity in differentiation of heart murmurs were calculated for each paediatrician. For patients evaluated by the same paediatrician, inter-rater agreement was calculated with a κ statistic. Sensitivity and specificity of paediatricians were related to physician and practice characteristics using Kruskal-Wallis analysis of variance,t tests, and linear regression analysis. Significance was set at the level of p < 0.05.
CHARACTERISTICS OF PAEDIATRICIANS
Eighty per cent of the office based paediatricians contacted enrolled in the study, a total of 30 subjects. Of these, 13 participated in the first session, 11 in the second, and six in the third. There were eight men and 22 women, with a median age of 53 years (range 40–76 years). Seventy per cent were trained in Canada. In addition to completing a paediatric residency training, 45% had some subspecialty training (none in cardiology) for a median duration of 12 months (range 12–24 months). While all were in office based practice, 86% spent > 70% of their time providing primary care, 10% spent > 70% of their time providing consultant care, and 4% spent > 70% of their time providing paediatric subspecialty consultant care. Nearly all (97%) indicated that they had teaching responsibilities to medical students or residents.
The majority (90%) had a paediatric cardiology consultant affiliated with the hospital at which they had admitting privileges. Thirty three per cent of paediatricians indicated that they could order an echocardiogram directly without a cardiology referral; this service was provided by a paediatric cardiologist in 67%, radiologist in 22%, and an adult cardiologist in 11%. To the question “In your opinion, how helpful are each of the following tests in the routine evaluation of children with presumed innocent heart murmurs?”, 100% believed auscultation was very important, compared with 45% for an electrocardiogram (ECG), 31% for a chest radiograph, and 24% for an echocardiogram. When asked “Do you tend to agree with the following statement: Most children with presumed innocent heart murmurs should have an echocardiogram and/or consultation with a paediatric cardiologist for medico-legal reasons”, responses selected were strongly agree 0%, agree 18%, disagree 57%, and strongly disagree 25%.
In response to the question “For a child with a presumed innocent heart murmur, how often do you further investigate (echocardiography or referral to a cardiology subspecialist) under the following circumstances:”, median (range) rankings from 1 = always to 6 = never for “sibling with organic heart disease” was 2 (1–5), “positive family history of heart disease” was 3 (1–5), “pre-sport evaluation” was 4 (1–6), “parental request” was 2 (1–5), “unusual anxiety” was 3 (1–6), and “preoperative evaluation” was 3 (1–6). When asked to rate their confidence in their clinical assessment of heart murmurs from 1 = very confident to 6 = not at all confident, median (range) score for assessment of neonates was 4 (2–6), infants 3 (2–6), children 2 (1–4), and adolescents 2 (1–4).
CHARACTERISTICS OF PATIENTS
Thirty seven asymptomatic children with heart murmurs participated in the study. Twenty one (57%) children had innocent heart murmurs. Their median age was 3.7 years (range 1–15.5 years). Sixteen (43%) children had pathological heart murmurs and were from 1–7.8 years of age, with a median of 5.5 years. Specific lesions included ventricular septal defect in five patients, pulmonary valve stenosis in four, aortic valve stenosis in three, subaortic stenosis in two, and one patient each with bicuspid aortic valve and mitral valve regurgitation. The proportions of patients with lesions and the types of lesions are similar to those of previously published studies of clinical assessment by paediatric cardiologists.2 5 6
ACCURACY OF CLINICAL ASSESSMENT
Each of the 30 office based paediatricians assessed five to nine patients, a total of 191 observations. Correct classification was achieved for 142 (74%) observations. The mean (SD) sensitivity was 82 (24)% with a mean specificity of 72 (24)%. To the question “How confident are you about your diagnosis (innocent versus pathological murmur) in this patient?”, paediatricians indicated entirely confident for 2% of observations, very confident for 18%, confident for 45%, unsure for 32%, and extremely unsure for 4%. Inter-rater agreement for paediatricians examining the same patient was very low, with a κ statistic of 0.05.
For each patient examined, paediatricians were asked to select their most probable approach if they were seeing this child in their own practice for the first time. They selected physical examination alone for 35% of the observations, chest radiographs would have been ordered for 49%, ECGs for 48%, echocardiograms for 41%, and a cardiology consultation obtained for 46% of observations. Thus paediatricians would have referred for echocardiography or cardiology consultation in 105 of the 191 (54%) observations, including 39% of the patients who had innocent heart murmurs and 77% of the patients with simple cardiac lesions. If it is assumed that patients having echocardiography or cardiology consultation would have accurate classification of their cardiac status, then allowing paediatricians to obtain echocardiography or refer for cardiology consultation for those study patients as above, this strategy would increase mean (SD) sensitivity to 87 (20)% and mean specificity to 98 (8)%.
FACTORS ASSOCIATED WITH ACCURACY
Sensitivity and specificity of clinical assessment were not significantly related to the following characteristics: sex, age, year of medical school graduation, subspecialty training, practice type, practice volume, availability of echocardiography, and number of patients referred for echocardiography per month. While sensitivity and specificity were not significantly related to the paediatricians’ overall level of confidence in assessing infants, children, and adolescents, decreased confidence in assessing newborns was significantly correlated with higher specificity (r = 0.40, p = 0.03) but not sensitivity (r = 0.17, p = 0.37). Higher sensitivity (r = 0.40, p = 0.03) and lower specificity (r = −0.47, p = 0.009) were significantly correlated with a greater proportion of study patients whom the paediatrician assessed that they would refer for echocardiography or cardiology consultation if seen in their own practice. Likewise, an increased number of echocardiograms obtained in the paediatricians’ usual practice was significantly correlated with an increased proportion of study patients assessed who might be referred (r = 0.41, p = 0.03). The proportion of study patients who might be referred was not significantly related to the paediatricians’ level of confidence in assessing murmurs in patients of any age group.
Auscultation is an important clinical manoeuvre that is performed everyday by all office based paediatricians, yet the accuracy of clinical assessment in the hands of non-cardiologists has not been the object of extensive evaluation. Clinical assessment is the primary evaluation tool used in the initial detection and evaluation of heart murmurs in children, whose initial contact with the health care system is at a generalist level. Evidence currently available shows that electrocardiography and radiology have a sensitivity as low as 10% in the evaluation of well children with heart murmurs.4 5 10-12 Our study found a suboptimal sensitivity and specificity of clinical assessment in the group of 30 general paediatricians tested, and a tendency to rely on these other less useful tests. The mean sensitivity of 82% implies that 18% of patients with a cardiac abnormality would be missed, and the mean specificity of 72% implies that 28% of patients with innocent murmurs would be referred for further investigation or assessment. These results are deemed suboptimal compared to studies of consultant paediatricians and paediatric cardiologists. Hanson and colleagues13 studied two consultant paediatricians who examined 100 children and found the sensitivity and specificity of clinical assessment to be 96% and 68%, respectively. Four prospective descriptive studies have examined the clinical assessment skills of between three and eight paediatric cardiologists.2 4-6They found the sensitivity to range between 92% and 96% and the specificity between 82% and 95%. Another study using retrospective data found that 47% of children referred to paediatric cardiologists by primary care physicians for murmur evaluation had normal hearts.14
Given the suboptimal accuracy of clinical assessment by office based paediatricians, educational interventions targeted at improving their diagnostic skills should be developed, as well as improvements in the training of medical students and residents. Teaching the six cardinal clinical signs2 in the differentiation of heart murmurs in children to office based paediatricians may improve their clinical acumen. This could be done with some practical experience, as was the case in our research project, although it was not the purpose of this study to assess the teaching impact of such a programme.
The results of this study must be interpreted in the light of its limitations. Volunteer bias may have led to an overestimation of the sensitivity and specificity of clinical assessment if highly skilled clinicians were more likely to have participated in the study. Most of the study paediatricians were Canadian medical school graduates, and the applicability of their performance to those trained and in practice in other countries is not known. The paediatricians in this study were all in office based fee for service private practice, and the majority spent > 70% of their time providing primary care. In Ontario, the vast majority of children receive primary care by family physicians or general practitioners, who may refer to a paediatrician (most commonly) or directly to a paediatric cardiologist (infrequently) when consultation for heart murmur is required. Thus, patients referred to a paediatric cardiologist from office based paediatricians may represent patients seen by the latter either for primary or consultative care. The vast majority of the study paediatricians (97%) had teaching hospital affiliations, and taught medical students and resident trainees. All had completed paediatric residency training, and 45% had at least some further subspecialty training, but none in paediatric cardiology. In the Ontario health care system, the office based paediatrician is usually the physician acting as the secondary “screener” before referral. Thus, the study paediatricians resemble a UK consultant general paediatrician in hospital practice, albeit with a greater proportion of their practice devoted to primary care, often for more complex types of patients.
Our physician questionnaire revealed that the participants were highly motivated individuals with a strong commitment to education. People have a tendency to change their behaviour when they are the target of special interest and attention in a study, regardless of the specific nature of the intervention (the Hawthorne effect). The paediatricians in our study may have been more meticulous in their cardiac examination, knowing that they were being tested. Finally, when physicians know that a patient possesses certain clinical features, they may carry out more detailed searches for the relevant diagnosis leading to diagnostic suspicion bias.
This study was supported in part by funds from Bristol-Myers-Squibb Pharmaceutical Group, Canada, The Hospital for Sick Children Research Institute Training Fellowship, and the Trainee Start-Up Fund.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.