Results

CHARACTERISTICS OF PAEDIATRICIANS

Eighty per cent of the office based paediatricians contacted enrolled in the study, a total of 30 subjects. Of these, 13 participated in the first session, 11 in the second, and six in the third. There were eight men and 22 women, with a median age of 53 years (range 40–76 years). Seventy per cent were trained in Canada. In addition to completing a paediatric residency training, 45% had some subspecialty training (none in cardiology) for a median duration of 12 months (range 12–24 months). While all were in office based practice, 86% spent > 70% of their time providing primary care, 10% spent > 70% of their time providing consultant care, and 4% spent > 70% of their time providing paediatric subspecialty consultant care. Nearly all (97%) indicated that they had teaching responsibilities to medical students or residents.

The majority (90%) had a paediatric cardiology consultant affiliated with the hospital at which they had admitting privileges. Thirty three per cent of paediatricians indicated that they could order an echocardiogram directly without a cardiology referral; this service was provided by a paediatric cardiologist in 67%, radiologist in 22%, and an adult cardiologist in 11%. To the question “In your opinion, how helpful are each of the following tests in the routine evaluation of children with presumed innocent heart murmurs?”, 100% believed auscultation was very important, compared with 45% for an electrocardiogram (ECG), 31% for a chest radiograph, and 24% for an echocardiogram. When asked “Do you tend to agree with the following statement: Most children with presumed innocent heart murmurs should have an echocardiogram and/or consultation with a paediatric cardiologist for medico-legal reasons”, responses selected were strongly agree 0%, agree 18%, disagree 57%, and strongly disagree 25%.

In response to the question “For a child with a presumed innocent heart murmur, how often do you further investigate (echocardiography or referral to a cardiology subspecialist) under the following circumstances:”, median (range) rankings from 1 = always to 6 = never for “sibling with organic heart disease” was 2 (1–5), “positive family history of heart disease” was 3 (1–5), “pre-sport evaluation” was 4 (1–6), “parental request” was 2 (1–5), “unusual anxiety” was 3 (1–6), and “preoperative evaluation” was 3 (1–6). When asked to rate their confidence in their clinical assessment of heart murmurs from 1 = very confident to 6 = not at all confident, median (range) score for assessment of neonates was 4 (2–6), infants 3 (2–6), children 2 (1–4), and adolescents 2 (1–4).

CHARACTERISTICS OF PATIENTS

Thirty seven asymptomatic children with heart murmurs participated in the study. Twenty one (57%) children had innocent heart murmurs. Their median age was 3.7 years (range 1–15.5 years). Sixteen (43%) children had pathological heart murmurs and were from 1–7.8 years of age, with a median of 5.5 years. Specific lesions included ventricular septal defect in five patients, pulmonary valve stenosis in four, aortic valve stenosis in three, subaortic stenosis in two, and one patient each with bicuspid aortic valve and mitral valve regurgitation. The proportions of patients with lesions and the types of lesions are similar to those of previously published studies of clinical assessment by paediatric cardiologists.2 5 6

ACCURACY OF CLINICAL ASSESSMENT

Each of the 30 office based paediatricians assessed five to nine patients, a total of 191 observations. Correct classification was achieved for 142 (74%) observations. The mean (SD) sensitivity was 82 (24)% with a mean specificity of 72 (24)%. To the question “How confident are you about your diagnosis (innocent versus pathological murmur) in this patient?”, paediatricians indicated entirely confident for 2% of observations, very confident for 18%, confident for 45%, unsure for 32%, and extremely unsure for 4%. Inter-rater agreement for paediatricians examining the same patient was very low, with a κ statistic of 0.05.

For each patient examined, paediatricians were asked to select their most probable approach if they were seeing this child in their own practice for the first time. They selected physical examination alone for 35% of the observations, chest radiographs would have been ordered for 49%, ECGs for 48%, echocardiograms for 41%, and a cardiology consultation obtained for 46% of observations. Thus paediatricians would have referred for echocardiography or cardiology consultation in 105 of the 191 (54%) observations, including 39% of the patients who had innocent heart murmurs and 77% of the patients with simple cardiac lesions. If it is assumed that patients having echocardiography or cardiology consultation would have accurate classification of their cardiac status, then allowing paediatricians to obtain echocardiography or refer for cardiology consultation for those study patients as above, this strategy would increase mean (SD) sensitivity to 87 (20)% and mean specificity to 98 (8)%.

FACTORS ASSOCIATED WITH ACCURACY

Sensitivity and specificity of clinical assessment were not significantly related to the following characteristics: sex, age, year of medical school graduation, subspecialty training, practice type, practice volume, availability of echocardiography, and number of patients referred for echocardiography per month. While sensitivity and specificity were not significantly related to the paediatricians’ overall level of confidence in assessing infants, children, and adolescents, decreased confidence in assessing newborns was significantly correlated with higher specificity (r = 0.40, p = 0.03) but not sensitivity (r = 0.17, p = 0.37). Higher sensitivity (r = 0.40, p = 0.03) and lower specificity (r = −0.47, p = 0.009) were significantly correlated with a greater proportion of study patients whom the paediatrician assessed that they would refer for echocardiography or cardiology consultation if seen in their own practice. Likewise, an increased number of echocardiograms obtained in the paediatricians’ usual practice was significantly correlated with an increased proportion of study patients assessed who might be referred (r = 0.41, p = 0.03). The proportion of study patients who might be referred was not significantly related to the paediatricians’ level of confidence in assessing murmurs in patients of any age group.

Discussion

Auscultation is an important clinical manoeuvre that is performed everyday by all office based paediatricians, yet the accuracy of clinical assessment in the hands of non-cardiologists has not been the object of extensive evaluation. Clinical assessment is the primary evaluation tool used in the initial detection and evaluation of heart murmurs in children, whose initial contact with the health care system is at a generalist level. Evidence currently available shows that electrocardiography and radiology have a sensitivity as low as 10% in the evaluation of well children with heart murmurs.4 5 10-12 Our study found a suboptimal sensitivity and specificity of clinical assessment in the group of 30 general paediatricians tested, and a tendency to rely on these other less useful tests. The mean sensitivity of 82% implies that 18% of patients with a cardiac abnormality would be missed, and the mean specificity of 72% implies that 28% of patients with innocent murmurs would be referred for further investigation or assessment. These results are deemed suboptimal compared to studies of consultant paediatricians and paediatric cardiologists. Hanson and colleagues13 studied two consultant paediatricians who examined 100 children and found the sensitivity and specificity of clinical assessment to be 96% and 68%, respectively. Four prospective descriptive studies have examined the clinical assessment skills of between three and eight paediatric cardiologists.2 4-6They found the sensitivity to range between 92% and 96% and the specificity between 82% and 95%. Another study using retrospective data found that 47% of children referred to paediatric cardiologists by primary care physicians for murmur evaluation had normal hearts.14

Given the suboptimal accuracy of clinical assessment by office based paediatricians, educational interventions targeted at improving their diagnostic skills should be developed, as well as improvements in the training of medical students and residents. Teaching the six cardinal clinical signs2 in the differentiation of heart murmurs in children to office based paediatricians may improve their clinical acumen. This could be done with some practical experience, as was the case in our research project, although it was not the purpose of this study to assess the teaching impact of such a programme.

The results of this study must be interpreted in the light of its limitations. Volunteer bias may have led to an overestimation of the sensitivity and specificity of clinical assessment if highly skilled clinicians were more likely to have participated in the study. Most of the study paediatricians were Canadian medical school graduates, and the applicability of their performance to those trained and in practice in other countries is not known. The paediatricians in this study were all in office based fee for service private practice, and the majority spent > 70% of their time providing primary care. In Ontario, the vast majority of children receive primary care by family physicians or general practitioners, who may refer to a paediatrician (most commonly) or directly to a paediatric cardiologist (infrequently) when consultation for heart murmur is required. Thus, patients referred to a paediatric cardiologist from office based paediatricians may represent patients seen by the latter either for primary or consultative care. The vast majority of the study paediatricians (97%) had teaching hospital affiliations, and taught medical students and resident trainees. All had completed paediatric residency training, and 45% had at least some further subspecialty training, but none in paediatric cardiology. In the Ontario health care system, the office based paediatrician is usually the physician acting as the secondary “screener” before referral. Thus, the study paediatricians resemble a UK consultant general paediatrician in hospital practice, albeit with a greater proportion of their practice devoted to primary care, often for more complex types of patients.

Our physician questionnaire revealed that the participants were highly motivated individuals with a strong commitment to education. People have a tendency to change their behaviour when they are the target of special interest and attention in a study, regardless of the specific nature of the intervention (the Hawthorne effect). The paediatricians in our study may have been more meticulous in their cardiac examination, knowing that they were being tested. Finally, when physicians know that a patient possesses certain clinical features, they may carry out more detailed searches for the relevant diagnosis leading to diagnostic suspicion bias.