Objectives The clinical diagnosis of pneumonia lacks specificity and may lead to antibiotic overuse, whereas radiological diagnoses can lack sensitivity. Point-of-care lung ultrasound is an emerging diagnostic tool. There are limited prospective data, however, on the accuracy of sonologists in the paediatric emergency department setting. We aimed to test the diagnostic accuracy of lung ultrasound for pneumonia using chest radiograph (CR) as the reference standard.
Methods This prospective observational cohort study in a paediatric emergency department enrolled children aged 1 month to <18 years, who had a CR ordered for possible pneumonia. Lung ultrasounds were performed by two blinded sonologists with focused training. Sonographic pneumonia was defined as lung consolidation with air bronchograms. Radiograph and ultrasound results both required agreement between two readers, with final results determined by an arbiter in cases of disagreement. Patient management was decided by treating clinicians who were blinded to lung ultrasound results. Follow-up was performed by phone and medical record review to obtain final diagnosis and antibiotic use.
Results Of 97 included patients, CR was positive for pneumonia in 44/97 (45%) and lung ultrasound was positive in 57/97 (59%). Ultrasound sensitivity was 91% (95% CI 78% to 98%) and specificity was 68% (95% CI 54% to 80%). Ultrasound results displayed greater consistency with CR and patient outcomes when sonographic consolidation exceeded 1 cm. Thirteen of 57 patients with sonographic consolidation improved without antibiotics.
Conclusion Lung ultrasound may have a role as first-line imaging in patients with possible pneumonia, with higher specificity for consolidations exceeding 1 cm.
Trial registration number ACTRN12616000361404, http://www.ANZCTR.org.au/ACTRN12616000361404.aspx
- paediatric emergency medicine
- point-of-care testing
Statistics from Altmetric.com
What is already known on this topic?
There is currently no feasible ‘gold standard’ for diagnosing pneumonia.
Lung ultrasound exhibits good diagnostic accuracy for consolidation when compared with clinical and radiological diagnoses.
Evidence is however limited to support use by sonologists in the paediatric emergency department.
What this study adds?
We performed a prospective observational cohort study in a paediatric emergency department with lung ultrasounds completed by blinded sonologists with focused training.
These demonstrated similar test characteristics to prior research.
A secondary outcome suggests some patients with sonographic consolidations do not require antibiotics.
Worldwide, paediatric pneumonia is the greatest infectious cause of death in children under 5 years.1 It has an incidence of 0.22 episodes/child-year in low/middle-income countries and 0.015 episodes/child-year in high-income countries.2 In Australia, where this study was completed, children are hospitalised for pneumonia at a rate of 0.85–5.92/1000 person-years, with the highest rates in <2 year-olds.3 International guidelines recommend that the diagnosis be made clinically in uncomplicated disease.4 5 However, clinical criteria alone may overlap with viral infection and lead to unnecessary antibiotic use.6 Radiological imaging is often used to support clinical findings, even though chest radiographs (CR) lack sensitivity7 and may not alter patient outcomes.8 Furthermore, no widely accepted nor clinically validated definition of a ‘positive’ CR exists. CT may be considered the diagnostic ‘gold standard’, but is uncommonly used in children.
Lung ultrasound (LUS) is an emerging diagnostic tool for pneumonia, with growing evidence in paediatric settings.7 9–31 LUS offers many advantages over CR and CT: it is low cost, radiation free and more readily available.13 21 However, evidence for the accuracy of inexperienced sonologists in an emergency department (ED) setting is limited, and the clinical significance of small sonographic consolidations remains uncertain.27
We aimed to contribute evidence for the accuracy of point-of-care LUS in the diagnosis of pneumonia, using CR as the reference standard, and guide future research that may reduce CR rates and improve antibiotic stewardship.
Study design and participants
We conducted a prospective observational cohort study in an Australian tertiary paediatric ED with an annual census of 89 000. Participants were enrolled from March to July 2016 as a convenience sample, with recruitment determined by sonologist availability. To identify patients, study sonologists were notified of CR requests sent by the ED. Included participants were aged 1 month to 18 years and received a CR for possible pneumonia. Children with a prior CR for the same illness or requiring life support were excluded. A parent/guardian information statement was provided and written informed consent was obtained before data collection. Children ≥12 years were given the opportunity to give written consent. The study was approved and performed in accordance with our institutional ethics committee, and complies with the STARD (Standards for Reporting of Diagnostic Accuracy Studies) protocol for studies of diagnostic accuracy.32
All patients received a single anteroposterior CR. On a standardised form, the treating clinician recorded clinical findings, CR findings of consolidation, interstitial infiltrates and pleural effusion, and an interpretation of whether the radiological findings were consistent with pneumonia. The clinician was blinded to LUS findings unless the sonologist identified findings that may have required an intervention and for which a delay in diagnosis may have led to clinical deterioration. The clinician made patient management decisions according to their normal practice and without knowledge of the LUS findings. The CR was later reported by the duty paediatric radiologist who was aware of the CR indication and patient demographics, but blinded to the LUS result. The duty radiologists’ reports, normally only available after initial management decisions had been made, were analysed using WHO definitions and a modified diagnostic protocol, as described in online supplementary file 1.33 The final radiological diagnosis required agreement between the treating clinician and the radiologist’s report. If results were discordant, or either result was indeterminate, an arbiter radiologist determined the final result. The arbiter was blinded to all information other than patient age. The study flow is presented in figure 1.
Ultrasounds were performed within 12 hours of the CR. Sonologists were a first-year paediatric emergency medicine fellow and a final-year medical student, both without prior ultrasound experience but trained specifically for this study by an emergency physician with 5 years of point-of-care ultrasound experience and a Diploma in Diagnostic Ultrasound. Training involved viewing a 1-hour online open-access video on LUS principles and findings created by an ED physician34 and a 1-hour practical tutorial. The latter involved demonstration and practice of LUS technique on consenting live paediatric models with normal lungs. Prior to recruiting, the enrolling sonologists each completed 10 LUS which were reviewed for acceptability of image acquisition and interpretation of both normal and pathological findings (online supplementary file 1). LUS included the upper and lower zones of the anterior, lateral and posterior lungs in longitudinal and oblique orientations.12 Ultrasounds were performed with a Zonare z.one ultra (Zonare Medical Systems, CA, 2013) using an L14-5w linear transducer. Ultrasound findings were recorded for each zone and orientation, using definitions determined prior to recruitment (online supplementary files 1–3). A sonographic diagnosis of pneumonia required lung consolidation with air bronchograms. The subpleural depth of consolidations was measured to allow subanalysis of consolidations >1 cm. Sonologists did not access patient’s medical records, discuss the patient with the treating clinician or parent/s or review the CR. However, complete blinding was not possible in the clinical environment. All ultrasounds were also interpreted by the aforementioned ED physician. If there was disagreement between the interpretations, the final result was arbitrated by a second sonologist with fellowship training in paediatric emergency point-of-care ultrasound and 5 years of clinical sonology experience. The reviewer and arbiter sonologists were blinded to all clinical and imaging findings.
Follow-up data were obtained from the patient’s medical record and a telephone call to the parent/guardian within 2–3 weeks. For further analysis, we identified patients as positive for pneumonia if they had either a positive CR or a final clinical diagnosis (pneumonia or lower respiratory tract infection (LRTI)), and were treated with antibiotics.
The primary outcome was the diagnostic accuracy of LUS, using CR as the reference standard. Final imaging interpretations were used for this calculation. Two subanalyses were performed. The first required a positive LUS to have a subpleural depth of >1 cm of consolidation27 and the second compared accuracy for the first half of patients scanned by each sonologist with the second half of patients scanned. Secondary outcomes included: (A) intertest reliability between CR and LUS, (B) rates of antibiotic use and (C) admission to an inpatient unit.
Categorical data were summarised using percentages, and continuous non-parametric data were summarised using median and IQR. Diagnostic accuracy was reported as sensitivity, specificity, positive/negative predictive values and likelihood ratios. Patients were excluded if their LUS was incomplete. Cohen’s kappa was used to assess intertest reliability between CR and LUS. With advice from a statistician, the intended sample size was 80–100 participants based on a minimum agreement of CR and LUS of 50%, resulting in a 95% CI of 34.5% to 65.5%. Data were analysed using Stata (V.14.2, StataCorp, TX, USA, 2015).
We considered 142 participants for eligibility (figure 1). Thirty-eight patients were excluded prior to LUS. Ultrasound was incomplete in four patients due to accidental omission of one to two lung views and consent was withdrawn in three. Ninety-seven patients had complete data for the primary outcome. Demographic and clinical characteristics are presented in table 1. CR was positive for pneumonia in 44/97 patients (45%) and LUS was positive in 57/97 (59%) (online supplementary file 4). Admission to an inpatient unit occurred in 23/44 (52%) patients with positive CRs and 23/53 (43%) patients with negative/indeterminate CRs. Ultrasounds were performed a median of 0.6 hour after the CR (IQR 0.3–1.2 hours). One sonologist performed 48 LUS; the other performed 49. The arbiter sonologist agreed with the reviewing sonologist in 13/15 LUS where there were initial discordant interpretations. The treating clinician was notified of LUS findings in three patients: two had pleural effusions potentially requiring drainage and the other had a lung abscess; however, these findings had already been identified on CR and were managed conservatively. No adverse events were identified. The follow-up phone call was completed in 94/97 patients (97%).
Lung consolidations were identified by the reviewing sonologist in 82/97 patients (85%) of which 57/82 (70%) also had air bronchograms. In the 25/82 patients (30%) with consolidation but no air bronchograms, three had a positive CR and a further two had a clinical diagnosis of pneumonia/LRTI. The consolidation was <1 cm in all 25 patients. B-lines and pleural line abnormalities were present, respectively, in 41/57 (72%) and 33/57 (58%) of positive LUS exams and 18/40 (45%) and 31/40 (78%) of negative LUS exams. Pleural effusion was identified in 34/57 (60%) of positive LUS exams and 16/40 (40%) of negative LUS exams. Pleural effusions were not quantified.
The sensitivity and specificity of the LUS diagnosis of pneumonia, when compared with the reference standard of CR, were 91% (95% CI 78% to 98%) and 68% (95% CI 54% to 80%), respectively (table 2). For the analysis of consolidations >1 cm, sensitivity decreased to 71% (95% CI 55% to 83%) and specificity increased to 85% (95% CI 72% to 93%). The diagnostic accuracy achieved in the first half of the study had a higher sensitivity and lower specificity than in the latter half of the study (table 2, online supplementary file 5). Intertest reliability between CR and LUS interpretations was moderate; κ=0.57 (95% CI 0.38 to 0.70).35
Of the four patients (4%) with false negative LUS results, all had discordant initial CR interpretations, and one patient had discordant LUS results (online supplementary files 6 and 7). Two patients in this group had chronic cardiorespiratory conditions and all were diagnosed with an LRTI and treated with antibiotics. The radiologist reported perihilar interstitial infiltrates on all four CRs. In addition, one patient had retrocardiac interstitial thickening, another had patchy bilateral lower zone atelectasis and another had patchy consolidation in the right middle lobe and lingula. On review, the LUS were confirmed to be adequate and complete studies.
Of the 17 patients (18%) with false positive LUS, two had discordant LUS results and four had discordant CR interpretations. Five patients had chronic cardiorespiratory conditions. Six patients received antibiotics for pneumonia/LRTI despite their treating clinicians considering their CRs negative/indeterminate (online supplementary files 8 and 9). Two further patients received antibiotics: one was treated for lymphadenitis though the duty radiologist reported consolidation on their CR; the other was treated for pneumonia when a CR at 48 hours showed consolidation. The remaining nine patients were variously diagnosed with asthma, bronchiolitis, upper respiratory tract infection, chest pain or viral illness and improved without receiving antibiotics.
LUS findings and follow-up outcomes
When sonographic consolidation was >1 cm, 34/39 patients (87%) had a clinical diagnosis of pneumonia/LRTI and/or a positive CR, and were treated with antibiotics (figure 2, online supplementary file 10). The other 5/39 patients (13%) improved without antibiotics and their treating clinicians considered their CRs negative. However, the duty radiologist reported consolidation for one patient and atelectasis in two, with these radiographic findings corresponding to the location of the sonographic consolidation (figure 3, online supplementary files 11 and 12).
When sonographic consolidation was ≤1 cm, 10/18 patients (56%) had a clinical diagnosis of pneumonia/LRTI and/or a positive CR and received antibiotics. The remaining 8/18 patients (44%) improved without antibiotics. In this group, the duty radiologist reported consolidation on one CR and atelectasis on two CRs. Findings again corresponded to the location of sonographic findings.
Our results suggest that point-of-care LUS has promising accuracy for the diagnosis of paediatric pneumonia when performed by sonologists with focused training. Diagnostic accuracy reported in the literature varies considerably with point sensitivities of 40%–100% and specificities of 44%–100%.7 9–15 17 19 20 23–29 31 The range in findings may be due to different training and levels of experience, different patient populations and clinical settings and also different definitions of both LUS and CR findings. A similar study by Shah et al 27 found a subgroup of novice sonographers in a paediatric ED setting had a sensitivity of 83% (95% CI 63% to 93%) and a specificity of 88% (95% CI 81% to 93%). When LUS is compared with the diagnostic ‘gold-standard’ of CT, LUS sensitivity and specificity for consolidation has been reported as 77% (95% CI 62% to 93%) and 75% (95% CI 64% to 84%), respectively.7 However, this study was performed on hospitalised patients with only 27% of them requiring CTs, suggesting these findings may not be generalisable to our population. Other studies have also combined clinical and imaging diagnoses, with or without laboratory findings, as their reference standard.9 10 12 19 23 25 28–30 Although this may address CR’s poor sensitivity, we used only CR as the reference for diagnostic accuracy as clinical diagnoses may be oversensitive, and improvement after antibiotics may incorrectly infer a course consistent with bacterial pneumonia in patients with viral illnesses.
Our false negative results may have arisen from axillary, subscapular or subclavicular consolidations inaccessible to ultrasound imaging, or from perihilar consolidations that did not reach the pleura. These are recognised limitations of LUS.36 Sonologist’s inexperience may have also reduced sensitivity. These patients did however have potentially equivocal CRs and were clinically diagnosed with and managed as having pneumonia. Lastly, chronic conditions in two patients may have complicated imaging interpretation.
There are several possible explanations for our false positives. The clinical diagnosis and treatment with antibiotics in 8/17 (47%) of these patients suggests LUS may have been accurate, and CR may have underdiagnosed small or early consolidations. Furthermore, the overlapping CR and LUS appearances of consolidation and atelectasis, and pneumonia and bronchiolitis, may have been complicating factors.29 37 38 There is also the potential for misinterpretation of lobar fissures with adjacent consolidation as consolidation with air bronchograms in the two patients for whom the LUS findings were discordant. Additionally, when compared with studies recruiting only inpatients, our sample may have had earlier and/or less severe disease with less specific LUS findings.7 10 14 17 19 23 25 28 Some studies used lateral CR in addition to frontal projections14 15 20 27 31 which likely increased their CR sensitivity to findings in the left lower posterior chest, an area Milliner and Tsung identify as the location of approximately 15% of lung consolidations (online supplementary file 8).39
Lung consolidation alone, without the requirement for air bronchograms, is sometimes used to diagnose sonographic pneumonia.12 13 19 20 23 28 We identified consolidation in 85% of patients, 24% of whom had neither air bronchograms, nor a positive CR or clinical diagnosis. Consolidation without air bronchograms most likely represents atelectasis and, as such, is a non-specific finding. The frequency of B-lines and pleural line abnormalities in our negative ultrasound group suggests the definition of sonographic pneumonia used by some studies may lead to overdiagnosis.7 10 14 18 25 29 Dynamic air bronchograms have good specificity for pneumonia in adults (94%) and allow differentiation from resorptive atelectasis, but their poor sensitivity (61%) limits their clinical utility.40
Requiring >1 cm for a positive finding led to improved specificity and a closer correlation with positive clinical diagnoses. This cut-off arose from evidence that CR may not identify subcentimetre lesions,27 and has been used to avoid reducing LUS specificity when CR is the reference standard.11 21 26 27 31 Zhan et al 31 also showed good specificity (91%, 95% CI 83% to 96%) when requiring >1 cm of sonographic consolidation with or without air bronchograms for a positive diagnosis. Their sensitivity was however lower at 40% (95% CI 30% to 51%). Our sensitivity was lower for consolidations >1 cm as CR did identify subcentimetre consolidations in nine patients, increasing our ‘false positive’ rate from 4 to 13. In contrast, Shah et al 27 maintained a sensitivity of 86% (95% CI 71% to 94%) in their subanalysis of consolidations >1 cm as all patients with <1 cm findings had negative CRs. Their specificity also improved from 89 (95% CI 83% to 93%) to 97% (95% CI 93% to 99%).
A potentially important new finding was that some patients with sonographic consolidation improved without antibiotics. Tsung et al 41 proposed that subcentimetre consolidations are associated with viral pneumonia whereas larger consolidations with air bronchograms are associated with bacterial pneumonia, although no prospective studies have investigated this hypothesis. Our findings partially support this proposition: 44% of patients with subcentimetre consolidations improved without antibiotics and the blinding of our treating clinicians to these LUS results ensured that diagnosis and treatment were not downstream effects of the LUS results.
The results of this study suggest that with time and experience sonologists improve their specificity at the expense of sensitivity. We postulate the decreased sensitivity may occur with increased confidence and potentially less time spent scanning, and that improved specificity may be due to improvement in interpretation of LUS findings.
This research supports the hypothesis that patients with possible pneumonia and a positive LUS may not require CR, though we suggest this would be most appropriate when lung sonographic consolidations exceed 1 cm. However, a negative LUS, or one with ≤1 cm of sonographic consolidation, cannot exclude the diagnosis.36 In these cases, a CR may be appropriate to detect consolidations inaccessible to ultrasound. Alternatively, Jones et al 21 propose that watchful waiting may reduce unnecessary antibiotic use in patients with subcentimetre sonographic consolidations. Although we found that some patients in this group improve without antibiotics, the risk of undertreatment must be considered.
This study has several limitations. The sample size was relatively small, significantly limiting subgroup analysis. Selection bias may have occurred through convenience sampling and the exclusion of patients without consent or with incomplete LUS. Additionally, our inclusion criteria required that a purpose of the CR was to confirm or exclude pneumonia. However, we did not require this as the primary CR indication or the clinician’s most likely differential diagnosis. We included children with chronic cardiorespiratory conditions to generate results applicable to a realistic ED population. However, without baseline LUS images for these patients, we were unable to distinguish acute from chronic changes. These patients may also have been more likely to receive antibiotics, reducing the utility of antibiotic prescription as a surrogate indicator of infection. Additionally, clinicians reported CRs as per their usual practice, which may have led to inconsistent interpretations, particularly regarding interstitial infiltrates. Patients also received single anteroposterior CRs, in accordance with local hospital practice. This may miss findings only identifiable on lateral or posteroanterior views, and increase the rate of false positive LUS.39 Clinicians were unblinded in three cases, potentially biasing follow-up findings for these patients; however, the appropriate diagnosis had already been made in all three. Finally, follow-up relied on parental accuracy and recall, and chart review was complicated by clinicians using different terminology for what may be the same condition, such as ‘pneumonia’ and ‘LRTI’.
Point-of-care LUS appears promising for the diagnosis of pneumonia in the hands of sonologists with focused training in the paediatric ED. LUS may be an appropriate first-line imaging modality in patients with possible pneumonia. A positive finding of >1 cm of lung consolidation may rule-in pneumonia without requiring CR confirmation. Subsequent studies with a greater sample size could confirm these findings, better clarify whether the size of consolidation has clinical significance and evaluate the role of LUS in decision-making regarding antibiotics.
We thank the Emergency Department staff at The Royal Children’s Hospital, Melbourne, for their assistance with recruiting patients for this study, and Dr Ed Oakley for his research expertise and advice. We also acknowledge Tania Griffiths and the Medical Imaging Department for their support and Dr Susan Donath for her statistical guidance.
Contributors CL: conceptualised and designed the study, enrolled patients, acquired, analysed and interpreted data, drafted the initial manuscript. PK: designed the study, enrolled patients, acquired, analysed and interpreted data. CO and EL: designed the study, analysed and interpreted data. MT: analysed and interpreted data. AOB: conceptualised and designed the study, provided teaching and technical support, analysed and interpreted data, supervised the overall study conduct and analysis. All authors critically revised and approved the final manuscript as submitted.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent Informed parental consent. Children over 12 years of age had the option of providing additional informed consent.
Ethics approval This study was conducted with the approval of the Human Resources and Ethics Committee, The Royal Children’s Hospital, Melbourne.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.