Background Non-invasive respiratory support using bubble continuous positive airway pressure (bCPAP) is useful in treating babies with respiratory distress syndrome. Despite its proven clinical and cost-effectiveness, implementation is hampered by the inappropriate administration of bCPAP in low-resource settings. A clinical algorithm—‘TRY’ (based on Tone: good; Respiratory distress; Yes, heart rate above 100 beats/min)—has been developed to correctly identify which newborns would benefit most from bCPAP in a teaching hospital in Malawi.
Objective To evaluate the reliability, sensitivity and specificity of TRY when employed by nurses in a Malawian district hospital.
Methods Nursing staff in a Malawian district hospital baby unit were asked, over a 2-month period, to complete TRY assessments for every newly admitted baby with the following inclusion criteria: clinical evidence of respiratory distress and/or birth weight less than 1.3 kg. A visiting paediatrician, blinded to nurses’ assessments, concurrently assessed each baby, providing both a TRY assessment and a clinical decision regarding the need for CPAP administration. Inter-rater reliability was calculated comparing nursing and paediatrician TRY assessment outcomes. Sensitivity and specificity were estimated comparing nurse TRY assessments against the paediatrician’s clinical decision.
Results Two hundred and eighty-seven infants were admitted during the study period; 145 (51%) of these met the inclusion criteria, and of these 57 (39%) received joint assessments. The inter-rater reliability was high (kappa 0.822). Sensitivity and specificity were 92% and 96%, respectively.
Conclusions District hospital nurses, using the TRY-CPAP algorithm, reliably identified babies that might benefit from bCPAP and thus improved its effective implementation.
- continuous positive airways pressure
- developing countries
- sensitivity and specificity
Statistics from Altmetric.com
- continuous positive airways pressure
- developing countries
- sensitivity and specificity
What is already known on this topic?
The TRY-CPAP algorithm has previously been shown to be reliable, sensitive and specific when used by nurses in a teaching hospital, using a neonatologist’s decision as the standard.
This was a single-centre study conducted in Malawi’s teaching hospital, so does not represent a typical Malawian health facility.
Other algorithms, previously evaluated elsewhere, only consider respiratory effort and do not tackle the problem of bubble CPAP (bCPAP) being applied inappropriately to newborns with hypoxic ischaemic encephalopathy.
What this study adds?
TRY was developed to support the use of bCPAP in district hospitals and smaller facilities, where there may not be any qualified doctors.
This study assessed TRY in a district hospital where there are few qualified doctors and no paediatricians. Results showed the algorithm to have good reliability, sensitivity and specificity.
This study therefore confirms the value of TRY in helping nurses to correctly implement CPAP in the absence of specialist support.
Globally 2.7 million newborns die each year in the first 28 days of life, comprising 45% of under-5 mortality.1 More than a third of these deaths (36%, representing 1 million babies) are attributable to complications of prematurity,2 and one of the most important causes is respiratory distress syndrome (RDS), resulting from surfactant deficiency.3 In well-resourced settings RDS is managed with ventilatory support—either in the non-invasive form of continuous positive airway pressure (CPAP), or as invasive mechanical ventilation with surfactant replacement therapy. These are costly interventions requiring a high level of newborn clinical skill to implement.
In response to this, bubble CPAP (bCPAP) has been developed as an affordable, practical, non-invasive mode of delivering CPAP. bCPAP provides respiratory support through a closed system via nasal prongs to a spontaneously breathing infant. By delivering blended oxygen and air, at variable pressures and flow rates, it can deliver adjustable levels of oxygen (fraction of inspired oxygen typically starting at 70%),4 keeping the lungs expanded and reducing the work of breathing. The Pumani CPAP system (figure 1) (Pumani meaning breathe in Chichewa) was developed for poor-resourced settings by clinicians and bioengineers at the University of Malawi College of Medicine and Rice University in Texas.5 It costs much less than any currently available commercial CPAP machine, uses a similar patient interface, and delivers airflows and pressures similar to bCPAP devices at Texas Children’s Hospital6 (figure 1).
Two non-randomised studies in South Africa showed bCPAP to be associated with improved survival.7 8 In Malawi, studies found Pumani CPAP to be efficacious9 and cost-effective.5 However. bCPAP is challenging to implement correctly; healthcare workers (HCWs) are inexperienced and untrained in when and how to use it.10 Previously, the standard of care for initiating CPAP was simply to apply it when the HCW felt that the baby had signs of significant respiratory distress. However, during the roll-out of bCPAP to over 2000 patients in 28 facilities in Malawi, it emerged that bCPAP was being given to floppy babies with severe birth asphyxia—hypoxic ischaemic encephalopathy (HIE), who may have respiratory distress caused by neurological damage rather than prematurity and surfactant deficiency. In such settings these babies are unlikely to benefit from bCPAP therapy, having a 75% combined probability of severe handicap or death11 and often die in the first few days of life. A robust implementation strategy is therefore important to minimise opportunity costs, maximise resource allocation cost-effectiveness and avoid the misconception that deaths could be attributed to bCPAP.
Other clinical-decision support tools for guiding bCPAP implementation in low-resource settings are available, namely the Silverman Anderson Respiratory Severity Score (RSS)12 and the Downes RDS score.13 These are both five-part scoring systems that only consider one factor (respiratory effort). Short clinical scoring systems and algorithms have been shown to be effective in resource-poor settings, for example the highly successful three-part Blantyre Coma Score (a simplified version of the paediatric Glasgow Coma Score).14 Furthermore, neither the RSS nor the Downes RDS algorithms address the concern around bCPAP being applied inappropriately to newborns with HIE. The TRY-CPAP algorithm (T: is Tone good; R: is Respiratory distress present; and Y: yes, heart rate is above 100 beats/min) (figure 2) is designed to address these deficits. It has three main steps and includes a measure of potential HIE.11 It aims to assist nursing staff in deciding on the appropriate application of bCPAP, rationalising the use of a limited number of bCPAP machines towards infants most likely to benefit, such as premature babies with RDS, and away from babies with HIE.
The word TRY is a useful aide-memoire to recall the main elements of the algorithm if the chart is not available; however, the steps in the flow chart (figure 2) do not follow the order T, R, Y and weight cut-offs are also included. The algorithm is for use on any baby with signs of respiratory distress and/or a birth weight of less than 1.3 kg. This is an estimate for gestation of 30 weeks, and early bCPAP is recommended for gestations <30 weeks because at this gestation the lungs are sufficiently immature to need CPAP regardless of the signs of respiratory distress.11
The first step of the algorithm (figure 2) is weight over 1 kg and is designed to select patients likely to survive. Survival at <1 kg in the absence of surfactant and intensive care is estimated to be low at around 10%15 16; hence, babies weighing less than 1 kg are given supportive care including intranasal oxygen (IN-O2). The next step selects babies that do not need active resuscitation, that is, those with a heart rate (HR) >100 beats/min. If the HR is <100 beats/min, the baby may need resuscitation and bCPAP would not be appropriate until successfully resuscitated. If the baby has good tone, the TRY algorithm then helps the user decide between early bCPAP for babies <1.3 kg and delayed bCPAP for larger infants. For larger babies, if respiratory rate (RR) is >60 and/or oxygen saturations (O2 sats) <90%, IN-O2 is commenced first. If O2 sats are persistently <90% after 2 hours, they are initiated on bCPAP.
This algorithm has shown promising results. In a prospective validation study conducted in Queen Elizabeth Central Hospital (QECH), Blantyre, Malawi (a tertiary referral centre), TRY-CPAP was shown to have a high inter-rater reliability (κ=0.9, 95% CI (0.86 to 0.95)), when comparing TRY assessments between doctor and nurses/nursing students.11 However, this study did not represent the more typical Malawian healthcare setting of a district hospital, where newborn care is delivered by non-physicians and predominantly by nursing staff.
To evaluate the reliability, sensitivity and specificity of the TRY-CPAP algorithm in a nurse-led district hospital environment in Malawi.
The study was conducted in a 40-bed special care baby unit in Zomba District Hospital, Malawi.
The inclusion criteria include all newly admitted babies with respiratory distress and babies with an admission weight of less than 1.3 kg with/without respiratory distress. Weight was measured using weighing scales. Respiratory distress was defined as any of the following signs: nasal flare, tracheal tug, chest indrawings, recessions, head bobbing or grunting.
The exclusion criteria included any baby with a suspected congenital abnormality requiring surgical intervention, such as a congenital abnormality of the respiratory system, and any baby weighing more than 1.3 kg without respiratory distress.
Demographic, admission and outcome information was collected and recorded prospectively in the notes of all medical admissions (eligible and non-eligible for TRY-CPAP). Any newly admitted babies meeting the inclusion criteria would undergo a triple assessment on admission or during the ward round immediately following admission, consisting of (1) TRY-CPAP assessment by a nurse (TRY-HCW), (2) TRY-CPAP assessment by the paediatrician blinded to the nurse’s assessment (TRY-Paed), and (3) clinical assessment by the same paediatrician without using the TRY-CPAP algorithm according to her overall subjective impression (figure 2 depicts the TRY-CPAP algorithm where RR and O2 sats are key components). The TRY-HCW assessment was recorded on a pro forma at the back of the patient’s notes. TRY-Paed was done immediately following the nurse’s assessment and documented in the patient’s medical record. Then after a full assessment of the baby, the paediatrician would document her final clinical decision regarding CPAP.
Diagnoses were made according to the paediatrician’s previous neonatal experience, local guidelines (Care of the Infant Newborn17) and diagnostics available (full blood count and Gram stain but no blood culture, C reactive protein or X-ray). Please see appendix for summary of diagnostic definitions. Ballard scores were conducted as often as possible on any baby weighing less than 2500 g or babies who appeared clinically premature irrespective of weight, but was done after the TRY triple assessment so as not to affect the initial decision on whether to instigate CPAP.
New nurses and students had not been trained to use TRY-CPAP, and so weekly interactive teaching sessions were conducted by the visiting paediatrician prior to and during the study period. These sessions included practising use of TRY in four theoretical case vignettes and real-time supervision on one real baby by the paediatrician. The TRY algorithm (figure 2) was attached to the bCPAP machines and copies were printed for the clinical notes.
Data were collected from Monday to Friday between the hours of 09:00 and 17:00, from 27 April 2015 to 21 June 2015.
Convenience sampling was used to achieve as many TRY assessments as possible within the available time period. Patients admitted overnight, on the weekend or who died before the next weekday ward round were not included.
Verbal consent was obtained from mothers to start CPAP where it was necessary but not for carrying out the TRY assessment as this did not affect the final clinical decision whether to instigate CPAP.
The number of eligible infants was compared with the total medical admissions (eligible and non-eligible) using univariate analyses. A score of 1–4 was obtained for each TRY-CPAP assessment according to its outcome (room air=1, IN-O2=2, CPAP=3, early CPAP=4). The kappa statistic was calculated to measure inter-rater reliability between the TRY-CPAP scores of the HCWs and the paediatrician. To facilitate sensitivity and specificity calculations, using the paediatrician’s final decision as the gold standard, TRY-CPAP scores were made dichotomous, and recorded as ‘CPAP’ or ‘no CPAP’ with a score of 3 or 4 indicating the ‘CPAP’ group and 1 or 2 the ‘no CPAP’ group.
There were 301 medical, neonatal admissions during the study period. Fourteen patient records were missing, leaving 287 patient records for analysis; of these 145 (51%) patients were eligible for the TRY algorithm: 60 (41%) eligible babies had a TRY pro forma completed by an HCW, of whom 57 (39%) also had a pro forma completed and clinical assessment made by the paediatrician (figure 3). Cadres of HCWs participating included matron, registered nurse midwife, nurse-midwife technician, student nurses and student clinical officers.
Table 1 shows the characteristics of the whole sample (287 patients) and the subsample of 57 patients who had triple assessments using the TRY algorithm. In accordance with the eligibility criteria, babies who had the TRY algorithm were of lower weight and earlier estimated gestational age, had lower O2 sats and more prematurity with RDS. The TRY subgroup had slightly lower admission temperatures (a difference in median of 0.6°C; P=0.024) in accordance with lower weights and earlier gestations. Otherwise, there were no significant differences. Of these 57, mortality was 42% (24/57), and the primary diagnoses (presumed causes of death) were prematurity and RDS (13/24=54%), severe birth asphyxia (8/24=33%), prematurity (2/24=8%) and meconium aspiration (1/24=4%). No significant complications of CPAP, such as pneumothorax, were suspected clinically when the visiting paediatrician was present. Chest X-ray was not available.
Of the 57 patients who had a TRY algorithm completed by both an HCW and a paediatrician, 51 received the same outcome score (table 2). Of the six who differed, the HCW overestimated the need for respiratory support for five and underestimated it in one. A kappa statistic of 0.82 (P<0.001, 95% CI (0.69 to 0.95)) was calculated as a measure of inter-rater reliability between the four possible outcomes of TRY in HCWs and the paediatrician.
With the paediatrician’s final decision as the reference standard and using dichotomous outcomes of ‘CPAP’ or ‘no CPAP’, the sensitivity of TRY was 92.6% (95% CI 76.6% to 97.9%) and the specificity was 96.7% (CI 76% to 97%). The area under the receiver operator curve or c-statistic was 0.946. The length of bCPAP treatment in the study ranged from 1 to 9 days (mean 3.8, median 3.0 days).
This is the only study in a resource-poor, district-level hospital, where newborn care is delivered predominantly by nursing staff, to assess the inter-rater reliability and validity of a clinical decision tool to aid the appropriate administration of bCPAP. We showed high inter-rater reliability, sensitivity and specificity of the TRY-bCPAP algorithm, supporting the results of the previous study.11 Key differences here are the lower resource district-level setting, only nursing staff of different cadres are included and babies of all weights are included to truly assess the whole algorithm.
A high kappa statistic (>0.8) suggests that the TRY algorithm gives reproducible results with an ‘outstanding measure of agreement’.18 A slightly higher kappa value in the previous study in QECH, Blantyre of 0.9 for nurses and 0.97 for paediatricians in training,11 is expected as nursing expertise in a district-level facility may not be equivalent to that in a tertiary centre. Nurses with less training and supervision may differ in their assessment, especially of tone and respiratory distress, and here a wide range of HCW cadres filled in the algorithm. A kappa of 0.82 implies that TRY can yield reproducible results even for different cadres of HCWs. Total numbers were too small to disaggregate by cadre.
In some cases, parts of the algorithm were not adhered to, for example HR of <100 beats/min was disregarded by both HCW and paediatrician in an older baby from the community due to extreme work of breathing, and both used clinical judgement to continue down the algorithm. Hence the ‘HR above 100 beats/min’ step of the algorithm is primarily to identify newborns that might need resuscitation straight after delivery, rather than older babies. The paediatrician’s final decision for this patient was ‘early CPAP’ despite a weight of 4.0 kg (well above the 1.3 kg threshold in TRY). These ‘over-rulings’ illustrate how the algorithm may be more useful for newborns rather than those admitted from the community.
In this study, an HCW allocated bCPAP to a baby with severe birth asphyxia in only one case, evidencing its success in directing bCPAP away from floppy babies with HIE. It is possible the HCW did not appreciate how floppy the baby was and instead focused on the low O2 sats of 80% and signs of respiratory distress. There may have been more than one diagnosis in this case, such as mild birth asphyxia and meconium aspiration. This baby was kept comfortable on oxygen and died after 3 days due to hypoxic brain injury. Giving this baby bCPAP may have supported the breathing temporarily but would not have improved his brain injury or survival.
The sensitivity of TRY-CPAP of 92% was higher than in QECH, Blantyre (83%), while its specificity in Zomba (97%) was slightly lower than in Blantyre (100%).11 A higher sensitivity (ie, less false-negatives) in a district hospital setting is encouraging and suggests the algorithm is safe, ‘missing’ less babies in lower resourced facilities. Indeed, if the algorithm is slightly less specific, that is, more likely to allocate CPAP unnecessarily, this is preferable to missing babies who need CPAP. A greater sensitivity in this study may also be related to study design and reflect a learning effect in HCWs over time, as they observed the paediatrician using the algorithm. The paediatrician’s TRY assessment was over-ruled by clinical judgement in 8 of 57 cases, demonstrating that no algorithm can completely compensate for clinical knowledge and experience.
Comparisons of mortality in the TRY group versus the group who did not receive TRY are not reported because these comparisons are significantly confounded by sicker babies being prioritised for TRY due to low resources (as shown in table 1), and the numbers are not high enough to stratify for severity of illness. Also the final decision to use CPAP or not was made clinically by the paediatrician independently of TRY. A future bigger trial comparing TRY-CPAP with no TRY-CPAP, used only by HCWs without a visiting paediatrician, significantly powered to detect differences in mortality is recommended.
If the babies assessed in this study were representative of newborn admissions to other similar-sized and resourced hospital, we can estimate the need for bCPAP in similar facilities. Twenty-four (42%) of the 57 TRY algorithms completed by the paediatrician deemed babies appropriate for bCPAP. If TRY is completed in all babies with respiratory distress (51% of admissions), a facility admitting 150 infants per month will need CPAP for 32 of these infants. If CPAP is on average continued for 3.8 days (as in this study), and given it is unlikely CPAP admissions will be spread evenly throughout the month, at least five machines may be needed per facility.
Other algorithms have been evaluated for their effect on mortality in low-income and middle-income countries. For example the Integrated Management of Neonatal and Childhood Illness (IMNCI) from the WHO19 is a large-scale algorithm-based strategy, the scale up of which was thought to be a key factor in Malawi reaching Millennium Development Goal 4.20 A recent Cochrane review,21 however, presented only moderate evidence that IMNCI ‘may reduce infant mortality’ and the strategy was difficult to evaluate because it employs multiple interventions across facility, primary care and community settings. The results of this TRY-CPAP evaluation (and the previous QECH study) add to the evidence that clinical algorithms are useful in facility settings. This is important for Malawi because 89% of women now give birth in a healthcare facility.22
The main limitation of the study was that it was impossible for the paediatrician to be blinded to her own TRY assessment. Although she was blinded to the nurses’ TRY outcome, completion of the algorithm herself in the second part of the triple assessment may have biased her final decision whether to give CPAP. Unfortunately, there were no local paediatricians present to overcome this limitation. The convenience sampling method was heavily influenced by time pressures and staff availability (see figure 3, box: ‘Reasons for TRY not done); hence, sicker babies were prioritised for TRY. The paediatrician was present throughout the study; hence, a follow-up study would be needed to assess how the algorithm is used in the absence of a clinician and could in addition examine intrarater reliability and compare different cadres of HCWs to guide implementation.
The TRY algorithm itself has limitations. For example it relies on weight as an estimate for gestation. It assumes that all babies with HIE are floppy, whereas they may be hypertonic. Some premature infants without HIE and weighing more than 1.3 kg may be hypotonic due to causes such as sepsis, hypoglycaemia or prematurity itself. However, in this setting there are many more markedly hypotonic newborns with severe HIE who do not survive. Respiratory support in these cases would be futile, and when CPAP equipment and trained staff are limited, the support given to such an infant may be at the expense of another infant who would benefit from CPAP support. One solution would be to adjust this step in the algorithm to say ‘marked hypotonia due to asphyxia’ rather than simply ‘hypotonia’. A footnote could also be added to the algorithm to state that ‘an experienced clinician may decide to use CPAP in circumstances beyond the scope of the TRY algorithm’.
Low-cost CPAP has been shown to be cost-effective and efficacious in improving outcomes for neonates in Malawi. Our study has shown the TRY-CPAP algorithm can help HCWs decide who would benefit from CPAP in a district hospital setting. However new technologies should only be introduced in a unit after careful assessment of the facility, equipment, workload and staffing numbers. A simple checklist of minimum requirements in terms of staff, power supply, space and other equipment should be drawn up, so that CPAP and TRY are used appropriately and in the right setting. CPAP should only be added when there is good evidence that basic neonatal nursing needs are already being provided and CPAP will not overstretch the unit’s capabilities to deliver good care. Maynard et al 23 have provided a list of technologies that are essential to newborn care and advise how to deliver them.
This study demonstrates that the TRY-CPAP algorithm has a good inter-rater reliability, sensitivity and specificity between HCWs and paediatricians. The TRY-CPAP algorithm was helpful in guiding HCWs in the safe and appropriate application of low-cost bubble CPAP in a district hospital setting where usually physicians are absent and care is nurse-led. This evaluation supports the use of clinical algorithms for improving the quality of newborn care in low-resource settings and suggests that more staff are needed to support implementation of new technologies.
We would like to thank the nurses at Zomba Hospital neonatal unit for their participation and support, particularly Indira Chikamoni, Senior Nurse Midwife Technician. We would also like to thank the CPAP Team at the Malawi College of Medicine/Rice University, Texas; the Paediatric Department of College of Medicine, Malawi; the paediatric department at Zomba Hospital; and the director of Zomba Hospital, Dr Mathias Joshua.
Contributors CC helped conceptualise and design the study, and carried out all field work, data collection, statistical analysis and write-up. TC was UK supervisor based at UCL who assisted in statistical analysis, critical review and editing of manuscript. MH provided further advice, critical review and editing of manuscript. EM supervised all aspects of the project in Malawi, overseeing fieldwork, data collection, data analysis and write-up, including initial conceptualisation of the study design.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent This project was an audit within a programme to roll out CPAP, which is approved by NHSRC. Guardians and staff were fully informed and gave verbal permission.
Ethics approval Ethical approval was obtained from both the Malawi College of Medicine and University College of London ethics committees (UK).
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.