Article Text
PDF
Abstract
Objective To assess the accuracy of sweat conductivity among newborns and very young infants.
Design Prospective, population-based, diagnostic test accuracy study.
Setting Public Statewide Newborn Screening Programme where the incidence rate of cystic fibrosis (CF) is ≈1:11 000.
Patients Newborns and very young infants with positive two-tiered immunoreactive trypsinogen.
Interventions Sweat conductivity and sweat chloride were performed simultaneously, on the same day and facility by independent technicians, with the cut-off values of 80 mmol/L and 60 mmol/L, respectively.
Main outcome measures Sensitivity, specificity, positive and negative predictive values (PPV and NPV), overall accuracy, positive and negative likelihood ratios (+LR, -LR) and post (sweat conductivity (SC)) test probability were calculated to assess SC performance.
Results 1193 participants were included, 68 with and 1108 without CF, and 17 with intermediate values. The mean (SD) age was 48 (19.2) days, ranging from 15 to 90 days. SC yielded sensitivity of 98.5% (95% CI 95.7 to 100), specificity of 99.9% (95% CI 99.7 to 100), PPV of 98.5% (95% CI 95.7 to 100) and NPV of 99.9% (95% CI 99.7 to 100), overall accuracy of 99.8% (95% CI 99.6 to 100), +LR of 1091.7 (95% CI 153.8 to 7744.9) and -LR of 0.01 (95% CI 0.00 to 0.10). After a positive and negative sweat conductivity result, the patient’s probability of CF increases around 350 times and drops to virtually zero, respectively.
Conclusion Sweat conductivity had excellent accuracy in ruling in or ruling out CF after positive two-tiered immunoreactive trypsinogen among newborns and very young infants.
- cystic fibrosis
- child health services
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Sweat conductivity is considered a screening test for cystic fibrosis (CF); accuracy studies are needed to assess its role as an alternative method to rule in/out CF.
WHAT THIS STUDY ADDS
Sweat conductivity showed an excellent diagnostic accuracy; for instance, positive and negative likelihood ratios were 1091.7 and 0.01, respectively.
In settings where the incidence rate of CF is around 1:11 000, post-test probability increases 350 times the likelihood of CF after a positive sweat conductivity, while negative results virtually rule out CF.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
In settings with a shortage of chloride titration, sweat conductivity can expedite diagnosis and allow early prompt treatment, minimising family and parental distress due to a positive newborn bloodspot screening.
Introduction
Due to the limited availability and affordability of CFTR gene mutations analysis in resource-limited settings—where, like in Brazil, only cases with abnormal chloride values undergo DNA testing, after positive two-tier IRT sweat chloride determination is the strategy implemented by several newborn screening programmes for cystic fibrosis (CF) worldwide.1–3 It offers maximal accuracy but requires a specialised centre and skilled personnel.4–6 In addition, the time-consuming Gibson and Cooke’s method is still used in those settings instead of the extensively recommended Macroduct system.3 5–7
In its turn, sweat conductivity (SC) analysis is cheaper, commonly used in many settings throughout the world,8 including many laboratories in high-income Australasia countries,9 and provides instant results1 3 5 7 like the more expensive chloridometer. Several studies comparing SC with conventional sweat test (ST) demonstrated that sensitivity, specificity, positive and negative predictive values (PPV and NPV) ranged from 83.3% to 100%.1 3 5 7 10 11 However, these studies had an underpowered sample size, although one recruited 3834 subjects, from newborns to adults.5 Currently, institutions such as Cystic Fibrosis Foundation (CFF) and the UK-based Association for Clinical Biochemistry & Laboratory Medicine consider SC a screening test.4 12–14
Therefore, the present study aimed to assess the diagnostic accuracy of SC solely among newborns and infants younger than 3 months with positive IRT/IRT-based NBS. Our hypothesis was that SC has comparable accuracy to the classical coulometric titration of sweat chloride for diagnosing CF in newborns and young infants.
Methods
Study design
We conducted this prospective, population-based, paired diagnostic test accuracy study between April 2013 and October 2020, following the Standards for Reporting Diagnostic Accuracy guidelines (STARD).15
Participants and predefined inclusion and exclusion criteria
We included newborns and infants younger than 3 months, public Statewide Newborn Screening Programme participants, who presented two IRT positive results (ie, >70 mg/L) in our CF referral centre, located in Belo Horizonte, Brazil. To ensure blinding and to avoid interference of other clinical conditions on the result of SC and chloride titration if they were done on different days, participants underwent both tests on the same day and facility, conducted simultaneously, once only, by independent technicians.
Exclusion criteria were skin lesions of any aetiology, clinical instability and insufficient sweat quantity.
Test methods and procedures
Data collection and recording were completed immediately after performing SC and chloride determination.
Collection of sweat samples by Macroduct system and SC analysis (index test)
Sweat collection was performed as described by others.3 5 7 10 11 Briefly, the technician cleaned the patient’s skin before placing the Macroduct coil. As recommended by the manufacturer, the sweat collection time was 30 min.16 Following the manufacturer’s guidelines, after sweat collection, the catheter was separated from the disk and a syringe was connected to one end of the catheter. To measure conductivity, the other end was connected to the Sweat-Chek analyzer.
Collection of sweat samples by Gibson and Cooke’s technique and chloride analysis (reference standard)
Gibson and Cooke’s method was performed according to well-established standards in the contralateral forearm.12 14 After pilocarpine iontophoresis, chloride quantitation was carried out through a chloridometer.12 14
Cut-off points for positive and negative results
For chloride concentration, we adopted the CFF and the reference values recommended by others, that is, results below 30 mmol/L and equal to or greater than 60 mmol/L, were considered negative and positive, respectively, indicating the definitive diagnosis of non-CF and CF.4 12 14 For SC, we followed both the manufacturer’s guidelines16 and the values reported by others.3 7 SC was considered negative when values were equal or lower than 60 mmol/L and positive when they were equal or greater to 80 mmol/L. Sweat chloride values of 30–59 mmol/L and SC values of 60–79 mmol/L were considered intermediate and analysed apart.
Statistics
Analysis
The primary analysis included only participants with positive and negative results for both methods; intermediate results were not accounted for at this stage. We conducted descriptive statistics and constructed a 2×2 contingency table to estimate sensitivity, specificity, PPV, NPV, positive and negative likelihood ratios (+LR and -LR) and their respective 95% CI.
Intermediate values are expected to occur with chloride titration and SC. According to STARD guidelines,15 they can be ignored altogether, be reported but not accounted for or handled as a separate test result category. It is also possible to reclassify all such results as false positives or false negatives, depending on the reference standard result (‘worst-case scenario’) or as true positives and true negatives (‘best-case scenario’). As suggested by STARD guidelines, we conducted secondary analyses to reassess SC performance using both scenarios.15
For the calculation of the post (sweat conductivity) test probability, we considered that in our setting, in the last 5 years, the average prevalence (pretest probability) of CF among newborns and young infants was at around 0.2%.
We also constructed the receiver operator characteristic (ROC) curve to estimate the area under the curve (AUC). Finally, we assessed the agreement between index and reference tests through Kappa coefficient.
Analyses were performed using SPSS software, V.23 (SPSS, USA).
Sample size
Sample size calculation was based on the classical equation, that is, N=4Zα2P(1 − P)/W2, in which Zα equals 1.96; P, a proportion that equals the sensitivity or specificity estimates and W, the width of the 95% CI.17
According to the published literature, the average sensitivity for SC is 98%; accepting a margin of error within 0.06 for the 95% CI, we would need a minimum of 83 patients with CF. Regarding specificity, the literature reports an average specificity of 99%;1 3 5 7 9 10 adopting a width of 0.02 for the 95% CI would require at least 380 individuals without CF. Thus, the final sample size would need at least 463 subjects.
Results
Participants
During the study period, 1 752 587 newborns were screened. A total of 2631 newborns and young infants had two positive IRT tests. According to predefined criteria, we excluded 1 750 099 and considered ineligible 1295 participants; therefore, 1193 were recruited for the study. Among them, 17 subjects (17/1193, 1.4%) presented intermediate results for ST, SC or both tests (figure 1).
Flow diagram displaying the study cohort retrieval process. SC, sweat conductivity; ST, sweat test.
Table 1 compares the general characteristics of 1176 participants with positive and negative ST and SC.
General characteristics of the patients with CF and without CF included in the primary analysis
There were no significant differences between the two groups, except that the CF group had a higher prevalence in rural areas and a lower proportion of preterm birth. (Gestational age<37 weeks).
The median age of the patients was 5 days old at the first IRT and 15 days of age for those in which the first dosage was higher than 70 mg/L, while the mean (SD) and median (IQR) age of the participants were 48 (19.2) and 43 days old (15-90), respectively. In the CF group, the mean and median values were 100.2 (21.8) mmol/L and 98.1 (66–242) mmol/L for sweat chloride and 103.9 (14.3) mmol/L and 106.0 (25-131) mmol/L for SC. In the non-CF group, the mean and median values for sweat chloride and conductivity were 10.9 (4.4) mmol/L and 10.2 (2.4–29.7) mmol/L and 29.2 (6.0) and 29 (11-103), respectively (data are not shown in table 1).
Test results and estimates: diagnostic accuracy of SC
Table 2 shows the results of ST and SC among the 1176 patients with no intermediate values to chloride analysis and conductometry.
Diagnostic accuracy of SC among the 1176 participants without intermediate results (primary analysis)
Sweat conductivity yielded a sensitivity of 98.5% (95% CI 95.7 to 100), a specificity of 99.9% (95% CI 99.7 to 100), a PPV of 98.5% (95% CI 95.7 to 100), a NPV of 99.9% (95% CI 99.7 to 100) and an overall accuracy of 99.8% (95% CI 99.6 to 100). Furthermore, the positive and negative LR were 1091.7 (95% CI 153.8 to 7744.9) and 0.01 (95% CI 0.0 to 0.1), respectively.
Post-test probability for positive and negative LR were 68.6 and 0.002, respectively, meaning that the patient’s probability of CF increases from 0.2% to almost 70% (ie, around 350 times more) and drops from 0.2% to virtually zero, after a positive and negative SC result, respectively.
The ROC curve was plotted with the 1176 patients without intermediate values to assess the overall accuracy of SC (see figure 2). The AUC was 98.8% (95% CI 96.6 to 100; p<0.001), demonstrating that SC potentially has an accuracy close to perfect to distinguish individuals with and without CF. Finally, the kappa correlation coefficient was 0.99 (95% CI 0.97 to 1.0). Please, like in the text inside the Figure replace 96.6%–1.0% to 96.6%–100%
The ROC curve obtained for 1176 subjects. Solid black line: area under the curve for sweat conductivity. Shadowed grey area: 95% CI. AUC, area under the curve; ROC, receiver operating characteristics.
Table 3 summarises the results of 17 participants with intermediate values for either SC, sweat chloride or both. As shown, the overall rate of borderline values was 1.4%, and the number of individuals with intermediate SC results was slightly lower than that for sweat chloride testing, that is, 8%–0.7%, and 12%–1.0% out of 1193, respectively. Only three participants (0.25%) remained with intermediate values to both tests.
Sweat chloride and SC results among the 17 (out of 1193) participants with intermediate results
The secondary analyses included all 1193 studied participants. For the worst-case scenario, where true positives=67, false-positives=8, false-negatives=11 and true negatives=1107, sensitivity, specificity, PPV and NPV and positive and negative LR were 89.3% (95% CI 80.0 to 95.2), 99.0% (95% CI 98.2 to 99.5), 85.9% (95% CI 77.0 to 91.6), 99.2% (95% CI 98.6 to 99.6), 90.8 (95% CI 50.1 to 164.3) and 0.1 (95% CI 0.0 to 0.2), respectively. For the best-case scenario, where true positives=74, false-positives=1, false-negatives=1 and true negatives=1117, sensitivity, specificity, PPV and NPV and positive and negative LR were 98.6% (95% CI 99.5 to 99.9), 99.9% (95% CI 99.5 to 100), 98.6% (95% CI 91.2 to 99.8), 99.9% (95% CI 99.3 to 99.9), 1.103,0 (95% CI 155.4 to 7.825,7), 0.01 (95% CI 0.0 to 0.1), respectively.
Discussion
Chloride titration and SC are two sides of the same coin. An increase in chloride anion reflects a proportionate increase in the total electrolyte concentration, which is the basis of analysis by electrical conductivity.18
To date, this is the first study that demonstrated the excellent accuracy of SC as an alternative diagnostic test for CF among 1193 newborns and very young infants—the target age group of the NBS—compared with the classical ST.
All accuracy estimates obtained in the primary and secondary analyses evidenced the discriminatory diagnostic power of SC, specially the very high positive LR and very low negative LR. Positive and negative LR have unequivocal clinical relevance and are independent of disease prevalence offering a more practical way of interpreting SC accuracy. The higher the positive LR, the better the test result for ruling in the disease; conversely, the lower the negative LR, the better the test result is for ruling out the disease.17 Irrespective of the way that primary and secondary analyses were done, all accuracy indices for SC remained relatively stable. Moreover, post-test probability values mean that the likelihood of CF increases 350 times after a positive SC, while negative results virtually rule out CF.
The ROC curve is a useful summary of the overall accuracy of any test. The very large AUC (98.8%) and the near-perfect agreement between index and reference tests (Kappa=0.99) provide additional evidence for the accuracy of SC. The rate of the intermediate ST (1.0%) and SC (0.7%) results also demonstrated the comparability between them.
As the mean and median age of the included participants were lower than 50 days when they underwent ST and SC, the first clinical appointment was accomplished before completing the second month of life irrespective of their gestational age, birth weight and weight at the testing day. These data are comparable to those described by Leung et al reporting a mean age of 2.58 (SD 0.69) months at initial examination among 231 American CF children diagnosed exclusively through classical ST.19
We searched studies on SC, disregarding investigations carried out with its related Nanoduct, and found only one prospective study including newborns.3 That study assessed 534 subjects and had similar estimates for SC, that is, 100% sensitivity, 96.2% specificity, 83.3% PPV, 100% NPV, 31.7 positive LR, and zero negative LR.3 Other studies in different age groups found good to high estimates for sensitivity (83.3%–100%), specificity (97.3%–100%), PPV (90.9%–100%) and NPV (94.7%–100%).1 3 5 7 10 11 To note, the present study had the narrowest width of the 95% CI for all estimates, demonstrating the highest statistical precision.
As a result of multiple determinants, a glaring gap is evident in the median age of survival of patients with CF living in affluent and non-affluent countries. For instance, the life expectancy of individuals with CF in Canada, the USA and European Union countries are in the mid-40s or above, while only half of that in Brazil and even lower in some countries of the former Soviet Union.20 It is widely recognised that individuals with CF in developed nations are diagnosed earlier than their counterparts in developing ones and are likely to receive sooner standard CF care because of organisational factors, appropriate healthcare infrastructure and human resources.20 Providing the cheaper and simple SC test in rural or remote settings could contribute to speeding up the diagnosis, bridging at least some of these critical gaps.
Then, in settings with a relative shortage of chloridometers, implementing an SC network in close cooperation with CF referral centres labs could reduce their burden with negative sweat chloride testing after NBS and contribute to the earlier diagnosis of CF. For instance, the CF Referral Centre labs must retest sweat samples through a chloridometer from either positive or negative SC results chosen randomly through a rigorous and periodical schedule. As shown, around 80% of our studied newborns and young infants resided in the countryside, that is, between 30 and 470 miles from our CF Centre, the sole facility for the whole State, which surface area is like that of France. A similar scenario seems to apply in similar settings worldwide.
We should pinpoint the limitations of our study. First, even though chloride titration is still considered a gold standard, it would be better if we had used a more robust gold standard, that is, the combination of sweat chloride plus DNA testing. That strategy would undoubtedly avoid the rate of intermediate results and would guide CFTR modulator therapies. Second, we recruited 72 out of 83 patients with CF, that is, 11 fewer than would be needed. However, the present work has enough power to assess the diagnostic accuracy of SC, given that the final sample size (n=1193) is much larger than that previously calculated (n=463). Finally, in terms of size and age group, this unparalleled sample may also have mitigated the ineligibility of the 1295 potential patients who could not simultaneously undergo both tests, one of the inclusion criteria.
In conclusion, SC demonstrated excellent reliability after two-tier IRT NBS. Therefore, it might be considered an alternative diagnostic tool for CF, especially in poor health infrastructure settings, because it contributes to early treatment and minimises family and parental distress due to a positive NBS.
Ethics statements
Patient consent for publication
Ethics approval
The Institutional Review Board of the Federal University of Minas Gerais approved the research protocol (reference CAAE 21958014.1.000.5149).
Acknowledgments
The authors acknowledge Roberto Vagner Puglia Ladeira, the CF referral centre laboratory head, for supporting the study as well as Dr José Dirceu Ribeiro, Dr Fabíola Vilac Adde, Dr José Nélio Januário, Dr Cláudia Ribeiro de Andrade and Dr Clésio Gontijo do Amaral for their helpful inputs on the preliminary version of this manuscript.
Footnotes
Contributors RMB verified the underlying data, contributed to the conception, writing, revision and editing of the manuscript. CGA contributed to the manuscript’s conception, writing, editing and critical revision. JVAJ and FHP contributed to data collection, analysis, validation and interpretation, verified the underlying data and contributed to writing the manuscript. LZ contributed to the manuscript’s conception, writing, revision and editing. PC conceived the study and its design, had primary responsibility for protocol development, contributed to the investigation, methodology, project administration, validation and verified the underlying data; conception, writing, editing and revising of the manuscript. PC is the guarantor. All authors confirm that they had full access to all the data in the study and accept responsibility for submitting it for publication.
Funding This work was supported by the Brazilian Council for Research and Technological Development, grant number 486201/2013-9, to PC.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Highlights from this issue