Study design and setting

The primary objectives of this prospective, observational, facility-based research are: (1) to assess agreement between repeat observations by the investigational device and the reference device for each relevant measurement parameter of interest based on a priori-determined accuracy threshold among neonates; (2) to compare clinical event detection performance between the investigational device and the reference device and (3) to determine whether the investigational device is feasible, usable and acceptable to hospital administrators, healthcare providers and caregivers of neonates. Secondary objectives include: (1) assessing diagnostic performance for each relevant measurement parameter of interest based on sensitivity, specificity, positive predictive value and negative predictive value compared with the reference device; (2) determining the downtime performance of the investigational device; (3) determining the alarm rate (events/hour) and the number of true/false alarms of the investigational device compared with the reference device; (4) determining the delay time between the investigational device and the reference device in true events and (5) determining the number of adverse device effects (ADEs) and serious adverse events (SAEs) during the use of the investigational device.

Beginning in June 2019 and anticipated to last approximately 18 months in Nairobi, Kenya, this research consists of two phases: (1) a verification and accuracy evaluation phase conducted at Aga Khan University-Nairobi (AKU-N), a private, not-for-profit university teaching hospital with a neonatal intensive care and high dependency units and (2) a clinical feasibility evaluation phase conducted at Pumwani Maternity Hospital (PMH), the largest referral maternity hospital in sub-Saharan Africa with no neonatal intensive care or high dependency units.

Study participants

Up to 500 neonates, corrected age of <28 days admitted for routine observation and care at AKU-N and PMH are recruited by trained study staff during routine intake and screening procedures. To avoid potential selection bias, neonates are screened for enrolment in a sequential manner, as much as possible. Trained study staff assess the neonate for all inclusion and exclusion criteria (table 1). Final eligibility determination is dependent on the results of the medical history, clinical examination, appropriate understanding of the study by the caregiver and completion of the written informed consent process. A neonate may be enrolled to the study more than once as long as they meet the eligibility criteria and the caregiver(s) is willing to have the neonate participate.

For the feasibility, usability and acceptability assessment, hospital administrators and study healthcare providers are enrolled if they are 18 years or older, involved in or aware of the ETNA study, and have provided written informed consent. Caregivers may be enrolled if they are 18 years or older, have a neonate enrolled in the study and are willing to participate in an in-depth interview as well as direct observation while their neonate is on or attached to the investigational device(s).

Investigational devices

Developed by Israeli-based EarlySense since 2009, the Insight system, released in 2016, is a contact-free monitoring system composed of a small piezoelectric sensor pad that can be placed under the patient’s mattress, and is designed to measure and record a patient’s heart rate (HR), respiratory rate (RR), motion and sleep status.6 Information from the sensor pad, in combination with Early Sense’s artificial intelligence analytics, is transmitted to a monitor to provide alert indications and vital sign trends to healthcare providers so that they can monitor changes in a patient’s condition. Currently in use in hospitals, rehabilitative centres, and nursing homes to measure vital signs in adults and children above 10 kg, the device is modified for use in neonates as part of this study. The adult device received regulatory approval from the US Federal Drug Administration and has a Conformité Européene mark for continuous and contactless measurement of HR, RR and motion. No adverse events (AEs) related to the system have been reported during 10 years of monitoring.

Developed in 2019, the advanced neonatal epidermal (ANNE) system from Sibel, a technology company spun out from the Center of Bio-Integrated Electronics at Northwestern University in the USA, is a system of two time-linked soft and flexible sensors designed to measure and monitor vital signs including HR, RR, oxygen saturation (SpO₂) and skin temperature in neonates.7 The chest sensor couples to the skin via a hypoallergenic, biocompatible hydrogel adhesive optimised for reduced peel force on removal, and the limb unit couples via a latex-free soft fabric wrap adaptable to a range of foot sizes and anatomies. Information from the sensors is wirelessly transmitted to a monitor or mobile device via encrypted Bluetooth for real-time streaming from a customised mobile software application as well as onboard memory storage on the sensors themselves. The device has been validated in more than 50 neonates in a neonatal care unit without AEs.

Reference devices

We are employing the Masimo Rad-97 and the Spengler Tempo Easy Bleu devices as our reference devices for this study. The Masimo Rad-97 provides continuous physiological monitoring of HR, RR, SpO₂ and capnography. The Spengler Tempo Easy Bleu non-contact infrared thermometer predicts core body temperature from the temporal artery temperature.

Study procedures

Following completion of screening for eligibility, a study comprehension checklist and written informed consent, study staff perform procedures (online supplementary appendix 1: Schedule of study procedures and evaluations) according to the most recently approved version of the protocol (current V.1.1, 18 June 2019). Enrolled neonates are assigned a participant identification number; information is collected on sociodemographic characteristics, current clinical status, medical history, medications; and a physical examination is performed.

Prior to initiating the accuracy evaluation of each investigational device, verification of the reference devices, Masimo Rad-97 and Tempo Easy Bleu, is undertaken at AKU-N to ensure they are robustly functional and to confirm their within subject repeatability and accuracy compared with standard clinical measurements (eg, manual, bedside electrocardiography) for the relevant parameters of interest. Neonates enrolled during reference device verification continue to receive local standard of care while being observed intermittently for vital signs collection for a minimum of 1 hour using the Masimo Rad-97 and intermittent measurements with the Tempo Easy Bleu. Observations may include video recordings of the neonate and the Masimo Rad-97 reference device monitor for later review to facilitate manual count observations. The reference device measurements will be compared with manual measurements, clinical monitor observations and video-assisted observations. Reliability information gathered from the reference devices is used to determine specific Go/No Go criteria for each parameter and each investigational device. Further evaluation of each investigational device only proceeds should these criteria be met.

Enrolment in the accuracy evaluation of the investigational devices, EarlySense Insight system and Sibel ANNE system, is initiated at AKU-N to formally assess their accuracy compared with the verified reference device using repeated observations. Enrolled neonates continue to receive local standard of care while having vital signs collected from the reference device as well as one or both of the investigational devices. Placement of the investigational and reference devices is done in a manner so as not to interfere with the neonate’s clinical care. Observations are collected for a minimum of 1 hour and potentially for the entire duration of their stay in the hospital. Observations may consist of videotaping and/or taking photos of the neonate during the observation period after obtaining informed consent from the caregiver. Any photos or videos takes are identified by patient identification number only and stored on a secure server until the analyses are completed and destroyed following analyses. During the observation, clinical status and any activities are updated and recorded including type and duration of care activities (eg, feeding, diaper changes, bathing, kangaroo mother care, etc), clinical procedures, interventions, therapies, laboratory tests, medications, environmental features and exposures during the hospitalisation. The device placement, output and signal quality are also monitored. In addition, the neonates are assessed for any safety issues. Agreement between the investigational and reference devices is determined using a priori-defined accuracy thresholds. Thresholds are determined largely based on repeated within and between subject observations during verification of the reference devices. This is complemented by previously published international standards where available, and clinical expert consensus opinion as needed. Two a priori-determined thresholds are determined: one lower threshold to allow the device developer to optimise the device for retesting, and a second higher threshold to allow the device to move on to the clinical feasibility phase of testing. A maximum of five rounds of testing and retesting are permitted for each investigational device. Each round of testing or retesting consists of using a cohort of 20 neonates. Should the lower threshold not be reached for at least one parameter, no further testing of the investigational device is performed. Thus, information collected during the accuracy evaluation along with the a priori-determined Go/No Go criteria established during verification of the reference devices define which, if any, of the investigational devices moves forward with additional rounds of testing or into the clinical feasibility evaluation phase at PMH.

An investigational device advances to the clinical feasibility evaluation phase once the agreement for the measurement parameters of interest exceed the higher accuracy threshold. Enrolment in the clinical feasibility evaluation phase of the investigational devices occurs at PMH up to 120 enrolled neonates who receive local standard of care while being monitored with the reference device(s)and one or both of the investigational devices. Observations are collected for a minimum of 1 hour and involve measurement of vital signs via the investigational and reference devices and monitoring for any critical event (ie, low or high HR, RR or temperature or oxygen desaturation and apnoea). Agreement between repeated observations from the investigation a land reference devices as well as diagnostic performance in clinical event detection is evaluated. Additional performance metrics such as alarm rates, alarm delays and uptime\downtime are compared between the investigational and reference devices. Participation in the study does not interfere with or unnecessarily delay the clinical care of the neonates.

Throughout all phases of the research, the investigational devices are not used to inform clinical care. During the clinical feasibility evaluation phase, ETNA site study staff and hospital healthcare providers are blinded to the data collected from the investigational devices to prevent interference with clinical care. The study site investigators are responsible for close safety monitoring of all participating neonates, including assessing for and reporting ADEs (eg, erythema or oedema at the investigational or reference device sensor site) and/or SAEs (ie, any ADE resulting in permanent skin damage). Any ADEs or SAEs will be treated until resolution or stabilisation, and may require removal of devices and withdrawal of the neonate from the study if necessary. If withdrawn by the study team, any enrolled neonate who completes at least 1 hour of monitoring will be included in the analysis and results.

Qualitative substudy

After written informed consent is received from the study participants, a mixed-methods evaluation and data collection through audio-recorded semistructured in-depth interviews and direct observations are conducted by trained qualitative study staff to assess the feasibility, usability and acceptability of the investigational devices for monitoring of neonates in an African setting. Questions around technology use, experience with continuous monitoring devices and specific to each investigational and reference device will be asked and their use observed. All hospital administrators and study healthcare providers may be involved in this portion of the study. Caregivers with a neonate enrolled in the study may also be asked if they would like to participate in the qualitative portion of the study.

Sample size

A total of up to 500 neonates are enrolled. For the verification of the reference devices at AKU-N, up to 30 neonates are enrolled. Once this initial testing and data collection of the reference devices are complete, for the accuracy evaluation phase at AKU-N, up to 120 neonates per investigational device are enrolled. Sample size estimates for the verification of the reference devices and the accuracy evaluation phase are based on the CIs desired for the limits of agreement. Sample sizes of 100–200 typically provide tight CIs. A sample of 20 neonates with 10 replications per neonate per device per round of testing provides limits of agreement with 95% CIs±0.24, calculated as 1.96*sqrt(3/ (20*10)), times the SD of the paired differences. The paired differences are from the reference device and manual measurements obtained during verification of the reference device, and from the reference device and investigational device measurements obtained during the accuracy evaluation phase. For the clinical feasibility evaluation phase at PMH, up to 120 neonates per investigational device are enrolled. The sample sizes for each phase have been selected to maximise the amount of information collected within the confines of the available resources.

For the feasibility, usability and acceptability assessment, the total sample size includes all hospital administrators and study healthcare providers willing to participate and provide consent as well as up to 30 caregivers willing to participate and provide consent study at each site.

Data collection and quality assurance

Quantitative study data are collected by clinical study staff using designated source documents as well as electronic or paper-based case report forms. Data are stored and managed by a database developed via Research Electronic Data Capture, a secure web application. Continuous physiological data and event data are recorded from the investigational and reference devices at least once a second. All electronic data are collected wirelessly or via a wired connection from the investigational and reference devices to a study laptop using custom software applications. Qualitative study data are collected using paper-based forms and audio recordings, which are subsequently transcribed for analysis.

Clinical research data, including data collected from the investigational and reference devices, are maintained through a combination of secure electronic data management system and physical files with restricted access to ensure confidentiality. Two distinct study databases are maintained separately: the primary study database and a database with participating neonate’s personally identifiable information. To ensure accuracy and completeness, data are routinely reviewed by the investigators through quality assurance reviews, audits and evaluation of the study safety and progress. Guideline for Good Clinical Practice (GCP)/ISO 14155 compliance is followed to ensure accurate, reliable and consistent data collection.

Data management

Primary data management activities, which include deidentified investigational and reference device data transfer using end-to-end encryption with two-factor authentication, data entry and validation, data cleaning, database quality control and disaster recovery plans are undertaken at the study site and are overseen by the on-site data manager. Data review and analysis, oversight and preparation of final study database is performed by the investigators in collaboration with the study site. Data are maintained and stored securely in databases hosted at the study site throughout the study and for at least 5 years after study closure. All data management activities are in compliance with International Council on Harmonisation (ICH) GCP E6, sponsor organisation and institutional requirements for the protection of children and confidentiality of personal and health information.

Outcomes

We hypothesise that the investigational device is accurate and reliable compared with the reference device for each relevant measurement parameter of interest among neonates and is feasible, usable and acceptable for use in neonates in low-resource settings. The primary endpoint and secondary endpoints are detailed in box 1.

Statistical analyses

Every second of data is automatically graded as optimal, acceptable and unacceptable based on predefined rules for each device and each measurement parameter of interest according to the quality of the data for each measurement parameter of interest. The Masimo Rad-97 provides a signal quality index that is used to determine data quality for HR and SpO₂. A custom algorithm has been produced to determine the capnography signal quality index. Each of the investigational devices also provides a signal quality index. The quality thresholds are determined following verification of the reference devices. All comparisons are performed from observations between two devices (or a single device during verification). At least 10 observations of 60 s of optimal quality data in each neonate, at least 5 min apart, are randomly selected for each measurement parameter of interest from the full recording. For the clinical feasibility evaluation phase, accuracy comparisons use optimal or acceptable data. At least 3 hours of recording to a maximum of 12 hours are used for the performance metrics such as alarm rates, alarm delays and uptime\downtime.

The repeatability of the reference device parameter estimates initially is assessed with the intraclass correlation coefficient (ICC). Additional training or standardisation of procedures is performed to ensure at least good repeatability (ICC >0.7). This is followed by measuring agreement between the repeated reference observations and between the manual, clinical monitor and video-assisted methods and the reference observations using the methods described by Bland and Altman for replicated observations.8 The agreement is reported as a mean bias with 95% CIs and 95% limits of agreement. Graphical representation of the data is assessed with agreement plots, Clarke error grids and Polar plots to identify extreme outliers and significant data trends.

In the accuracy evaluation, the root mean square difference and ICC are calculated for each measurement parameter of interest to compare the multiple repeated observations between the investigational and reference devices. The agreement between each investigational device and reference device(s) is then calculated using the methods described by Bland and Altman for replicated observations. The agreement is reported as a mean bias with 95% CIs and 95% limits of agreement. Graphical representation of the data is be assessed with agreement plots, Clarke error grids and Polar plots to identify extreme outliers, impact on clinical decisions, and significant data trends. An a priori-defined accuracy margin for agreement is used as a threshold value to allow for decisions regarding proceeding to additional testing.

In the clinical feasibility evaluation phase, agreement between each investigational device and reference device(s) is assessed as in the accuracy evaluation phase. Event detection rates, alarm rates, alarm delays and uptime/downtime are summarised with means, medians SD and IQRs as appropriate. Summaries of sensitivity, specificity, positive predictive values and negative predictive values comparing each measurement parameter of interest in the investigational device(s) to the reference device(s) are produced. Comparisons of binary events are assessed using Cohen’s weighted Kappa and McNemar’s test. The non-inferiority of alarm rates, alarm delays and uptime/downtime are evaluated based on prespecified thresholds.

Qualitative data are collected through in-depth interviews and/or semistructured questionnaires and analysed to assess feasibility, usability and acceptability of the investigational devices among hospital administrators and healthcare providers, and acceptability among caregivers of enrolled neonates. Questions that explore familiarity, knowledge, perceptions, attitudes and behaviours regarding the devices are included. The qualitative data are in narrative format and the results are descriptive. The questionnaires are coded and analysed using a codebook with identified themes, including feasibility of using each investigational device, barriers and facilitators to use, and perceived value. Qualitative data analysis software is used to organise, code and analyse the qualitative data in an iterative process. The study team starts by identifying an initial set of codes and themes based on the categories from the interview guides. During the coding process, attention is paid to identifying emergent issues and themes that are added to the codebook and included in the analysis. Responses from the interviews are coded and discrepancies discussed and resolved for the final analysis and theme identification.

Ethics and dissemination

Possible risks

Caregivers may feel compelled to enrol in the study in order to receive care for their neonate within a research setting, which may be perceived as of a higher quality than the standard of care. In order to minimise the risk of coercion, during the informed consent process, study staff emphasise that the neonate will receive the required medical care whether enrolled in the study or not. Other potential risks to study participation may include those associated with the placement and attachment of the investigational and reference devices, and delayed medical management. Study staff are trained in the appropriate placement of investigational and reference devices’ sensors to minimise discomfort to the neonates as well as to avoid interference with any assessment, treatment or intervention necessary for clinical care. There is a potential risk of skin irritation with the ANNE sensor system and neonates will be closely monitored and treated for any AEs. Study staff are also trained in integrating study procedures with clinical care and to always prioritise clinical care above study procedures. Extreme care is taken to ensure that no necessary treatment is delayed to accommodate study procedures.

Dissemination

We plan to disseminate study results in peer-reviewed journals and international conferences, targeting those involved in the clinical care of neonates in low-resource settings as well as those who develop and advise on policies and guidelines in those settings.

Efforts towards rigorous protocol

Dedicated study staff trained in GCP, operation, use and maintenance of the investigational and reference devices, and study-specific procedures follow neonates enrolled in the trial to assure the protocol and standard operating procedures are followed and data are accurately collected.Standardised study-specific training, supervision and oversight are undertaken to ensure quality, consistency and harmonised trial procedures and implementation. Regular monitoring is provided by the coinvestigators to assess compliance with human subjects and other research regulations and guidelines, adherence to the study protocol and procedures, and quality and accuracy of data collected.

Limitations and bias

Limitations to this study and potential sources of bias include the sampling strategy, the uncertainty inherent in the measurements from the reference devices, the limited standardisation of time of day of recording and the inability to control the conditions and standardise the context. Because there is a large variation in the various ages, weights, sizes, disease states, clinical presentations, interventions received and conditions of the participating neonates, it is not possible to control for all these variables. Likewise, the environment cannot be controlled, does not allow for complete standardisation and may introduce additional sources of bias. These limitations may also be viewed as strengths.