Article Text

other Versions

Robustness of reported postacute health outcomes in children with SARS-CoV-2 infection: a systematic review
  1. Julian Hirt1,2,
  2. Perrine Janiaud1,
  3. Viktoria Luise Gloy1,
  4. Stefan Schandelmaier1,3,
  5. Tiago V Pereira4,5,
  6. Despina Contopoulos-Ioannidis6,
  7. Steven N Goodman7,8,9,
  8. John Ioannidis7,8,9,10,11,
  9. Klaus Munkholm12,13,
  10. Lars G Hemkens1,9,11
  1. 1Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
  2. 2International Graduate Academy, Institute of Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany
  3. 3Department of Clinical Epidemiology and Biostatistics, McMaster University Faculty of Health Sciences, Hamilton, Ontario, Canada
  4. 4Applied Health Research Centre, Li Ka Shing Knowledge Institute, St Michael's Hospital, Toronto, Ontario, Canada
  5. 5Department of Health Sciences, College of Medicine, University of Leicester, Leicester, UK
  6. 6Division of Infectious Diseases, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA
  7. 7Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA
  8. 8Department of Medicine, Stanford University School of Medicine, Stanford, California, USA
  9. 9Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA
  10. 10Department of Biomedical Data Science, School of Medicine, Stanford University, Stanford, California, USA
  11. 11Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
  12. 12Department of Clinical Research, Centre for Evidence-Based Medicine Odense (CEBMO) and Cochrane Denmark, University of Southern Denmark, Odense, Denmark
  13. 13Open Patient Data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark
  1. Correspondence to Dr Lars G Hemkens, University of Basel, Basel, Switzerland; Lars.Hemkens{at}usb.ch

Abstract

Objective To systematically assess the robustness of reported postacute SARS-CoV-2 infection health outcomes in children.

Methods A search on PubMed and Web of Science was conducted to identify studies published up to 22 January 2022 that reported on postacute SARS-CoV-2 infection health outcomes in children (<18 years) with follow-up of ≥2 months since detection of infection or ≥1 month since recovery from acute illness. We assessed the consideration of confounding bias and causality, as well as the risk of bias.

Results 21 studies including 81 896 children reported up to 97 symptoms with follow-up periods of 2.0–11.5 months. Fifteen studies had no control group. The reported proportion of children with post-COVID syndrome was between 0% and 66.5% in children with SARS-CoV-2 infection (n=16 986) and between 2.0% and 53.3% in children without SARS-CoV-2 infection (n=64 910). Only two studies made a clear causal interpretation of an association between SARS-CoV-2 infection and the main outcome of ‘post-COVID syndrome’ and provided recommendations regarding prevention measures. The robustness of all 21 studies was seriously limited due to an overall critical risk of bias.

Conclusions The robustness of reported postacute SARS-CoV-2 infection health outcomes in children is seriously limited, at least in all the published articles we could identify. None of the studies provided evidence with reasonable certainty on whether SARS-CoV-2 infection has an impact on postacute health outcomes, let alone to what extent. Children and their families urgently need much more reliable and methodologically robust evidence to address their concerns and improve care.

  • COVID-19
  • infectious disease medicine

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information. All data generated or analysed during this study are included in this publication and supplement content.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Children usually have mild or no symptoms of SARS-CoV-2 infection1 2 and are rarely hospitalised with extremely rare fatal events.3 However, symptoms persisting beyond the acute stage have been observed not only in adults4 but also in children.3

Such persistent symptoms or postacute health outcomes are often referred to as long-COVID or post-COVID syndrome, but there is no consensus on how to define it.5 The term long-COVID may encompass both ongoing symptomatic COVID-19 (4–12 weeks after the initial infection) and post-COVID-19 syndrome (≥12 weeks after the initial infection)6 or defined as post-COVID-19 condition occurring 3 months from the onset of COVID-19 with symptoms that last for at least 2 months.7

Evidence syntheses aiming to assess postacute SARS-CoV-2 infection health outcomes in children are difficult, given the highly heterogenic study designs and complex limitations. There are methodological concerns about the validity of reported causal effects of infection on long-term outcomes,3 4 including the absence of a control group, missing outcome data, and detection and misclassification biases.4 Moreover, there is serious heterogeneity in study populations (diverse settings, eligibility criteria and sampling strategies) and lack of standardisation of cases and outcomes3 4 that limits the interpretation of estimates for absolute risks and post-COVID-19 syndrome prevalence. The validity of relative risk estimates critically depends on the risk of confounding bias resulting from systematic differences between the compared groups. However, confounding is often ignored when interpreting epidemiological studies,8 9 and their authors rarely call for cautious interpretation.9

Given the potential impact on child health worldwide, we would expect that studies assessing these risks are conducted with the greatest possible care and the intention to meet the highest available standards in reporting and transparency to deal with confounding, other biases and causal claims.

Here, we systematically assessed the robustness of reported outcomes of studies that aimed to determine the effect of SARS-CoV-2 infection on post-COVID syndrome in children.

Methods

We used a similar search strategy and study selection as a previous related analysis.10 Our study protocol is published10 and no major deviations occurred. We used the Preferred Reporting Items for Systematic reviews and Meta-Analyses 2020.11

Eligibility criteria

Eligible studies included a cohort of children (<18 years) defined by the presence of SARS-CoV-2 infection; reported frequency of health outcomes (ie, any symptoms) for this cohort (accepting subgroup analyses); had a clearly defined follow-up of ≥2 months since detection of SARS-CoV-2 infection, onset of symptoms, COVID-19 diagnosis and/or hospital admission, or a clearly defined follow-up of ≥1 month since recovery from acute illness and/or hospital discharge (follow-up defined as in previous related analysis10; and were published as preprint or peer-reviewed journal article in English.

We considered studies regardless of the severity of the acute infection or the setting (eg, outpatient, hospitalised or intensive care), and regardless of whether the investigation included a comparator group of participants without SARS-CoV-2 infection. We excluded meta-analyses, evidence syntheses, abstract-only publications and conference proceedings.

Information sources and search strategy

We adapted an existing search strategy4 (online supplemental eMethods) and searched PubMed and Web of Science Core Collection since 1 January 2020; the living systematic COVID-19 map provided by the EPPI-Centre (using paediatric keywords)11 and the L·OVE platform (for preprints).12 The last searches were on 22 January (PubMed and Web of Science) and 25 January (L·OVE and EPPI-Centre) 2022. We contacted investigators of registered systematic reviews on long-COVID in children to cross-check eligible studies13–15 (last search in the International Prospective Register of Systematic Reviews (PROSPERO) on 5 November 2021, authors’ request 23 November 2021) and screened citations of further relevant reviews.3 16 17

Study selection

One researcher screened titles and abstracts (JH, PJ or LGH). Two researchers independently screened full texts (two of JH, PJ, SS, TVP or VLG). Disagreements were resolved by discussion or third-party arbitration (LGH).

Data extraction and methodological appraisal

Two researchers (two of JH, PJ, SS, TVP or VG) independently extracted study characteristics, assessed the consideration of confounding bias and causality, and the risk of bias.8 9 18 19 Details on extracted items and algorithms for assessments were prespecified.10 Disagreements were resolved by discussion or third-party arbitration (LGH).

Targeted outcomes

We considered symptoms and quality of life outcomes or other patient-relevant outcomes reflecting how children feel, function (or survive).20 Imaging or laboratory measures were not considered. We determined the most frequent symptoms assessed among all studies.

We recorded the proportions of children presenting the main outcome. The main outcome was defined as stated by the authors or, if not available, the most inclusive one (eg, any symptoms).10 We also assessed if the main outcome was analysed in relation to specific participant characteristics (eg, comorbidities) or infection-related factors (eg, severity acute disease). Full details are provided in online supplementary eMethods.

Consideration of confounding bias and causality, and risk of bias assessment

We assessed the consideration of confounding using a previously developed approach8 9 based on prespecified questions focusing on reporting of confounders and bias in the abstract and discussion, and on what the findings mean and what the limitations are. For the analysis of limitations in the conclusion, we used clearly stated conclusion paragraphs (ie, conclusion subheading or paragraph starting with a phrase such as ‘In summary’ or ‘In conclusion’).We assessed three aspects related to consideration of causality, following a similar but simplified approach as Haber et al.18 We determined whether an association of SARS-CoV-2 infection and the main outcome was interpreted causally and if any recommendations were made based on such causal implications; whether a conceptual causal model describing causal mechanisms (eg, a directed acyclic graph21 22) was used; and whether any explicit causal disclaimer was made.

We assessed the risk of bias for an estimated effect of SARS-CoV-2 infection on the main postacute symptom outcome using “Risk Of Bias In Non-randomised Studies of Interventions” (ROBINS-I),19 23 replacing the notion of ‘intervention’ by ‘infection’ (see protocol for full details).10 The ROBINS-I categories are ‘low’, ‘moderate’, ‘serious’ and ‘critical’ risk of bias.10

Statistical analysis

We used R V.4.1 for all analyses. We report medians and IQRs and calculated proportions with 95% CIs using the ‘metaprop’ function from the ‘meta’ package V.5.1–1.24

Results

Twenty-one studies were eligible (table 1 and online supplemental eFigure1). Six studies had a control group25–30 and 15 were uncontrolled.31–45

Table 1

Characteristics of cohort studies reporting postacute health outcomes in children with SARS-CoV-2 infection

Study populations

Overall, the 21 studies included 81 896 children (range 14–71 700, median 58, IQR 25–151).25–45 Nine studies included >100 children (including controls).26–30 36 38 42 45 Children were analysed as a subgroup in four studies.25 26 33 37

Four studies, all controlled, primarily recruited children from the general population (ie, random sample from schools in Switzerland27; consecutive sample of home-isolated children in Norway25; based on large health insurance databases in Germany26; and based on a national health system database in the UK.28 Seventeen studies recruited only hospitalised children,30–32 34 37–44 only children attending the hospital as outpatient35 or both.29 33 36 45 Five of these 17 studies included children with multisystem inflammatory syndrome.30 39 40 44 45

Positive SARS-CoV-2 infection status was confirmed using RT-PCR only,28 31–38 41 43 a mix of tests (RT-PCR, antigen test or serology)25 29 30 39 40 44 45 or serology only.27

Negative infection status in the six controlled studies was defined as no diagnosis or symptoms of acute COVID-19,26 29 30 determined by serology testing25 27 or by RT-PCR28 (table 1).

Outcome data collection methods and outcome ascertainment

Outcome data were collected over 2–7 months (median 5.3, IQR 3–6) in the 12 studies starting follow-up at infection or onset25–30 35 40 42–45 and over 2.0–11.5 months (median 5, IQR 3.5–8) in the nine studies starting follow-up at recovery31–34 36–39 41 (table 2 and online supplemental eTable1). Outcome data were collected by phone,29 31 33 34 36 38 41 43 per clinical visit,32 35 39 42 44 routinely collected,26 30 40 45 online,27 28 by personal interview,25 or by a mix of case reports, medical records and self-reports.40

Table 2

Details on reported postacute health outcomes

Eight studies considered data on symptom duration.27 29 33 35 38 40–42 Symptom trend (n=3),36 38 40 severity (n=3)31 41 43, and frequency (n=2)31 38 were uncommonly considered. The main outcome was analysed in relation to participant characteristics or infection-related factors in eight studies.26 28 29 31 36 38 41 43

Outcome definition and outcome types

Only two studies provided clear background information on how they defined postacute symptoms (ie, questionnaire based on international working group31), expert opinion and published literature26 (table 2 and online supplemental eTable1).

Thirteen studies focused only on symptoms,26–29 31 36–39 41–44 while the others also reported numerous laboratory parameters.25 30 32–35 40 45 Reported symptoms ranged from 1 to 97 per study (median 9, IQR 1–20); however, for 11 of the 21 studies, the total number of assessed outcomes was unclear.27 29 30 32–36 39 44 45

The main outcome was a composite of any symptom in 16 studies25–29 31 33–39 42 43 45; the main outcomes in the remaining studies were fatigue44; respiratory,32 gastrointestinal40 and cardiac symptoms30; and olfactory symptoms.41 The three symptoms most frequently assessed across studies were headache (n=13),25–29 31 33 36–38 42 43 45 fatigue or tiredness (n=12),26–28 30 31 33 37 38 42–45 and cough (n=10).25–29 31 33 37 38 42

Outcome results

The reported proportion of children with post-COVID syndrome (main postacute health outcomes) was between 0% and 66.5% in children with SARS-CoV-2 infection (median 13%, IQR 0–22%, 17 studies; figure 1). Between 2.0% and 53.3% of children without SARS-CoV-2 infection also had such symptoms (control groups of six studies; figure 1).

Figure 1

Proportions of children with reported main postacute health outcomes. Orange-coloured: estimates and 95% CIs for children with SARS-CoV-2 infection; blue-coloured: estimates and CIs for children without SARS-CoV-2 infection in the six controlled studies.25–30 No data reported for children without SARS-CoV-2 infection.25 ,30

Only one study described a formal statistical comparison between SARS-CoV-2 infected children and controls, reporting an incidence rate ratio of 1.30 (95% CI 1.25 to 1.35) for all health outcomes combined.26

Consideration of confounding

Sixteen of the 21 studies did not allude to or mention confounding bias at all somewhere in the abstract or discussion section.25 27 31–44

Three studies alluded to confounding bias29 30 or acknowledged specific non-adjusted confounders,30 45 with one of them presenting a statement on residual confounding.29

Two studies statistically considered confounding26 28 (matching on age, sex and geographical area28; matching on age, sex and comorbidities26). Only one study clearly discussed confounding: ‘We cannot exclude that our results may be affected by unmeasured confounding, although we minimised differences between COVID-19 and control cohort via matching’.26

Ten studies clearly discussed other biases, for example, as information, referral, detection, response or recall bias, or alluded to other potential biases affecting, for example, missing data.26–31 38 40 44 45

Potential limitations were mentioned in the conclusion of the main text of one study26 (five studies had no clearly stated conclusion section or paragraph,25 27 40 42 43 while no study mentioned any limitations nor made a clear statement for cautious interpretation in the abstract (four studies had no abstract.27 36 42 43

Consideration of causality

A clear causal interpretation of an association between SARS-CoV-2 infection and the main outcome was made in 2 of the 21 studies; both were controlled studies and recommended actions regarding prevention measures25 26 (online supplemental eBox1). Only one of them had a statement that the results may be impacted by confounders.26 This was also the only study providing a clear causal disclaimer in the discussion, stating ‘Due to the observational nature of our study, a main limitation is that its design does not induce a causal interpretation of results’.26 No studies used a conceptual causal model.

Risk of bias

All 21 studies had an overall critical risk of bias with critical risk in at least one domain (figure 2 and online supplemental eTable2). Risk of bias due to confounding was critical in all studies.25–45

Figure 2

Summary of risk of bias assessment.

Risk of bias due to selection of participants was serious or critical in all studies except for two controlled studies that recruited from electronic health databases regardless of acute symptoms and hospitalisation.26 28

Risk in classification of infection status/exposure was low for 14 uncontrolled studies that identified patients based on RT-PCR tests during acute COVID-19.26 28 31–40 43–45 Risk of bias due to missing data was serious in 14 studies, because loss-to-follow-up was either >20%25 27–30 33 35 36 38 39 44 45 or the information provided was unclear.31 34 One study37 reported following all participants but used multiple data sources without providing further details and was deemed to be of moderate risk of bias.

Risk of bias in outcome measurement was serious for 12 studies that used self-report methods28–32 34 36 37 39 41 44 46; we assumed that participants who knew they were infected were more likely to report symptoms.46 47

Risk of bias in selection of reported outcome was critical in 16 studies either because they did not provide a protocol and/or the questionnaire used, failed to clearly define outcomes, and/or did not report all outcomes assessed.25 27–31 33–37 39 40 42 44 45

Discussion

This systematic review of 21 studies found critically limited robustness of reported SARS-CoV-2 infection health outcomes in children that may be perceived as post-COVID syndrome. There was huge heterogeneity in the definition, assessment and reported frequency of symptoms with frequently missing important information. Overall, none of the studies provided evidence with reasonable certainty on whether SARS-CoV-2 infection has an impact on postacute health outcomes, let alone to what extent.

There are two ongoing systematic reviews aiming to determine the prevalence of persistence of symptoms associated with SARS-CoV-2 infection in children,13–15 and two have been published.16 48 However, none specifically assessed the robustness of reported postacute symptoms with a focus on an integrative assessment of confounding, other biases and causal claims.

We avoided any quantitative synthesis of risk estimates, given the biases and heterogeneity in these studies; any combined estimate would most likely be misleading for patients, parents, clinicians, the general public and policymakers. Children in the control groups of the two largest studies26 28 very often had symptoms of post-COVID syndrome without infection (affecting 34% and 53% of children, respectively). This is much higher than any estimate for children with infections in the uncontrolled studies (except for one small study31), although these children were mostly hospitalised. This underscores that a control group is essential for a meaningful interpretation.

The exact relationships and inter-relationships of the various factors that influence both the risk of infection and the risk of symptomatic disease need to be understood. These may include age; social factors (eg, housing conditions, education level of children but also of parents and guardians, and family situation); psychological and mental factors (eg, mental illness or impairment, at least of the children themselves); and economic factors (eg, financial situation and additional financial burden on the family due to the pandemic). This may also include information on the parents as they most likely have serious impact on the children’s risk to be infected and becoming aware of symptoms. Of 21 studies, only 6 used control groups, and only 2 adjusted for some confounders, but their results were likely at risk of unmeasured or residual confounding. Hence, for all studies, the risk of confounding was critical.

Another level of complexity is the detection of outcomes due to the inconsistently used and non-standardised definition of ‘long COVID-19’, but also, and above all, the recording of the outcomes themselves, which are often self-reported and subjectively assessed in an unstructured, non-standardised way. A strong association between risk of infection (eg, being a close contact of a family member) and recognition of an outcome (ie, recognition of symptoms) can be assumed in this situation. It is conceivable that great concern about long COVID-19 leads to more cautious behaviour and more contact restrictions (possibly leading to a lower risk of infection but also higher psychological and mental distress), and to greater attention to symptoms and more frequent contacts with the healthcare system, increasing the probability of receiving such a diagnosis. Structured prospective data collection with a parallel control group would help to avoid such issues. However, given that most infections are unrecognised, a control group would need to be defined based on a strategy with high sensitivity (eg, both negative PCR and negative serology). Test-negative designs may help to address biases resulting from awareness of exposure (ie, infection).49

Overall, the 21 studies had mostly serious risk of bias for the outcome measurement; most studies had missing data for more than 20% of the participants and had a critical risk of selective reporting without clear prespecification of analyses, without protocols and with unclear definitions of results.

We did not consider studies reporting only surrogate outcomes (such as laboratory markers or healthcare use) unless we found information on pertinent patient-relevant outcomes.20 Studies were also not eligible if they reported only cumulative incidences of outcomes not allowing to differentiate the acute situation from the longer follow-up and acute symptoms from those persistent over time.

Overall, authors have mostly refrained from causally classifying their results or making exaggerated interpretations of their findings; most study groups have been cautious in their interpretations. While this restraint is commendable, none of the studies communicated the limitations of this evidence clearly or called for caution in their conclusions.

We encourage authors of future studies to report more granular details. Previous work has outlined recommendations that can help improve the design, conduct and reporting of such studies.4 For studies in children, specific issues require special attention. This includes consideration of confounding due to factors not related to the patient but to parents, guardians and the family that may be associated with both, infection risk and outcome (eg, educational level, housing situation, financial situation and additional financial burden on the family due to the pandemic). Other critical factors relate to schooling, for example, remote learning or school closure. All these factors are typically complex and not included in routinely collected data sources, requiring specific and very granular active data collection and careful consideration in the analyses.

Overall, studies on the effect of SARS-CoV-2 infection on post-COVID syndrome in children would at least require (1) to compare two groups of children with and without SARS-CoV-2-infection with parallel follow-up, (2) to avoid (or control for) any systematic differences of prognostic factors between the comparison groups and (3) to avoid any systematic differences in data collection and classification between the comparison groups.

Limitations

Since we included only articles published in English, we may have missed reports from authorities or published in other languages. However, a comparison with related reviews did not reveal such studies.

Our assessment of risk of bias and study characteristics involves a degree of subjectivism. However, given that many extracted items related to existence of reported information (eg, availability of study protocols), this left little room for inconsistency. For the risk of bias assessment, vital items were clear and unambiguous, and these alone determined the overall risk of bias assessment (eg, the critical risk of confounding bias that affects all studies). Here it needs to be highlighted that the tool we used has been designed for observational studies of interventions, but we felt it is the best available choice and it is important to note the underlying logic determining causal effects is the same for interventions (eg, drugs and vaccines) or exposures (eg, infections from viruses). Due to the often-limited reporting quality, isolated misinterpretations cannot be ruled out despite our predefined processes, but this would not change the overall interpretation.

Conclusions

Clarifying the frequency and severity of post-COVID syndrome remains an important research aim. The best possible research is needed to clarify the current conundrum. Children and their families need reliable and methodologically robust evidence to address their concerns and improve care.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information. All data generated or analysed during this study are included in this publication and supplement content.

Ethics statements

Patient consent for publication

Ethics approval

Not applicable.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors The authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Concept and design, critical revision of the manuscript for important intellectual content: all authors; acquisition, analysis or interpretation of data: JH, PJ, VLG, SS, TVP and LGH; drafting of the manuscript: JH, PJ and LGH; statistical analysis: JH and PJ; obtained funding: TVP (Chevening Scholarship Programme); supervision: LGH.

  • Funding TVP is funded by the Chevening Scholarship Programme (Foreign and Commonwealth Office, UK).

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.