Article Text

Download PDFPDF

The evidence-base for complementary medicine in children: a critical overview of systematic reviews
  1. Katherine Hunt,
  2. Edzard Ernst
  1. Complementary Medicine, Peninsula Medical School, Universities of Exeter and Plymouth, Exeter, UK
  1. Correspondence to Dr Katherine Hunt, Complementary Medicine, Peninsula Medical School, Universities of Exeter and Plymouth, 25 Victoria Park Road, Exeter EX2 4NT, UK; Katherine.Hunt{at}pms.ac.uk

Abstract

Background The use of complementary and alternative medicine (CAM) in paediatric populations is common yet, to date, there has been no synthesis of the evidence of its effectiveness in that population. This overview of systematic review evaluates the evidence for or against the effectiveness of CAM for any childhood condition.

Methods Medline, AMED and Cochrane were searched from inception until September 2009. Reference lists of retrieved articles were hand-searched. Experts in the field of CAM were contacted. No language restrictions were applied.

Results 17 systematic reviews were included in this overview, covering acupuncture, chiropractic, herbal medicine, homeopathy, hypnotherapy, massage and yoga. Results were unconvincing for most conditions although there is some evidence to suggest that acupuncture may be effective for postoperative nausea and vomiting, and that hypnotherapy may be effective in reducing procedure-related pain. Most of the reviews failed to mention the incidence of adverse effects of CAMs.

Conclusions Although there is some encouraging evidence for hypnosis, herbal medicine and acupuncture, there is insufficient evidence to suggest that other CAMs are effective for the treatment of childhood conditions. Many of the systematic reviews included in this overview were of low quality, as were the randomised clinical trials within those reviews, further reducing the weight of that evidence. Future research in CAM for children should conform to the reporting standards outlined in the CONSORT and PRISMA guidelines.

View Full Text

Statistics from Altmetric.com

Introduction

Several surveys have suggested that, in paediatric populations, the usage of complementary and alternative medicine (CAM) is high.1,,3 Herbal medicine, homeopathy, reflexology and acupuncture are among the most popular treatments.2 4 5 The reasons for this high prevalence of use are diverse and might include dissatisfaction with conventional medicine and positive reports from friends and family.6 7 Children whose parents use CAM are almost five times more likely to use CAM than children whose parents do not use it.8

Whatever the motivations are, this level of popularity begs the question whether any of the treatments in question are effective.

In this overview, we aimed to critically review the literature pertaining to the effectiveness of CAM for children. In particular, we wanted to evaluate the evidence from systematic reviews on this subject.

Methods

We searched AMED using the EBSCO interface, Medline using the OVID interface and the Cochrane Library from inception to August 2009. Search terms were constructed using a series of keyword terms covering synonyms for ‘child’ and ‘paediatrics’ and also for key childhood diseases in order to identify those reviews which do not mention ‘children’ in the title; ‘Homeopathy for enuresis’, for example (see appendix 1). We defined CAM using the definition from the House of Lords Select Committee Report9 where “CAM is a title used to refer to a diverse group of health-related therapies and disciplines which are not considered to be a part of mainstream medical care. CAM embraces those therapies that may either be provided alongside conventional medicine (complementary) or which may… act as a substitute for it.” We did not include vitamins and supplements or exercise therapy in our definition of CAM but did include acupuncture/acupressure (of any form), Alexander technique, aromatherapy, (Bach) flower remedies, chiropractic, cranio-sacral therapy, herbal medicine, homeopathy (any combination of homeopathic tinctures), hypnotherapy/hypnosis, massage (of any form), naturopathy, osteopathy, spiritual healing, Tai chi, traditional Chinese medicine and yoga.

Search terms were adapted to run in each database. We included studies of infants, children and adolescents. We also sought systematic reviews from key researchers and clinicians in the field and searched our extensive departmental database. No language restrictions were applied.

We included systematic reviews (and meta-analyses) of randomised clinical trials (RCTs) of CAM for children. We only included systematic reviews of RCTs to align with Cochrane systematic review methodology and because non-randomised studies are more likely to produce effect estimates that indicate more extreme health benefits than randomised trials.10 This decision increases the reliability and validity of the review while reducing bias. However, this approach does have the limitation of excluding all other forms of evidence, something that may be more problematic when higher levels of evidence in the form of RCTs are not available.

We defined ‘children’ as babies, infants and adolescents. Reviews were defined as systematic if they included a repeatable and specific method of literature searching and used repeatable and explicit inclusion criteria. Non-systematic reviews, overviews and clinical trials were excluded. We also excluded reviews that were superseded by a more up-to-date review of the same topic for example, Richardson et al, Gotlib and Rupert11 12 or reviews that included studies on children as well as adults for example, Lee and Done13.

Abstracts of reviews located were read by one reviewer (KH) and those appearing to meet the inclusion criteria were retrieved and read in full by both reviewers. The reference lists of articles included in the overview were searched for additional studies. Two reviewers (KH and EE) independently extracted the data and judged the methodological quality of the reviews. Disagreements between the two reviewers regarding these judgements were resolved through discussion.

Results

After the removal of duplicates, our searches identified 420 articles. The majority were excluded because they were not reports of systematic reviews (figure 1). The included reviews fall into several categories:

  1. Those of (a specific) CAM for a specific condition (table 1).14,,22

  2. Those of a range of treatments (including CAM) for a specific condition (table 2).23,,28

  3. Those of a specific CAM for any paediatric condition (table 3).29,,32

Figure 1

Flowchart showing systematic review selection.

Table 1

Specific CAMs for specific conditions

Table 2

SRs including CAMs for specific conditions

Table 3

Specific CAMs for any paediatric condition

Although we conducted searches designed to capture a wide range of CAMs we only identified a small number of different CAMs. Twenty systematic reviews met the inclusion criteria and were included in this overview.

Methodological aspects of the included reviews

There was diversity in the quality of conduct and reporting of the systematic reviews included in this overview. For many of the reviews the QUORUM33 or PRISMA34 guidelines were not adhered to and in two, referencing errors and inconsistencies cause considerable confusion.31 32 (The QUORUM guidelines, now replaced with PRISMA guidelines were published to provide a standard for the satisfactory reporting of systematic reviews.) The use of power calculations, allocation concealment procedures and general assessments of methodological quality were markedly absent making assessment of the effectiveness of treatments problematic. These methodological issues are highlighted by the inconsistency and level of disagreement over the same RCTs. In some cases the same RCTs are discussed in multiple reviews and results are considered positive in some while not in others. This occurs, for instance, when only intra/within-group and not inter/between-group differences are presented. This is problematic because clinical effectiveness in an RCT can only be rigorously examined through analysis of the differences in outcome between treatment and control groups.

Acupuncture

Systematic reviews of acupuncture in children are of variable quality which makes any conclusions on the effectiveness of acupuncture for any condition problematic. In addition, RCTs of acupuncture commonly have methodological problems which further limit the ability to draw meaningful conclusions. For nocturnal enuresis for example, one systematic review identified only one (low quality) RCT (Jadad score of 1)35 which reported that conventional treatment with antidiuretic medication was superior to acupuncture.15 Conflicting results were reported by Glazener et al25 in a review of seven RCTs where acupuncture was shown to be superior to sham treatment and to desmopressin. However, these trials were undermined by their poor quality highlighted by an absence of blinding, allocation concealment and inadequate power.

For postoperative nausea and vomiting (PONV) the evidence is also mixed. One review23 of three RCTs for post-tonsillectomy or adenoidectomy vomiting showed no evidence of effectiveness. However, another systematic review reported positive findings from five placebo-controlled and two non-placebo-controlled RCTs for PONV.32 These findings should be interpreted with caution, however, because no statistics were provided to support the results, methodological quality was inaccurately assessed and referencing inconsistencies and inaccuracies were present. For chemotherapy-induced nausea, the results from one RCT are more promising with acupuncture significantly reducing the need for anti-emetics26 although this should be independently replicated.

Acupuncture was reported to be effective for pain relief, but no statistics were provided for any RCT and it is unclear whether intergroup analyses are presented.32 The same review reported improvement in laryngospasm and rhinitis while stridor was worsened by acupuncture.

Chiropractic

Fewer systematic reviews were available for chiropractic in children but the results were mixed and unconvincing. Three RCTs of chiropractic for colic were identified in two systematic reviews.17 31 Two of which were methodologically poor and the third, which was conducted more rigorously, did not suggest chiropractic is effective for that condition. None used validated outcome measures and relied upon parent self-report of colic symptoms despite the fact that parents were not blinded to group assignment. A third review16 found no RCTs of chiropractic for otitis media, despite the claims made by chiropractors that it is an effective treatment for that condition.

Two further reviews25 31 concluded that there is insufficient data to recommend chiropractic as an effective treatment for nocturnal enuresis in children. In the Glazener et al25 review the first RCT reported a significant difference in favour of chiropractic while the second did not report intergroup differences and inadequate data were provided to perform secondary analyses. In addition, there were differences in the severity of enuresis at baseline between the two groups and so these findings should be interpreted with caution.

Two trials reported conflicting results for the treatment of asthma; one poor quality study reported benefit while the other more rigorous RCT did not.31 Methodological problems associated with the review limit the interpretation of these results.

Herbal medicine

Although there are numerous systematic reviews of herbal remedies in adult populations, there is a paucity of such evidence for paediatric populations. Our searches revealed only two systematic reviews.18 24 The first assessed the effectiveness of ivy leaf extract for childhood bronchial asthma from three RCTs18 concluding that it is more effective than placebo. However, there were considerable methodological problems with the trials included and only one was placebo-controlled.

A systematic review of treatments for colic included one study of herbal treatments.24 That RCT tested a herbal tea containing chamomile, vervain, licorice, fennel and balm-mint, and found it to be effective in reducing colic symptoms as per the Wessel criteria but did not significantly reduce the number of night wakings in the treatment compared to the control group. The authors expressed concern about the volume of tea required to create this effect (32 ml/kg/day) and the impact that this could have on milk consumption.

A Cochrane review of Echinacea purpurea for the common cold included two RCTs of children.22 The first trial included children with early symptoms of common colds (mean age 7-years-old) and randomised participants to one of three groups: E purpurea juice from flowering plants, Andrographis paniculata with Eleutherococcus senticosus and no treatment. The authors did not report a conclusion for the comparison between herbs although the Cochrane authors do conclude a non-significant trend towards better symptom control in the Echinacea group compared to no treatment. The second trial compared E purpurea with a placebo syrup in children aged between 2 and 11 years. Healthy children were randomised to group and asked to commence treatment (7.5–10 ml/day depending on age, for 10 days) should they experience common cold symptoms. The study was rigorously designed, conducted and written-up but did not report significant differences in the severity or duration of cold symptoms between groups. In addition, the frequency of adverse effects (AEs) was significantly greater in the experimental group than in the control.

The effectiveness of Withania somnifera (Ashwagandha) was investigated by a systematic review that included one trial of children.28 Sixty healthy children aged between 8 and 12 years were randomised to one of three groups: W somnifera (Ashwagandha), W somnifera (Ashwagandha) or Boerhaavia diffusa (Punarnava) or placebo. A series of biological and haematological parameters were measured and a significant increase in haemoglobin and total protein was noted in the Withania group compared to the placebo group (p<0.01). In addition, an increase in haemoglobin and handgrip strength at 60 days was reported in Withania and Boehaavia group compared to placebo (p<0.05). However, there is no information on whether a power calculation was conducted, which given the small numbers in each group (n=20) and the high number of comparisons made, may reduce the reliability of the results. Nonetheless, these results are promising and further research is warranted into the effectiveness of these lesser-known herbs.

Homeopathy

Our searches identified two systematic reviews of homeopathy in children.19 29 The first of these assessed the effectiveness of homeopathy for childhood diarrhoea, concluding that duration of diarrhoea was significantly reduced in the treatment compared to control groups in three RCTs.19 However, all of the RCTs included in the review were conducted by the author of the systematic review. This calls the methodological quality of the review into question and it therefore requires independent replication. Furthermore, when the same three RCTs were analysed in another systematic review of homeopathy,29 only two were reported to show significant intergroup differences, but these were in favour of homeopathy.

Two of the three RCTs of homeopathy for attention deficit hyperactive disorder reported results in favour of homeopathy although the primary outcome measure was the Connor's Parent Symptom Questionnaire, thus relying upon parent self-report, a potential source of bias. One RCT of otitis and one of postoperative agitation also reported findings in favour of homeopathy yet the former again relied upon parent-report of symptoms.

Hypnotherapy

The hypnotherapy trials were generally of better quality than the other CAMs, but because of the nature of the intervention, none were placebo-controlled and hypnotherapy was compared with distraction or standard medical care. One review of 13 RCTs concluded that there is consistent evidence of effectiveness of hypnosis for procedure-related pain in children.14 Eight of these RCTs reported significant intergroup reductions in pain; one reported a reduction in length of stay and another, observer-related distress. However, it is unclear whether observer-related stress was the primary outcome and not procedure-related pain, for which the data are not presented.

The review conducted by Rheingans26 failed to report intergroup differences in seven of the nine included RCTs. The two studies for which these analyses were present showed hypnotherapy to be effective in reducing PONV or the use of anti-emetics. In addition to a lack of clarity in the type of analyses presented, this systematic review also provided a very poor analysis of the methodological quality of included trials making it difficult to make conclusions on the quality of the evidence presented.

Hypnotherapy was shown to be effective in treating enuresis in one of two RCTs25 and in one RCT for assisting the induction of anaesthesia.27

Massage

Two large Cochrane reviews have been published on massage for children. The first of which assessed the effectiveness of massage for promoting physical and mental health in infants aged less than 6 months.20 Data were meta-analysed and revealed no significant increases in weight, length, head circumference or mid-arm/leg circumference. However, massage was associated with a significant improvement in sleep hours compared to the control. Thirteen Chinese studies were reported to have a very high risk of bias and were meta-analysed separately as a consequence. These RCTs all had highly positive results on all parameters which led the review authors to surmise that selective reporting of results may have taken place.

The second Cochrane review assessed the effectiveness of massage in promoting growth and development of preterm and/or low birth weight infants.21 For that review, searches of Chinese databases were not conducted. Infants receiving massage had a reduced length of hospital stay and gained more weight than controls although the difference was of low clinical relevance. No difference was noted for orientation, state regulation or in infants receiving gentle touch without rubbing/stroking.

Yoga

One large, comprehensive, systematic review of 24 RCTs of yoga was identified.30 A trend was reported for improvement in motor planning and mental, social acuity and motor performance (improved reaction time, planning and execution times, motor speed) although some were poorly conducted studies with small sample sizes. Six RCTs with adequate methodological quality reported improvements in concentration and memory and one methodologically poor RCT reported significant improvements in IQ and social parameters in children with learning disabilities. Positive effects of yoga for anxiety and grip strength were reported in three and five RCTs, respectively although they were also of fairly low methodological quality.

Although the reviewers conducted systematic database searches, they failed to report the findings of the included studies adequately. This means, as with many of the systematic reviews in our analysis, it is unknown whether the RCTs have adequate power, whether allocation concealment was adequately carried out and insufficient statistics are provided to support the results. p Values are rarely provided and judgement of the overall quality of the included RCTs limited. In addition, there is considerable variation in the duration and type of yoga sessions across all the studies which makes their comparison problematic and it difficult to make a judgement of the effectiveness of yoga as a treatment for any condition.

Discussion

Our analyses do not reveal consistent evidence to suggest that CAM is effective for paediatric conditions. In fact, for some conditions, CAM seems to worsen symptoms, as was noted in children with stridor treated with acupuncture. However, despite great diversity in the quality of the evidence and resoundingly negative evidence for some CAM therapies, there are positive results for others. For instance, there are some encouraging findings that suggest acupuncture may be helpful for PONV, that homeopathy may relieve childhood diarrhoea, massage may increase sleep hours and that hypnotherapy may be effective in reducing procedure-related pain. In addition, there is limited evidence to suggest that Echinacea may also be effective in treating the common cold but external replication is required to further test this conclusion. Although there were some positive results for yoga, trials were of poor quality and a lack of standardisation of yoga practices makes their comparison problematic, thus necessitating additional, rigorous testing in the form of RCTs.

Although there are a plethora of RCTs of CAMs in children, there has been relatively little evidence synthesis assessing the weight of the evidence for a given CAM. More systematic reviews of CAM in children should be undertaken but they should, unlike the majority included in this report, be conducted according to the PRISMA guidelines.34 Indeed, we found that several of the systematic reviews had poorly developed search strategies, did not describe the strategies or imposed language restrictions causing bias and reducing the number of included RCTs. The fact that there was very little duplication of RCTs between reviews suggests that searches were frequently not designed or conducted appropriately. These limitations further reduce the ability to draw conclusions about the effectiveness of CAM in children.

It is not just the quality of the systematic reviews that makes analysis of CAM data complex, the quality of the RCTs included plays an even greater role. Studies assessing the RCTs of homeopathy have suggested that RCTs of better methodological quality tend to yield less positive results than RCTs of high quality.36 37 Moreover, publication bias is a common problem in CAM research,38 which combined with the selective reporting of results including failure to define the primary outcome, further reduces the robustness of findings.

There are several limitations to this overview. The decision to align with Cochrane methodology and exclude non-RCTs reduces the risk of bias but also reduces the coverage of this overview: CAM therapies that have not been tested in clinical trials are not represented. This report is also ultimately reliant on the quality of the systematic reviews and not the RCTs themselves: an overview of systematic reviews rather than of RCTs may have understated the quantity of evidence on CAM for children. In addition, we did not conduct an exhaustive search of all relevant databases nor potential CAMs and we do not report a full assessment of the systematic reviews. We also did not systematically go back to the original papers included in the review and so are unable to make judgements about their methodological quality if the reviews failed to do so. However, we did not set out to conduct a systematic review of systematic reviews, we aimed to provide a critical overview of the key systematic reviews of CAM for children. Furthermore, we did contact experts in the field of CAM paediatrics who identified a further three reviews which we subsequently included in our analyses. Another limitation relates to the definition of CAMs. We had to rely on the review and RCT's definition of the CAMs we included and therefore the ability to make comparisons between the acupuncture or homeopathy provided in one trial compared to that in another is reduced.

Any assessment of effectiveness of a treatment should be undertaken in tandem with consideration of safety. However, unlike in conventional medicine where AEs are routinely collected as part of clinical care and research, this is rarely the case in CAM.39 Most of the systematic reviews included in this overview did not report AEs. It is not clear whether this is a result of a lack of information in the original reports or in the quality of the systematic review. Indeed, it has previously been noted that RCTs of CAM frequently fail to report AEs of treatments40 and we know that reports of AEs of CAM probably represent the tip of the iceberg because at present it is impossible to provide reliable incidence figures.41 For acupuncture, for example, AEs are more common than previously reported, estimated at 5.36 per 10 000, compared to previous estimates of 0.05 per 10 000 treatments.42 Moreover, AEs of CAM extend beyond the actual treatments themselves. It has been shown that children who consult naturopathy and chiropractic practitioners are significantly less likely to receive the recommended vaccinations and were significantly more likely to be diagnosed with a vaccine-preventable disease.43

Before recommendations for the use of CAM in children can be made, there needs to be a greater emphasis on the rigorous testing of therapies using standardised techniques and a concerted emphasis on the reporting of AEs in practice and research. For children, this is a particularly pertinent issue given that they are unable to give informed consent to treatment and thus rely upon appropriate decisions about their care being made on their behalf.

Appendix

Appendix 1

Search strategy

References

View Abstract

Footnotes

  • Competing interests None.

  • Provenance and peer review Commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.