Tonsillectomy with or without adenoidectomy is one of the most commonly performed surgical procedures in children in western countries.¹ The most common indications are recurrent upper respiratory infections (URIs) and obstructive sleep apnoea. For children with obstructive sleep apnoea due to adenotonsillar hypertrophy, adenotonsillectomy has proven effective.^2–4 Regarding URIs, however, evidence for its efficacy is limited and no nationally or internationally accepted guidelines on the indications for this procedure are available.^5,6

In 1998 two reviews were published on trials of the efficacy of (adeno)tonsillectomy (T±Ads) for recurrent throat infections in children.^7,8 Both concluded that the available studies were of poor quality. For several reasons, we wondered whether the current evidence on the effectiveness of (adeno)tonsillectomy is as poor. First, a literature search from Medline with search term “tonsillectomy” produced 5771 hits. Although most of these were non-randomised studies, they may offer important additional evidence. Second, both the Cochrane reviewers and Marshall focused on the reduction of sore throat episodes, but for patients and practising physicians, outcomes such as sore throat associated school absence and upper respiratory infections are also important.^9–12 Third, the recent report of a second trial by the Pittsburgh group may offer additional evidence.¹³

Therefore, we performed a systematic review to provide a quantitative estimate of the effects of (adeno)tonsillectomy on sore throat episodes, upper respiratory tract infections, and sore throat associated school loss, based on available evidence from randomised trials and non-randomised controlled studies.

METHODS

Study retrieval and selection

A computerised literature search was done in the Medline (Index Medicus 01/1966–06/2003), OldMedline (Index Medicus 01/1963–12/1965), and Cochrane databases for articles containing original data on the efficacy of (adeno)tonsillectomy in children (appendix 1).

Reference lists from identified publications were screened to identify pre-1963 studies. Only articles published in English were retrieved.

Eligibility

Studies were included that met the following criteria: (a) randomised trial or non-randomised controlled study investigating the efficacy of (adeno)tonsillectomy; (b) the control group underwent either no surgery or adenoidectomy only; (c) age at inclusion below 18 years; (d) clinically relevant outcome measures were reported—that is, sore throat episodes and/or upper respiratory infections and/or sore throat associated school loss and/or fever episodes; (e) results were published before June 2003.

Selection of articles

Two reviewers (BvS, EHvdA) independently assessed eligibility of studies. Randomised trials and non-randomised controlled studies were summarised separately.

Randomised trials

Methods appraisal

The two reviewers performed the quality assessments of the randomised trials independently. The maximum quality score for each study was 10 for internal validity and 8 for external validity (that is, generalisability) (see appendix 2).¹⁴ For each criterion the reviewers assessed the completeness of the information (“yes” 1 point; “no” 0 points, “unclear” ?). Validity scores were used to rank studies.

Data extraction

Information on patient characteristics (P), interventions (I), the contrast between the interventions (C), and outcomes measured (O) were extracted from all included studies.

Clinical outcomes

The main outcome measures were sore throat episodes, days of sore throat associated school absence, and upper respiratory infections. Results as reported by the authors in the articles were used; no attempt was made to retrieve the original data from the authors as four of six trials were performed more than 15 years ago.^15–18 The incidence of upper respiratory infections for the trials by McKee and colleagues^15,16 was calculated by adding up the episodes of cold, cough, influenzal illness, other respiratory illness, and otitis media. For the trial by Mawson and colleagues,¹⁷ the incidence of upper respiratory infections was calculated by adding up the episodes of earache, otitis media, and head cold.

Statistical analysis

Effects on the outcome measures were summarised as risk differences and ratios.

For the outcome parameters sore throat episodes, sore throat associated school absence, and upper respiratory infections in randomised trials, pooled estimates of the rate ratio and rate difference with 95% confidence intervals were calculated. Poisson regression was used assuming that the number of observed episodes followed a Poisson distribution. Incidence rates per person-year were calculated to account for differences in duration of follow up between studies. Random effect estimates were reported because the study results were statistically heterogeneous. To test whether the association between (adeno)tonsillectomy and the outcomes was homogeneous in the two types of analyses (per protocol versus intention to treat), the significance of the respective interaction term was tested. When a significant difference was found, it was tested whether the difference resulted in different treatment effects.

Non-randomised controlled studies

Data extraction, clinical outcomes, and statistical analyses were performed in the same way as in the randomised studies. However, no quality assessment scores were derived and pooled estimates were not calculated.

RESULTS

Study selection

Seventy one studies on the efficacy of (adeno)tonsillectomy for upper respiratory infections in children were identified. Fifty two studies were excluded for the following reasons: 19 were uncontrolled studies; seven included only a before-after treatment comparison; 10 included a control group of healthy controls; and 16 were reviews. Multiple publications were excluded from our analyses.^19–24 Hence, six randomised trials^13,15–18 and seven non-randomised controlled studies^18,25–30 were included.

Validity criteria of the randomised trials

Internal validity of all the randomised trials was rather poor (maximum score 6 out of 10), whereas the external validity (that is, generalisability) was generally better (minimum score 5 out of 8); see table 1.

Characteristics of the randomised trials (table 2)

The inclusion criteria of the trials varied from mild and non-specific to severe and very strict. Marked differences existed between the inclusion periods of the studies and varied from 6 months to as much as 12 years. In five trials the control group received watchful waiting or non-surgical management, and in one study adenoidectomy.¹⁶ Mean age at inclusion varied from 6.0 to 8.1 years. In all trials except one,¹⁶ the loss to follow up was considerable: 8% to 39%. The percentage of switchers from the watchful waiting to the (adeno)tonsillectomy group varied, except in one trial,¹⁶ from 12% to 28%. None of the studies supplied a power analysis.

Characteristics of non-randomised controlled studies (table 3)

Five studies were prospective cohort studies and two were retrospective cohort studies. Main limitations, apart from of course the non-randomised allocation to T±Ads, were that the loss to follow up and the number of children who changed from the watchful waiting to the (adeno)tonsillectomy group were not described.

Efficacy

Outcomes of the randomised trials

The outcome sore throat episodes was studied in all trials, sore throat associated days school absence in five trials, and upper respiratory infections in three trials. Outcomes were assessed at 1, 2, and 3 years. In none of the studies data from children with a follow up of less than one year were included in the analyses.

Pooled estimates

For the outcome sore throat episodes, 2483 person-years were analysed (54% in intervention group). The pooled risk difference was −1.2 episodes per year (95% CI −1.3 to −1.1). The heterogeneity between the studies analysed per protocol versus intention to treat was statistically significant (p < 0.05), but no significant treatment effect was observed (p = 0.80). The pooled risk ratio for sore throat episodes was 0.49 (95% CI 0.30 to 0.79).

For the outcome sore throat associated school absence, 1669 person-years were available (56% in intervention group). The pooled risk difference was −2.8 days per person-year (95% CI −3.9 to −1.6) and the risk ratio for sore throat associated school absence was 0.50 (95% CI 0.26 to 0.97). No significant effect of the different types of analyses (that is, intention to treat or per protocol) was found.

For the outcome upper respiratory infections, 1596 person-years could be analysed (54% in the intervention group). The pooled risk difference was −0.5 episodes per person-year (95% CI −0.7 to −0.3). The risk ratio for upper respiratory infections was 0.97 (95% CI 0.69 to 1.36). Exclusion of the study by McKee,¹⁶ the only study comparing adenotonsillectomy versus adenoidectomy, yielded similar results.

Outcomes of the non-randomised controlled studies

Except for frequent fever attacks (rate ratio ⩾1.0), rate ratios for all outcomes (episodes of throat infection, sore throat associated days school absence, upper respiratory infections, and doctor’s visit for respiratory diagnosis) were ⩽1.0, indicating a beneficial effect of (adeno)tonsillectomy.

DISCUSSION

(Adeno)tonsillectomy reduces the incidence of sore throat episodes by 1.2 episodes per year (95% CI 1.1 to 1.3), sore throat associated school absence by 2.8 days per year (95% CI 1.6 to 3.9), and upper respiratory infections by 0.5 episodes per year (95% CI 0.3 to 0.7).

In contrast with the Cochrane reviewers,⁷ who excluded all trials in which children of the surgical group were randomised to adenotonsillectomy instead of tonsillectomy alone, all randomised trials studying the efficacy of (adeno)tonsillectomy were included in the present meta-analysis. In daily practice most children suffering from recurrent throat infections undergo tonsillectomy combined with adenoidectomy and not tonsillectomy alone; in the Netherlands 90% of tonsillectomies in children are combined with adenoidectomy, in the USA this percentage is 84%, in Canada 75%, and in England 32%. Our meta-analysis shows that randomised trials comparing adenotonsillectomy versus watchful waiting or tonsillectomy alone versus watchful waiting provide similar results.

Our meta-analysis was performed to assess the effects of (adeno)tonsillectomy for upper respiratory infections. The effects of (adeno)-tonsillectomy in children with obstructive breathing during sleep were not considered.^2–4

It is important to realise that all trials had serious methodological limitations, which precludes definite conclusions about the effects of (adeno)tonsillectomy on upper respiratory infections. First, the generalisability of the results of the trials can be questioned, since only a very small proportion of children undergoing T±Ads was included in the trials. (Adeno)tonsillectomy is one of the most commonly performed surgical procedures in children in western countries; in 1998, for example, 65/10 000 underwent T±Ads in England and 50/10 000 in the United States.¹ Yet the three Pittsburgh trials^13,18 included only 233 children in the T±Ads group and 186 children in the watchful waiting group, with an inclusion period of respectively 11 and 12 years. Second, all studies had significant loss to follow up. This can be associated with either good or poor outcome. However, in four^13,17,18 of six studies, information about the children who were lost to follow up was provided, and in these studies the rates of throat infection during the preceding follow up period did not differ significantly from the corresponding rates in the respective treatment groups as a whole. Third, three studies were analysed per protocol.^15,16,18 These per protocol analyses underestimate the treatment effect as in surgical trials only children of the watchful waiting group with severe complaints can change treatment group, whereas children of the surgical group, who may experience serious complaints, cannot change treatment group.^15,16,18 Fourth, information bias may be considerable since trials on adenotonsillectomy, as most surgical trials,^31,32 cannot be performed in a true double blind fashion. None of the trials tried to minimise information bias by choosing an objective outcome measure, such as fever measured daily by a validated thermometer automatically storing data.³³ Fifth, none of the trials provided a power analyses. As all trials, but especially the Paradise trials, included relative few patients, their power may be too low, leading to a type II error.

The pooled risk difference for recurrent throat infections was −1.2 episodes per year (95% CI −1.3 to −1.1). However, in three trials the sore throat episode immediately following the operation was not counted.^15–17 Had these been counted, the differences between the groups would have been smaller.

The pooled risk difference for sore throat associated school absence was −2.8 days per year (95% CI −3.9 to −1.6). In none of the trials, however, was sore throat associated school absence immediately following surgery counted. If these days had been included, the rates would probably not have been different. Thus, although (adeno)tonsillectomy reduces the total number of sore throat episodes by a modest 1.2 episodes per year (95% CI 1.1 to 1.3), the reduction in sore throat associated school absence is even more modest. This indicates that the severity of the throat infections in the children of the control group was likely not serious enough to cause substantial school absence.

The pooled risk difference for upper respiratory infections was only −0.5 episodes per year (95% CI −0.7 to −0.3), indicating that (adeno)tonsillectomy has little effect on the incidence of upper respiratory infections. This is important since several recent studies have shown that many ENT surgeons and general practitioners still regard upper respiratory infections as an indication for (adeno)tonsillectomy.^9–12

In all studies children of the control group had more sore throat episodes and more upper respiratory infections than the children of the surgical group. In all studies, however, the children of the control group experienced fewer episodes during the follow up period than before study entry (table 2). This natural decrease of the incidence of throat infections is probably attributable to maturation of the immune system with growing age, whether surgery is performed or not. Regression to the mean could also play a role. As a result, surgery induces an additional reduction of sore throat episodes of only 1.2 episodes per year (95% CI 1.1 to 1.3).

It should be emphasised that the results of all trials are indicative of a difference of a strategy involving (adeno)tonsillectomy and a strategy involving initial watchful waiting, knowing that a proportion of the latter will switch. As in many other surgical trials, the number of switchers was high in most trials. It is very likely that these children have had more throat infections than the children who remained in their original allocated group. This may have lead to an underestimation of treatment effect.

The non-randomised controlled studies, except one,¹⁸ show the classical shortcomings of non-experimental studies: incomparability of the study groups at baseline, which leads to confounding by indication.³⁴ In these older studies, techniques that can be used to control for these imbalances of known or suspected risk factors such as multivariate adjustment were not used. In their critical article, Selkirk and Mitchell³⁵ already recognised these problems in 1931. Overtime, the quality of the non-randomised controlled studies has improved. For example, in the older studies by Kaiser and Monroe,^25–27 inclusion criteria and outcomes are ill defined, while in the more recent studies they are better defined.^18,29,30 Despite these shortcomings the results of the more recent and better controlled studies^18,28–30 are surprisingly similar to those of the randomised trials and therefore support evidence on the efficacy of (adeno)tonsillectomy from the trials.

This systematic review shows that all trials and controlled studies have important limitations. Throughout all of the studies the frequency of sore throat infections and upper respiratory infections reduces with time, whether (adeno)tonsillectomy has been performed or not, highlighting the importance of controlled studies. Available evidence from both the randomised trials and non-randomised controlled shows that (adeno)tonsillectomy gives an additional, but small, reduction of sore throat episodes, sore throat associated school absence, and upper respiratory infections compared to no surgery.

APPENDIX 1: MEDLINE SEARCH

((“tonsillectomy”[mh] OR tonsillectomy[all fields] OR adenotonsillectomy[all fields] OR “Adenoidectomy”[mh] OR Adenoidectomy[all fields]) AND ((placebo[all fields] OR “drug therapy”[sh] OR “therapeutic use”[sh:noexp] OR “random*”[all fields] OR “randomized controlled trial”[pt] OR “Clinical Trials”[mh] OR “Comparative Study”[mh]) OR (“incidence”[mh] OR “mortality”[mh] OR “follow-up studies”[mh] OR “mortality”[sh] OR prognos*[all fields] OR predict*[all fields] OR course[all fields] OR “Population Surveillance”[mh] OR “Remission, Spontaneous”[mh]) OR (“Review Literature”[MH] OR Meta-Anal* OR “meta-analysis”[pt] OR metaanal*) OR ((quantitativ*[tw] OR systematic*[tw] OR methodologic*[tw]) AND (review*[tw] OR overview*[tw])) OR ((“review”[pt] OR review*[tw]) AND (“medline”[tw] OR “cinahl”[tw] OR “embase”[tw] OR “excerpta”[tw] OR “odds ratio”[tw] OR “pooled”[tw] OR “pooling”[tw])))) NOT (letter[pt] OR editorial[pt] OR comment[pt] OR in vitro[mh] OR (“animal”[mh] NOT (“human”[mh] AND “animal”[mh]))) Field: All Fields, Limits: All Child: 0–18 years.

APPENDIX 2

Criteria for the assessment of internal validity

(V1) Was the treatment allocation performed in an unpredictable sequence?

(V2) Was the treatment allocation concealed (sealed envelopes, allocation by telephone, etc)?

(V3) Were the groups similar at baseline regarding prognostic indicators and baseline scores?

(V4) Was the care provider blinded to the treatment (use of a placebo)?

(V5) Were co-treatments avoided or standardised?

(V6) Was the compliance rate (in each group) unlikely to cause bias?

(V7) Was the patient blinded to the allocated treatment?

(V8) Was the crossover/dropout rate unlikely to cause bias?

(V9) Was the outcome assessor blinded to the treatment?

(V10) Was the timing of the outcome assessment in both groups comparable?

Criteria for the assessment of external validity

(D1) Were the eligibility criteria specified?

(D2) Were the compared treatments explicitly described?

(D3) Was information about the method of assessment of outcome measures presented?

(D4) Were there a short term (immediately after treatment) and a long term follow up measurement?

(D5) Were adverse effects described?

(D6) Was sample size for each group described, after allocation and at outcome measurement?

(D7) Did the analysis include an intention-to-treat analysis?

(D8) Were point estimates and measures of variability presented for primary outcome measures?

V1 Random (unpredictable) assignments sequence. Methods of allocation using date of birth, date of admission, hospital numbers, or alternation should not be regarded as appropriate.

V2 Assignment generated by an independent person not responsible for determining the eligibility of the patients. This person has no information about the persons included in the trial and has no influence on the assignment sequence or on the decision about eligibility of the patient.

V3 To receive a “yes”, groups must be similar at baseline regarding age, frequency of prior episodes, duration of complaints, and severity of complaints.

V4 (Attempt for) blinding described in paper.

V5 Co-interventions should either be avoided in the design or comparable between the index and control group (use of antibiotics, ventilation tubes, adenoidectomy, and attention from researchers, etc).

V6 Treatment should be provided as randomised. Non-adherence (protocol deviation, crossover) is acceptable if it is <15% for both groups separately and <5 % between groups. Qualitative measurement.

V7 (Attempt for) blinding described in paper.

V8 Quantitative measurement. A yes is scored if “non-adherence” and missing data do not lead to substantial bias.

V9 The reviewer determines (per outcome parameter) when enough information about the blinding is given in order to score “yes”.

V10 Timing of outcome assessment should be identical for all intervention groups and for all important outcome assessments.

D1 The reviewer determines if the eligibility criteria are well described.

D2 Adequate description of both the index and control intervention should be given, so that others could replicate the treatment.

D3 Adequate description of the method of assessment of outcome measurements should be given so that others could replicate the study.

D4 Outcome measurements ⩽1 year after the intervention and outcome measurements at the end of the intervention period. Follow up time and characteristics of effect measurements should be the same in both groups.

D5 Each event should be described and correctly attributed to the allocated treatment (for example, postoperative bleeding, psychological disturbances).

D6 To be presented for each group at randomisation and for the most important outcome assessments.

D7 All randomised patients are reported/analysed for the most important moments of effect measurement (minus missing values) irrespective of non-compliance and co-interventions.

D8 Both point estimates and measures of variability should be presented (to be scored for each important outcome parameter separately). Point estimates are means, medians, modes, etc. Measures of variability are: standard deviations, 95% confidence interval, etc).