Statistics from Altmetric.com
The meta-analysis of adenotonsillectomy trial results reported by van Staaij and colleagues,1 from which they concluded that the operation confers “an additional, but small, reduction of sore throat episodes...compared to watchful waiting”, falls short on numerous counts and misleads the unwary reader.
Their analysis of the trials conducted between the 1920s and 1960s not only overlooks previously published critiques of those trials,2–4 but fails to incorporate the sense of those critiques’ most telling criticisms, namely: the exclusion from most of the trials, on ethical grounds, of children thought to be severely affected; the inclusion of children who were only mildly affected and in whom, therefore, the benefits of surgery could at best be modest; and the limited nature of follow up procedures and, accordingly, of ascertainment of the numbers and types of succeeding episodes of throat infection. Relatedly, their analysis of trials conducted since the 1960s5,6 (each of which I led) omits mention of key elements and, most importantly, misconstrues the trials’ results and consequently their clinical implications.
The main single shortcoming of the meta-analysis is its failure, by considering only pooled risk differences across studies, to relate the outcomes of individual randomised trials to the stringency of the indications used in determining trial eligibility. Although the authors note that “the inclusion criteria of the trials varied from mild and non-specific to severe and very strict”, they pursue this issue no further and appear not to recognise its importance in determining outcomes. Thus, for example, the results of tonsillectomy in children meeting the stringent eligibility criteria of our first trial5 were more distinctly favourable than the results of either tonsillectomy or adenotonsillectomy in children who met the somewhat less stringent criteria of our second trial,6 even though the latter criteria were considerably more stringent than those used in the studies conducted before 1970.
Another important shortcoming of the meta-analysis is its failure to consider, in addition to the mean numbers of episodes overall occurring after randomisation, the nature and severity of the episodes that occurred and how they were distributed among the trial participants. Thus, in our first randomised trial in the first year after randomisation, there was a 2.5-fold reduction among surgical subjects in the mean number of throat infection episodes overall (47 episodes in 38 surgical subjects versus 108 episodes in 35 control subjects), but a 4-fold reduction in the proportion of surgical subjects who had three or more episodes of any type (13% [5 of 38] versus 54% [19 of 35]) and a 14-fold reduction in the mean number of episodes rated clinically as moderate or severe (3 episodes in 38 surgical subjects versus 41 episodes in 35 control subjects).5
It is certainly the case that, overall, children severely affected with recurrent episodes of throat infection who do not undergo tonsillectomy will likely have fewer episodes during succeeding years, but it is also the case that the risk of frequent and/or severe episodes continuing to occur in such children is sharply reduced by tonsillectomy. Physicians and parents considering the advisability of tonsillectomy must be cognisant of this balance of probabilities. Finally, it is important to emphasise that, in children meeting throat infection criteria less stringent than those of our first trial, the likelihood of substantial benefit from tonsillectomy is indeed small.6 However, to generalise from results obtained in children most of whom were not severely affected to children who are, and who might benefit substantially from surgery, is unjustified, and tantamount to throwing the baby out with the bathwater.
Dr Paradise criticises our review and claims that:
We overlooked previously published critiques of the controlled non-randomised trials conducted between 1920 and 1960
We considered only pooled risk differences across studies and we thereby did not relate the outcomes to the stringency of the inclusion criteria of the individual trials
We failed to consider the nature and severity of the episodes that occurred among trial participants
We generalised our results obtained in children of which most were not severely affected to children who are.
Dr Paradise brings up some relevant items, but we disagree with some.
First, controlled non-randomised studies are indeed less valid than randomised controlled trials. We therefore presented the results of the controlled non-randomised studies as circumstantial evidence in a separate table (table 3 of the article). Despite their shortcomings, the results of the more recent and methodologically sounder controlled non-randomised studies1,2 appeared to be surprisingly similar to those of the randomised trials.
Second, pooling of the results of the randomised trials was certainly justified in view of the lack of relevant heterogeneity among the studies. To take heterogeneity due to differences in design into account, we used a random effect model. Unfortunately, it was not possible to identify subgroups of children that might benefit (more) from the operation since the randomised trials published so far did not provide detailed results according to clinically relevant subgroups. We do, however, agree with Dr Paradise that there are subgroups of children that do benefit from (adeno)tonsillectomy. An efficient approach to detect these subgroups is to perform an individual patient data meta-analysis, using the original data from all (adeno)tonsillectomy trials performed in children so far, including the trials by Dr Paradise and his group.
Third, we indeed did not assess the influence of adenotonsillectomy on the severity of throat infections among trial participants. The main reason for this is the notion that any effect on the severity (and not the number) of the infections is likely to be attributable to information bias, caused by definition by the non-blinded nature of the studies. Parents of children in the watchful waiting group may be more likely to report a sore throat or upper respiratory infection as “severe” than parents of children in the surgical group, which would result in an overestimation of the effect of (adeno)tonsillectomy.
Finally, we agree that very frequent throat infections (seven or more in past year) is an adequate indication for tonsillectomy in children. That is why we excluded such children from our trial, which was published in 2004.3 Consistent with the results of the most recent trial by the group of Dr Paradise in “less severely affected children”,4 we found that in children with mild to moderate symptoms of throat infections, the likelihood of substantial benefit from tonsillectomy is indeed small. We therefore concluded that in these children the operation had no relevant clinical benefits to offer over a watchful waiting policy.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.