Objective Randomised controlled trials (RCT) offer an opportunity to learn about frequency and character of adverse drug reactions. To improve the quality of reporting adverse effects, the Consort group published recommendations. The authors studied the application of these recommendations in RCTs performed in children.
Design Literature search.
Results The authors found 107 articles on paediatric RCTs written in English and published between 1 January 2006 and 1 April 2009. 83 articles (78%) mentioned adverse drug reactions, 36 articles (34%) used standardised methods for the disclosure of adverse drug reactions, 33 articles (31%) tabulated the reactions, and 27 (25%) stated numbers of and reasons for withdrawal. Registration of the RCT did not influence reporting. Sponsoring did lead to better reporting. According to the Consort guidelines, 19 (18%) reported safety data adequately.
Conclusion Reporting of adverse drug reactions in RCTs is often inadequate. Authors should anticipate and define adverse drug reactions. During research all adverse events should be monitored and assessed actively. Monitoring and assessment should be described in articles in accordance with the extended Consort recommendations. Both authors and editors share the responsibility for the improvement of reporting safety data.
Statistics from Altmetric.com
Randomised controlled trials (RCTs) offer a unique opportunity to learn about the benefits of treatments and can provide important clues to undesired effects of the therapy studied. The original Consort guidelines advise on the improvement of reporting undesired effects.1 Table 1 sums up the more accurate and extended recommendations formulated in 2004 by the Consort group.2 Those recommendations were formulated in cooperation with Ioannidis, author of several studies on the quality of reporting adverse effects.3 4 Table 2 sums up the five recommendations to improve the quality of reporting adverse effects as supplied by Ioannidis.4
We studied whether the recommendations on reporting safety data were followed in recently published RCTs conducted in children.
For this study we searched PubMed for RCTs written in English, published from 1 January 2006 up to 1 April 2009 and performed in children aged 0–18. Articles about RCTs performed in children containing references to these RCTs in titles or abstracts were included. Articles retrievable directly from the internet on RCTs via the library of the University Hospital Groningen on medication with systemic availability (oral, inhaled or intravenous) were included in the study. We defined medications as chemically produced substances and therefore studies on probiotics and vaccines were excluded. Also, studies on hormone replacement therapies, for example, insulin, were not included.
What is already known on this topic
Randomised controlled trials are well established for giving evidence for or against beneficial effects of drug studied.
What this study adds
▶ In many articles published on paediatric randomised controlled trials (RCTs), data on adverse drug reactions are not reported according published guidelines.
▶ Adequate reporting of adverse drug reactions in paediatric RCTs is a responsibility of both authors and editors.
General information, such as year of publication, region of origin, formal registration, organ system studied, Anatomical Therapeutical Code (ATC) group of the medication, study duration and sponsoring by pharmaceutical industry, were captured.
All parts of the articles were checked on safety data, allowing a wide range of adjacent terms such as harm, tolerability, side effects, adverse drug reactions and adverse effects.
When such a term was mentioned in the method section, we assumed that the researchers had been searching for drug-related side effects actively. When a protocol for the study of adverse drug reactions was mentioned, when patients were questioned about adverse drug reactions, or when a history, a physical examination or laboratory research was carried out, we assumed that a standardised protocol for the disclosure of adverse drug reactions was followed. Also we searched for standardised methods of categorisation of adverse drug reactions (fatal, serious or severe; moderate or mild).
The results sections were searched for descriptions of adverse effects. When the number of children experiencing adverse drug reactions was given or the numbers of adverse drug reactions, these were included. We looked for numbers of fatalities, severe or serious events and for tables of adverse effects. We also searched the text for statements about the withdrawal of individuals from the study due to adverse drug reactions and for numbers of adverse drug reactions per study arm.
On the basis of publications by Ioannidis and the Consort group, two surrogate quality markers were constructed. Each recommendation complied with was scored and the scores were added up. So the maximum outcome for the Ioannidis score was 5 and we considered a score of 3 or higher to be sufficient. One of the Consort recommendations is about a specific group and because we only studied paediatric RCTs we decided to omit this one. Hence, the maximum Consort score was 9. A Consort score higher than 5 was arbitrarily chosen as being sufficient. Statistical analysis was performed using SPSS for Windows V.13. We used two-sided χ2 tests. A p<0.05 was considered statistically significant.
We found 107 articles, 21 of these (20%) registered and 40 (37%) sponsored by a pharmaceutical company. Thirty-four studies (32%) were performed in USA and Canada, 28 studies (26%) in Asia and three were performed on a global scale. One-third of the studies were of the ATC group N (nervous system), a major 25% of the studies was on the respiratory system (table 3).
Table 4 summarises the findings. In 83 articles (78%) adverse events were mentioned. Forty-nine articles (46%) contained remarks on adverse events in their method sections, 36 (34%) of these used a standardised method for the disclosure of adverse drug reactions and 10 (9%) mentioned a standardised categorisation of side effects. Eighty-one articles (76%) discussed adverse events in their results sections, 28 articles (26%) reported the number of children with adverse drug reactions. In 27 studies (25%) the number of patients withdrawn as a result of adverse effects was reported. The number of side effects per study arm was given in 69 articles (65%).
The mean Ioannidis score was two (median two); 31 out of 107 manuscripts (29%) scored three or higher and were considered of sufficient reporting quality. The mean Consort score was 3 (median 3), 19 articles (18%) scored 6 or higher and could therefore be considered sufficient.
The percentage of studies registered increased significantly from 4% in 2006 (2 out of 47) to 42% in 2008 (13 out of 31). The mean Ioannidis scores did not differ significantly between registered and non-registered studies (2.2 vs 1.5) nor did the Consort scores (3.8 vs 3.2). Forty studies were sponsored by industry, the mean of Ioannidis and Consort scores were 2.3 and 4.3 for sponsored studies, and 1.3 and 2.7 for studies not sponsored; these differences are statistically significant (27).
A total of 24 056 children participated in the studies and 7246 adverse events were reported. Of these events, 22 were fatal and 338 were classified as serious. A majority of 19 fatalities occurred in the study of Kobbe et al5 who studied the efficacy of antimalaria medication in 1070 children. It was not indicated whether these fatalities were the result of malaria or of the medication. There were no differences in numbers of fatalities between study group and comparison group (46). There was no correlation between the number of adverse effects reported and the duration of the studies.
In this study we found that reporting safety data in paediatric RCTs is often insufficient. In one-fifth of the studies adverse drug reactions or adverse events were not mentioned at all, whereas in only one-third of the studies a protocol for the disclosure of adverse drug reactions was used. When the Consort score was used as quality marker, 18% of the studies reported adverse drug reactions in a sufficient way and only one study gave a full report according to these recommendations. When using the Ioannidis recommendations as quality markers, 29% of the studies could be considered sufficient in reporting side effects.
There are not many articles on the quality of reporting adverse drug reactions in RCTs. Ioannidis et al reviewed 60 publications on HIV studies. They found that only a few studies provided sufficient information. As a proof they demonstrated that the space allocated to safety issues was smaller than the space used for contributor names.3 In a later study they found that 29% of the 192 studies provided data on safety adequately.4 Although in the present study the numbers are higher, there is room for improvement.
We propose to define adverse drug reactions before the beginning of a study on the basis of experience in both adult studies and phase I and II studies, as well as on knowledge on adverse drug reactions in medications from the same chemical group. Also, medication-independent adverse events that can be expected to occur in the group of patients studied should be defined. Only then undesired drug reactions, due to the treatment studied, can be discovered. An independent pharmacist and an expert in the field should review adverse events and decide whether an event is an adverse drug reaction or not.
Another important issue in reporting safety data is the terminology used. Terms such as ‘adverse effects’, ‘adverse events’, ‘side effects’ and ‘adverse drug reactions’ are used in different contexts and meanings by various authors. This inconsistency has been criticised before.6 In some studies ordinary events in children's lives, like diarrhoea with toddlers, were reported in studies of inhaled corticosteroids as adverse events. This confuses the judgement on the numbers of side effects. Obviously, this is one of the reasons for the Consort group's recommendation to use the words ‘harm’ and ‘safety’.2
Some studies reporting more than one side effect in patients also confuse the interpretation of the results of safety data. Eleven studies even reported more side effects than patients studied. We suggest that not only numbers of side effects should be provided, but also numbers of patients suffering from any adverse drug reaction or, even better, numbers of patients not suffering from any adverse drug reaction. Also, numbers of individuals with adverse drug reactions in both study and comparison group should be given. By doing so the number of patients needed to harm can be compared with the number needed to treat.
We were disappointed that registration did not affect the quality of reporting, although we must add that the number of studies registered was relatively low. When a study was sponsored by an industry the quality of reporting safety was significantly higher. We do not wish to speculate on possible explanations for this phenomenon.
There are a few possible explanations for the low reporting of safety data we found. First, in some studies, medications like antibiotics that have been used widely were studied for a different strategy or therapy duration. However, we feel that even when these medications are studied, a survey of the occurrence of adverse drug reactions is warranted. Patients and doctors consider the choice of a therapy not only on its beneficial effects, but also take into account its undesired effects.
Another possible explanation for our findings is that we missed a large amount of RCTs in which adverse drug reactions were reported more adequately. We used PubMed as sole source and included only articles we could retrieve via the University Hospital Library and excluded studies on hormone replacement therapy, probiotics and vaccines. However, since PubMed was used as a source of the publications and all the studies we encountered were included, we suppose that the most important, most widely read and easily available articles have been studied. We feel that the articles include reflects the practice of reporting adverse drug reactions.
By assuming that when a remark on looking for adverse drug reactions was mentioned in the methods section a standardised protocol was used, we possibly overestimated this item. Because there are no formal tools for studying the quality of reporting adverse drug reactions this is not a systematic review, but merely a survey. Despite this, we argue that reporting adverse drug reactions should improve.
Of course, RCTs are not the sole source of data on adverse drug reactions. National and international pharmacovigilance organisations and postmarketing surveillance are important tools for the discovery of adverse drug reactions.7 In conjunction with the data on adverse drug reactions gathered in RCTs, a sound and realistic estimate on risks can be made.
Not only authors are responsible for adequate reporting on safety data, but journal editors may also play an important role, demanding their authors to adhere to the recommendations. We argue that an assessment of safety reporting is warranted in every report on comparison of treatments.
We demonstrated that reporting safety data in paediatric RCTs is often insufficient. Since any treatment can have adverse effects, research should anticipate the disclosure of adverse drug reactions by defining them prior to the study and by monitoring them during the study. Authors of articles as well as journal editorial boards should be stimulated to follow current recommendations.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.