Several studies have shown that adverse healthcare events (critical incidents and near misses) occur frequently, and that they often remain undetected.^1–8 Traditionally, a combination of cultural, institutional, and legal factors has led to widespread under-reporting of such events.^9,10 Reporting rates are strongly influenced by the institutional climate (whether or not there is a blame culture)^11,12 and by profession.^13,14 The medico-legal implications of owning up to failure, a history of punitive action against individuals who make errors, fear of loss of professional reputation, a culture of silence, and lack of trust, have been identified as potential barriers to reporting.¹⁵

The influence of each of these variables on reporting behaviour may vary and needs further exploration, although evidence suggests that nurses typically report more often than doctors.^13,14 To date, there is little information on how organisational learning in the NHS is impaired by the low reporting rates among doctors. Similarly, it is not known what types of adverse events tend to escape detection when doctors become disenfranchised from a hospital’s official reporting system, and the information available on the extent and implications of “covert reporting” is limited. Covert reporting occurs when a healthcare professional becomes disillusioned with official reporting methods, and seeks to learn from previous experience by setting up his or her own reporting method.

High profile failures such as the Bristol Royal Infirmary children’s heart surgery affair,¹⁶ constant media scrutiny of medical errors, landmark reports on both sides of the Atlantic,^17,18 and a special edition of the British Medical Journal,¹⁹ have raised awareness among healthcare staff of the need to learn from failure. Such awareness, combined with the prevailing blame culture, may induce healthcare professionals to view covert reporting as the safe way to learn, sheltered from institutional and medico-legal repercussions.

This study sought to investigate the influence of two key factors, anonymity and profession (doctors versus nurses), on adverse event reporting.

METHODS

The adverse event reporting databases

Reports submitted to two different databases, which collect information from the same paediatric cardiac intensive care unit (CICU), were analysed and compared.

Database A (DA), the hospital’s official incident reporting system, is non-anonymous, and is predominantly used by nurses, only occasionally by doctors. Incident forms are filled in immediately after an adverse event is detected. There are no clear guidelines as to what needs to be reported and who should fill the reports. Forms are then submitted to the hospital’s Risk Management Team for analysis. The results of these analyses are summarised in quarterly reports that describe the frequency and severity of adverse events per ward/department.

Database B (DB) was developed by a consultant in cardiac intensive care (APG). In contrast to DA, DB is anonymous, and all adverse event reports are filled in at the end of the morning ward round by a consultant intensivist working in the paediatric CICU. Data collection is subjective, and based on the consultant’s direct experience of an adverse event or on information provided by members of the CICU team (other consultants, junior doctors, nurses, etc). To date, no detailed analysis of the information entered in this database has been carried out.

DA has been in use for many years. In DB, data collection began in April 1998. To allow a meaningful comparison of data from both sources, only those reports submitted between 1 April 1998 and 31 July 2001 were included in the analysis.

Incident type

Both databases classify adverse events into incident type (drug errors, ventilation, cannulae/indwelling lines, chest drains, blood transfusion, equipment, operational) (see Appendix for definitions) (fig 1).

Incident severity

Both databases also classify adverse events in terms of actual outcome severity (that is, the real consequences of the incident). There were notable differences in the actual outcome severity scale used in the two databases. However, to allow a uniform comparison of incident severity, severity scales were combined into a common scale, so that incidents were reclassified as follows: 0 = near misses, 1 = minor, 2 = major and 3 = life threatening consequences (fig 2). Incidents were reclassified by consensus of a team of two healthcare givers who are familiar with incident reporting issues but have no direct experience or involvement with the specific incidents.

Incident recovery

Data regarding incident recovery could not be compared because data regarding recovery were present only in DB and not in DA.

RESULTS

Between 1 April 1998 and 31 July 2001, a total of 211 adverse events involving 178 patients (11.87%) were reported among 1500 patients admitted to the paediatric CICU. Of these, 112 events had been reported in DA, 143 in DB, and 44 (20.85%) in both databases. Of the 112 reports made in DA, 9/112 (8%) were submitted by doctors, 14/112 (12.5%) by other staff (including ventilator technicians, laboratory managers, and perfusionists), and the majority, 89/112 (79.5%), by nurses. In DB all adverse events were reported by doctors, usually at consultant level.

Incident type

Figure 1 shows the frequency of incident types per database, together with the number of adverse events reported in both DA and DB. In DA the most frequent incident type was equipment (n = 31; 27.7%), followed by drug errors (n = 23; 20.5%), operational problems (n = 22; 19.6%), cannulae/indwelling lines (n = 12; 10.7%), ventilation (n = 11; 9.8%), chest drains (n = 4; 3.6%), and blood transfusion (n = 3; 2.7%). Six (5.4%) adverse events in DA were non-classifiable.

In DB the most frequent type of adverse event was cannulae/indwelling lines (n = 39; 27.3%), followed by drug errors (n = 25; 17.5%), equipment (n = 20; 14%), operational problems (n = 15; 10.5%), chest drains (n = 12; 8.4%), ventilation (n = 11; 7.7%), and blood transfusion (n = 4; 2.8%). Seventeen (11.8%) adverse events in DB were non-classifiable.

Incident severity

Figure 2 shows the frequency of incidents per level of severity. In DA, 15 incidents were classified as “no consequences” (15/112; 13.4%), 69 incidents were categorised as “minor consequences” (69/112; 61.6%), 24 as “major consequences” (24/112; 21.4%), and four as “life threatening consequences” (4/112; 3.6%).

In DB, 34 incidents were assessed as having “no consequences” (34/143; 23.8%), 51 as “minor consequences” (51/143; 35.7%), 55 as “major consequences” (55/143; 38.4%), and three as “life threatening consequences” (3/143; 2.1%). For those classified as “life threatening”, there was no overlap between the two databases (that is, the adverse events reported in DA did not coincide with those in DB).

Recovery per incident type

An analysis of incident recovery, using potential versus actual severity ratings from DB only, showed that rescue rates varied widely per type of adverse event: 23/25 (92%) “drug errors”, 3/4 (75%) “blood transfusion”, 6/11 (54.5%) “ventilation”, 9/15 (60%) “operational problems”, 18/39 (46.2%) “cannulae/indwelling lines”, 11/20 (55%) “equipment”, and 5/12 (41.7%) “chest drain” incidents were successfully recovered. None of the incidents resulted in fatal consequences.

DISCUSSION

There is widespread under-reporting of adverse events amongst healthcare staff.^{9,11,13,17,18} The impact of factors such as profession and anonymity on reporting rates remains unknown. Our comparison of DA and DB shows that they both gave a misleading picture of the frequency and severity of adverse events. Overall, the degree of concordance between the two databases was low, as only 44 (20.85%) adverse events were simultaneously reported by both databases.

Our study shows that the discrepancy between the two databases was obvious when data regarding incident type were analysed. Whereas DA identified “equipment problems” as the most common incident type, “cannulae/indwelling lines” was the most frequently reported in DB. Similarly, the lack of consistency between DA and DB is striking when “major” incidents are analysed: DA failed to detect 61.9% of the major adverse events which were otherwise identified solely by DB. DB missed a large proportion of “minor” incidents (51.4%). Most notably, there was complete lack of consistency and overlap between the two databases in respect to “life threatening” events.

Regarding incidents whose severity was classified as “none”, as previously noted this group includes incidents that are also referred to as “near misses”, or any event that could have resulted in negative consequences but did not. It is widely believed that these incidents occur much more frequently than actual errors or adverse events.^15,19 Unfortunately our study does not provide data on incidents that were not reported by either database, and therefore the proportion of “near misses” that escaped detection remains unknown. However, our data seem to suggest (fig 2) that these incidents were probably under-reported widely in both databases, and that the magnitude of under-reporting was especially notable in DA. While the reasons for this remain obscure, these findings could reflect a general tendency of the healthcare staff not to report incidents that are felt to be less critical, or incidents that did not actually occur but could have been associated with serious consequences if they had. The inability of both databases in detecting “near misses” is a concerning limitation, and it may indicate a trend in which an operational system consistently fails to learn proactively from opportunities and “free lessons”. As awareness of “near misses” may provide information as to the safety of the environment and as to how harm was avoided in specific circumstances, the failure of sharing near miss data may prevent the development of critical strategies to promote patient safety.

Our analysis of incident recovery was incomplete as data were provided only by one of the two databases and a comparison between the two systems was not feasible. Research in the aviation industry has shown that important safety lessons may be learnt by analysing error management (that is, how cockpit crews detect, manage, and recover from unsafe situations).^20,21 The omission of “potential outcome severity” data in the official hospital database (DA) is therefore a serious limitation because it precludes any analyses of recovery and error management.

Previous research has shown that healthcare organisations can improve the quality and safety of patient care by increasing adverse event reporting among staff.^14,17,18 Such studies point to the advances that have been made in improving safety in other high technology domains such as aviation and the nuclear industry as a result of robust reporting methods.^15,17,18 The Department of Health expert panel report An organisation with a memory concluded that healthcare staff should learn from critical incidents andnear misses in a non-punitive, blame-free culture.¹⁷ In order to emphasise its non-punitive and learning rationale of reporting, it should be voluntary, anonymous, and confidential.^11,15,17

In our study, a CICU consultant (APG) chose not to use the hospital’s official reporting system (DA) because it is non-anonymous and has been used punitively in the past. The end result is two databases containing widely divergent information collected from the same CICU over the same time period of time. Our study shows that organisational learning may be severely impeded when doctors become disillusioned with official reporting methods and engage in covert reporting. DB summarises the efforts of one clinician, in one unit of a hospital, and yet 46.9% (99/211) of the total number of adverse events, which would have been missed by the official reporting system, were detected solely by these efforts. Hence, there is massive data loss when doctors in CICU are not using a hospital’s official reporting process.

The extent to which these findings can be extrapolated to other medical disciplines remains speculative. Previous studies have shown that in general there is a higher incidence of critical incidents, near misses, and errors in ICUs compared to other wards in the same institution.^22,23 The magnitude of data loss observed in our study may therefore not be so great in other areas of the hospital. It is also open to question whether our results can be extended to other healthcare systems throughout the UK and abroad, although these trends may be widespread.

The results of our analysis also show a wide divergence in type and severity of adverse events reported by doctors and nurses. Although the reasons for this are unclear, one could speculate that their differing perceptions of which adverse events should be reported are a reflection of their differing roles in patient care. In this context, any drive to increase reporting rates only among nurses, for example, would invariably result in an increase in reporting of only some types of adverse events, with a sustained failure on the part of the institution to learn from the doctors’ perspective about others. Only an integrated and multidisciplinary approach to reporting, which involves healthcare staff at different levels, will lead to a thorough representation of the entire spectrum of medical incidents in a given environment, thus maximising learning. It should also be noted, however, that the two databases differ in that one is anonymous, and the other one is not. As a result, it may be misleading to ascribe the differences noted between the two reporting systems entirely to the fact that these systems are predominantly or exclusively utilised by different categories of healthcare professionals. Anonymity, or lack thereof, may have played a significant confounding role, the magnitude of which cannot be disentangled from that of the healthcare profession in this study.

In conclusion, our data show that incident reporting in a highly technological medical domain is a complex process that seems to be heavily influenced by the profession of those who report as well as anonymity of the reporting system. When adverse event reporting is based predominantly on the efforts of one set of professionals, and is not the result of an integrated multidisciplinary approach, the loss of data is significant and the information on frequency and severity of incidents are grossly inaccurate. Covert reporting creates an organisation with a fragmented memory and limits the ability to learn from past experience. Our investigation does not provide sufficient data to discern the relative impact of profession and anonymity in isolation. However, to ensure that data are representative, it is likely that reporting systems should be multidisciplinary and anonymous.

APPENDIX: CLASSIFICATION OF INCIDENT TYPE

1. Drug errors: Adverse events involving the administration of any drug (wrong drug, wrong timing, wrong patient, wrong dose, wrong route, etc).

2. Ventilation: Adverse events involving the establishment or management of mechanical ventilatory support, insertion and management of endotracheal tubes, etc.

3. Cannulae/indwelling lines: Relating to the insertion or management of central and peripheral, venous and arterial, catheters and lines, including arterial and venous lines used for extracorporeal membrane oxygenation (ECMO) support.

4. Chest drains: Adverse events involving the insertion and management of chest drains.

5. Blood transfusion: Adverse events involving the organisation and administration of blood and blood products.

6. Equipment: Failure or unavailability of medical devices used in the paediatric CICU.

7. Operational: Problems relating to the organisation and planning of any aspect of healthcare provision, including miscommunication between healthcare providers, delays in the delivery of scheduled treatment, and coordination problems between teams.

8. Non-categorisable: Resulting from factors other than the above.