Survival after surgery or therapeutic catheterisation for congenital heart disease in children in the United Kingdom: analysis of the central cardiac audit database for 2000-1

Data collection

We designed a minimum dataset of 20 fields with the simple aims of the project in mind. All 13 congenital heart disease centres in England, Scotland, and Northern Ireland participated. To ensure patient confidentiality the central cardiac audit database employs advanced data encryption technology to control access to data through a secure key system. We used lists with fixed choices consisting of all but the rarest and most complex combinations of diagnoses and procedures to minimise the potential complexities of diagnostic and procedural coding for congenital heart disease.

Data validation

The minimum dataset included date of death, but we linked with the Office for National Statistics by using NHS numbers to assess mortality wherever possible. We compared volunteered mortality data with centrally tracked data. In Northern Ireland and Scotland we used the general register offices to track mortality centrally.

The central cardiac audit database includes other forms of independent data validation carried out by visiting centres, when two weeks' submitted data, chosen at random, are compared with hospitals' written medical records, with operating theatres' records, and with laboratory records on cardiac catheterisation. We checked entries in the logbooks for operating theatres and catheter laboratories for the entire year in each centre, to ensure complete ascertainment of caseload. We also compared submitted data with nationally held hospital episode statistics whenever these were accessible.

Data analysis

We used the online Lotus Domino version of the central cardiac audit database to collect data and transferred these for analysis to SPSS 10.0 for Microsoft Windows. We used Wilson's score method to calculate confidence intervals for survival.1 2 We used 99% confidence intervals (table) to make allowances for the high number of multiple comparisons, to minimise false positive results. We did not consider an individual survival value to be significantly different from the mean if the confidence intervals overlapped the mean.

We used analysis of survival after “benchmark” operations to compare results from different centres, to eliminate the effect of different case mix. We chose six benchmark procedures for surgery (repairs of atrial septal defect, ventricular septal defect, atrioventricular septal defect, tetralogy of Fallot, simple transposition of the great arteries, and coarctation) and three for therapeutic catheterisation (atrial septal defect closure, arterial duct closure, and pulmonary balloon valvoplasty). We did not undertake detailed risk stratification as no validated method exists.

We calculated 30 day postoperative survival to facilitate comparison with results from previous UK registry data and to comply with practice in the United States.3 Central tracking of mortality has, however, also allowed us to plot one year survival curves, in contrast to previous registry analyses. We included foreign nationals but censored them from survival analysis after the perioperative period unless specific follow up data were available (central tracking was not possible for this group). We analysed individual operators' results anonymously, but each centre agreed to be identified. We have not calculated freedom from reintervention statistics for this report as follow up is currently too short to allow meaningful interpretation of results.

Data collection and quality

Overall completeness of the dataset was 96.8%, with completeness for individual data fields ranging from 75% (for NHS number) to 100%. Data were received for a total of 5494 procedures, of which 3666 were surgical and 1828 were therapeutic catheterisations.

We have reported all cause mortality, choosing not to attempt detailed investigation of cause of death and its relation to treatment. We found substantial differences in volunteered and centrally tracked mortality, with seven of 11 centres in England under-reporting death within 30 days. Central tracking of mortality identified 469 deaths, 194 occurring within 30 days and 275 later. Forty two of the 194 deaths within 30 days were detected by central tracking but were not in the volunteered data. Nineteen of these patients were discharged alive but subsequently died within 30 days of the operation. The remainder had been incorrectly coded as alive at discharge; using reported discharge status would have underestimated 30 day mortality by 22%. Data on hospital episode statistics were available for 2716 patients and under-reported death within 30 days by 9%, classifying 1% of surviving patients as deceased. Hospital episode statistics data also under-reported the total number of procedures by 10%. During validation visits we found a total of 143 procedures to be missing from the data submissions to the central cardiac audit database, predominantly related to systematic errors in data collection. The visits resulted in submission of missing or revised data from all of the 13 centres.

Survival

Figures 1, 2, 3 show national survival curves after cardiopulmonary bypass surgery, non-cardiopulmonary bypass surgery or therapeutic catheterisation. The table shows survival at 30 days and one year after all procedures. We assessed benchmark procedure survival anonymously for individual operators (41 surgeons and 63 cardiologists) as well as for different centres. No significant difference from the national mean survival was detectable for any individual. Figures 4 and 5 and figures A-I (see bmj.com), respectively, show individual centre's survival data for pooled and individual benchmark procedures.

Fig 1

One year survival after cardiopulmonary bypass surgery for age groups <1 month, 1 month-1 year, and 1-16 years for the United Kingdom

• Download figure
• Open in new tab
• Download powerpoint

Fig 2

One year survival for non cardiopulmonary bypass surgery for age groups <1 month, 1 month-1 year, and 1-16 years for the United Kingdom

• Download figure
• Open in new tab
• Download powerpoint

Fig 3

One year survival for therapeutic catheterisation for age groups <1 month, 1 month-1 year, and 1-16 years for the United Kingdom

• Download figure
• Open in new tab
• Download powerpoint

Fig 4

Survival at 30 days and at one year reported by individual centre, with 99% confidence intervals for all benchmark surgical procedures. The shaded areas represent the national means with 99% confidence intervals. If a centre's confidence intervals overlap the shaded area their survival does not differ statistically from the national mean. For a list of the abbreviations see bmj.com

• Download figure
• Open in new tab
• Download powerpoint

Fig 5

Survival at 30 days and at one year reported by individual centre, with 99% confidence intervals for all benchmark catheter procedures. The shaded areas represent the national means with 99% confidence intervals. If a centre's confidence intervals overlap the shaded area their survival does not differ statistically from the national mean. For a list of the abbreviations see bmj.com

• Download figure
• Open in new tab
• Download powerpoint

Data quality

We found a striking difference in deaths identified by centres' own records and by central tracking. This was most marked when death occurred after the perioperative period, but deaths were missing from submitted data even when death occurred within 30 days. It seems inevitable that previous registry reports (including the register of the Society of Cardiothoracic Surgeons, used in the Bristol inquiry) that have relied on voluntary reporting of death have also under-reported mortality, casting doubt on their validity. The introduction of the NHS number as a unique and permanent patient identifier and the ability to establish electronic linkage with the Office for National Statistics is a major advance in tracking patients' outcomes.

Risk stratification

The use of benchmark procedures minimises the effect of varying case mix for the purposes of comparison of outcomes in different centres. Attempts have been made to establish a consensus view of risk assessment in the United States4 and case complexity in the United Kingdom.5 These protocols, applied to the data in the central cardiac audit database that have been accumulated over several years, should facilitate development and validation of risk stratification for the future.

Patient confidentiality

We did not include patients' consent for data submission to the central cardiac audit database in our protocol. Our current understanding of the Data Protection Act 1998 is that patients' consent is not required if anonymised data are used for the purpose of research or audit. The exception to anonymising data is the NHS number, which we have protected by encryption with a key held only by the data managers and used only for record linkage. In the United Kingdom we have an almost unique opportunity to carry out effective and believable national audit because we have a single healthcare system with an Office for National Statistics where the life status of an individual patient, based on their NHS number, is known.

With appropriate precautions central tracking is possible while maintaining patient confidentiality.

Diagnostic and procedure coding

Several groups have devised coding systems for congenital heart disease.3 6 7 The central cardiac audit database plans to adopt the coding system of the Association for European Paediatric Cardiology7 from 2004, to facilitate future international compatibility of data.

Outcomes after treatment

Centre specific data analysis shows that quality of treatment is high throughout the United Kingdom; no centre and no individual operator has detectably different survival from the national mean after benchmark procedures. We have been unable to assess accurately how our results compare with those of other nations, although our data seem to compare favourably with unvalidated registry reports from North America and Europe.

We believe that this study is the first to present validated, centre specific survival data for nationwide treatment of congenital heart disease. Population based, 45 year, actuarial survival has been reported for the whole of Finland,8 but individual centres' performance was not included. Most previous reports have concentrated on multicentre, mean perioperative mortalities (defined as death within 30 days of operation),9 and a similar approach was used for the register of the Society of Cardiothoracic Surgeons.10 Although this simplistic approach may be convenient, our data show how misleading 30 day results may be as a descriptor of overall outcome.

We calculated 99% confidence intervals for the purpose of assessing survival in different centres. Even at this level, having performed a total of 178 comparisons, we think that it is likely that we will generate spuriously significant results: we calculate that the chance of at least one spuriously significant difference in survival is 83% (this would have been 99.99% had we used 95% confidence intervals).

This early data analysis has concentrated on survival, which is a crude indicator of overall performance. For smaller centres, as well as for individual operators, analysis of a single year's data has limited power to detect underperformance by institutions or individual operators and year on year analysis will be necessary to provide more robust reassurance. Freedom from reintervention is likely to be a powerful indicator of overall performance, but several years' data will be required before our capability to track reintervention can be put to use.