Article Text
PDF
Abstract
Aim To validate a descriptive tool for the causes of child death, which was designed to circumvent problems posed by the analysis of a confidential enquiry.
Method 3 participants from different healthcare backgrounds used clinical data, including the entries on the medical certificate of the cause of death, to classify the root cause of 783 deaths from the Confidential Enquiry into Maternal and Child Health child death review. A bespoke hierarchical system was used. Unanimity of allocation within categories and inter-rater and intra-rater agreement were assessed. Two methods for treating disagreements were compared by assessing their effect upon the apparent incidence of different causes of death.
Results The participants were most consistent in grouping deaths due to trauma, malignancy and sudden infant death. Each was highly consistent in allocating cases to groups (κ 0.85–0.99), but the agreement between participants, although “good”, was worse (κ 0.66–0.78). The greatest number of discrepancies was between diseases identified as congenital by the doctor and as chronic medical conditions by others. The method for treating disagreement between participants does not affect the commonest cause of death (trauma) but alters the ranking of the subordinate causes.
Conclusion Agreement within diagnostic categories might be improved by greater training of assessors in the use of the technique. This level of performance compares well with that of other coding systems upon their target groups.
Statistics from Altmetric.com
In a recent confidential enquiry into child death,1 medical certificates of the cause of death were frequently found to have been poorly completed. This finding is not new or unusual.2 3 Specifically, the multidisciplinary expert panels in the enquiry found the death certificate inaccurate in one-third of those they reviewed. In two-thirds of these, an additional diagnosis should have appeared on the death certificate, and in more than half of those, the missing diagnosis was a major, direct or overwhelming cause of death. At a panel review, there is access to more data than may have been available at the time that the death certificate was completed, for example, postmortem results4 5 and the results of coroners' inquiries. Therefore, the conclusions of the panel may differ in terms of the most significant cause of death.
When trying to group deaths for further comparison/analysis, the variety and complexity of childhood illness can be a significant impediment. Existing comprehensive and exact coding systems (such as Read,6 the International Classification of Diseases 10th Revision (ICD10),7 Snomed-CT and the ANZPICS coding system)8 are largely organ-system based and, when aggregated, tend to leave a large miscellaneous group (see table 3 of the accompanying paper) that confounds further analysis.
Frequency of cause of death using the majority verdict when there was partial agreement or disagreement between reviewers
What is known on this topic
▶ Existing ‘organ system based’ disease coding and classification systems when applied to causes of child death leave a large miscellaneous group.
▶ The cause of death given on medical certificates of the cause of death is not comprehensive and is often inaccurate.
▶ Mortality data from the office for national statistics is based upon death certificates.
What this paper adds
▶ Local safeguarding children boards use a new hierarchical system for classifying child deaths which takes into account social priorities such as suicide and homicide as well as medical descriptions.
▶ Classifications made using the new system are reproducible within and between observers.
▶ The new classification system leaves no miscellaneous group.
Hence, in the course of performing the child death review, we developed a descriptive tool to categorise child deaths (shown in box 1), which has since been adopted by the Department for Children Schools and Families for use by child death overview panels.9 In an attempt at validation of the approach, using this tool, we assessed the frequency of agreement within the categories of the system and the agreement between different participants.
Box 1 Each death is assigned to the highest possible category, this providing the most useful classification of the death
| Category | Name and description of category |
|---|---|
| 1 |
|
| 2 |
|
| 3 |
|
| 4 |
|
| 5 |
|
| 6 |
|
| 7 |
|
| 8 |
|
| 9 |
|
| 10 |
|
Method
An intuitive hierarchical system (box 1) comparable with those described by Wigglesworth10 and Alberman et al11 was used, which in theory:
▶ enabled the distinction of clinically important discrete groups
▶ highlighted social and political priorities such as suicide and homicide
▶ allowed for the ability of congenital or perinatal disease to cause or contribute to death in later childhood.
To test the system, we used diagnostic information from the CEMACH child death enquiry described in the accompanying paper. Death certificate data and data from additional text fields such as the suspected cause of death (used before issuing a death certificate) were selected from the records of the confidential enquiry and entered onto a spreadsheet. Independent participants (a doctor, a regional enquiry manager and an experienced medical data analyst) then used the system to categorise the data set twice. The second attempt was upon a randomly reordered version of the same dataset. Where cases had been subject to panel enquiry, looking for avoidable factors, this information was withheld from the assessors. In order to classify the death, the reviewers were given the instructions reproduced in box 1 and the instruction to classify the most useful cause of death taking into consideration the root cause. To expand upon this instruction, when a child dies, the final mechanism can usually be described as respiratory failure or cardiovascular failure; however, these are not particularly helpful terms to enter on a death certificate. For example, respiratory failure is a final common path to death in patients with severe degenerative neurological disease. The immediate cause of death may be pneumonia or aspiration, but the underlying cause is the neurological disease, which itself may have a deeper root cause.
The selection of categories was by a drop down menu within the spreadsheet. Unanimity within categories of assessment is presented as a percentage. Inter-rater and intra-rater agreement between the participants were assessed using the κ statistic. Disagreements were treated first by allocating the highest category according to the hierarchy and second by selecting the majority verdict (when two reviewers agreed). The impact of such disagreements is presented by comparing the effect of these strategies upon the apparent incidence of different causes of death.
Results
Of the 957 child deaths in the enquiry, 783 contained sufficient data to allow classification. Table 1 shows the proportion of cases where there was complete (all three reviewers), partial (two of three) and no (three different categories) agreement between the reviewers for each cause of death. Reviewers were most consistent in grouping deaths due to trauma, malignancy and sudden infant death. When faced with disagreement, the death was assigned to the highest ranking group according to the hierarchy in box 1 (table 2) or the commonest assignment (table 3). Comparison of tables 2 and 3 shows that the effect of the way in which disagreements between reviewers were treated was sufficient to change the order in tables of the causes of child death. All the reviewers found trauma to be the dominant single category overall, but this cause is heavily skewed towards older children. There was greater agreement between reviewers (but still significant variation) when the deaths of patients under one were separated.
Agreement between reviewers upon classification by cause of death
Frequency of cause of death using the hierarchy in box 1 when there was partial agreement or disagreement between reviewers
For example, when the cause of death was assigned as sudden infant death, in 86% of cases, this was on the basis of full agreement between the reviewers. In 13%, there was partial agreement (2:1), and in 1%, there was no agreement between the three reviewers.
The assessment of intra-rater agreement (between classifications of each case by the same reviewer) is summarised in table 4, and inter-rater agreement (ie, that between reviewers for each case) in table 5. When interpreting κ, values 0.61–0.8 represent “good” agreement, and those above 0.81 are “very good”. Hence, each reviewer was highly consistent in allocating cases to groups, but the agreement between the reviewers, although good, was worse.
Intra-rater comparisons
Inter-rater comparisons
Examination of tables 1–3 and also the κ tables in each case showed by far the greatest number of discrepancies to be between diseases identified as congenital by the doctor and chronic medical conditions by others.
Discussion
In this study, when given more information about each case, intelligent coders were consistent in their interpretation of the root cause of death, but their perspectives differed to an extent that significantly affected the ranking of the different categories. Many genetic and congenital conditions manifest as chronic medical conditions before causing death, and a degree of medical knowledge may confuse or clarify the distinction. Chronic conditions sit higher in the hierarchy but are stated specifically to include cystic fibrosis and other illnesses such as immune deficiencies, which a medic might recognise to have a genetic root cause. Such differences invite reflection as to how well the ratings reflect the truth that we are trying to define and whether the right method of analysis has been used.
Nevertheless, the root cause of death should be a discrete variable and a limited number of groups analysable by the κ statistic. The levels of agreement achieved in this study are comparable with those of the Intensive Care National Audit and Research Centre coding method, which is another hierarchical system that has been analysed using κ.12 In coding systems with larger numbers of categories, there are inherently more opportunities for disagreement. When comparing the ICD9 and ICD10 systems (which are enormous by comparison to ours), Quan et al13 chose to compare indices that can be derived from a decision matrix (sensitivity, specificity and positive and negative predictive power) for each of 32 selected diagnoses. They found the systems comparable but reported an extreme range of results with sensitivities as low as 9% in some cases, probably reflecting the different prevalence of their choices.
We have shown that a relatively simple hierarchical classification of causes of death can be used with reasonable agreement between classifiers. However, we recognise that no system can altogether avoid ambiguity because the very concept of causation is complex. For example:
A sixteen year-old unmarried primigravida from social class V has an antepartum haemorrhage at 28 weeks' gestation followed by spontaneous labour and the premature delivery of an infant who rapidly develops respiratory distress syndrome and dies on the second postnatal day. The postmortem examination demonstrates hyaline membrane disease and intraventricular haemorrhage….
The paediatricians will see death in terms of pathophysiology. The obstetricians will consider that the infant died from maternal antepartum haemorrhage with its predictable complications. The pathologist tends to be impressed more by the final event than the initial one and will attribute death to the intraventricular haemorrhage. The epidemiologist may observe that the pregnancy could have had a different result had not the girl had such an unpromising social background. The girl's mother, with some truth and equal conviction, will see the boyfriend as the cause of the whole unhappy episode (source unknown).
As demonstrated by this vignette, the root cause of a child's death can be inherently difficult to define, depends upon the perspective of the observer and may include a complex interaction of multiple factors. Broadly speaking, deaths can result from:
▶ a set of contributory factors acting more or less simultaneously, where it is hard to give precedence to one cause over another
▶ a sequence of events where one thing leads inexorably to another
▶ a mixture of these two as in the example above.
The scenario could be extended such that the child survived with quadriplegic cerebral palsy, dying at the age of 15 with respiratory failure due to chronic pulmonary infection with recurrent aspiration, yet the root causes of that death would be the same.
The practitioner completing a medical certificate of the cause of death or a coroner at inquest has to come to a conclusion within the constraints of the certification system. The neonatal medical certificate allows for a more contributory model than the medical certificate for all other deaths, and the coroner has the option of open or narrative verdicts. Nevertheless, the cause of death will ultimately be coded for national statistics according to the international classification and neither this nor any other classification based on organ systems captures the causes of child death in ways that make sense from the point of view of public policy or in relation to prevention. There was, therefore, a need for a simple, robust and intuitive system for use with the new child death review procedures. Other systems have been described,14 but none crossed the entire age range of children or classified the deaths with the potential advantages of this method.
One of the concerns about classifying causes of death such as that reported by Taylor and Emery14 is that too much weight can be given to the proximate rather than the root causes of death. The hierarchical system should counter such an effect because the ranking is predefined and explicit. Hence, the method of dealing with disagreement between observers where the highest ranking cause from box 1 is used is to be preferred. This approach, combined with the limited number of categories, helps promote consistency.
Where relevant, we withheld information from panel discussions about avoidable factors in relation to the death from the assessors categorising the cause of death. This was because only a subset of cases was considered by panels. Furthermore, in the CEMACH study, avoidable factors were only determined once for each case. Hence, we could not perform an analysis of agreement in this respect. The question as to how reproducible panel conclusions are is frequently aimed at confidential enquiries. To this end, it is intended to have some cases evaluated by more than one panel in the current CEMACH enquiry into head injury in children. In the accompanying paper, we have referred to studies where avoidability has been generically assigned to diagnostic codes. We avoided this approach primarily because the existing coding systems leave such a large proportion of cases as miscellaneous.
We considered how the methods in this study could be changed to improve agreement. The discrepancies between coders may in part be explained by insufficient training in the method. Also the fact that the two categories most commonly interchanged were adjacent to each other on the computer's drop down list may be relevant, but the differences do seem more to represent a difference in interpretation of the root cause of death. Furthermore, in some instances, the number of cases in the groups was so similar that their relative positions in a frequency table should really be treated as equal. The differences may become less significant when more cases or more coders are involved. In the future, uniformity in approach might also be increased by formal training in the use of the system or using coders from similar health service backgrounds, especially because nosologists have been reported to achieve higher levels of agreement in an arguably less diverse population.15
Conclusion
This hierarchical method of classifying causes of child death eliminates the problem of generating a large miscellaneous group. The reproducibility of results compares well with that of other coding systems upon their target groups. Agreement within diagnostic categories might be improved by greater training of assessors in the use of the technique.
Acknowledgments
Marjorie Renwick and John Stickley classified the deaths (twice), as did the first author. Amy Sullivan (data analyst at the Centre for Maternal and Child Enquiries) checked the calculations.
References
Footnotes
-
Funding Funding for the national confidential enquiry on which this article is based was provided by the National Patient Safety Agency and by the Department of Health, Social Services and Public Safety of Northern Ireland. The views of the authors are their own and do not necessarily represent those of the funding bodies.
-
Competing interests None.
-
Ethics approval This study was conducted with the approval of the NW Multicentre Ethics Committee 05/mre08/51.
-
Provenance and peer review Not commissioned; externally peer reviewed.