Article Text

The cost-effectiveness of diagnostic management strategies for children with minor head injury
  1. M W Holmes,
  2. S Goodacre,
  3. M D Stevenson,
  4. A Pandor,
  5. A Pickering
  1. School of Health and Related Research, The University of Sheffield, Sheffield, England
  1. Correspondence to Michael Holmes, School of Health and Related Research, Regents Court, 30 Regent Street, Sheffield, S1 4DA, UK; m.w.holmes{at}shef.ac.uk

Abstract

Aim To estimate the cost-effectiveness of diagnostic management strategies for children with minor head injury and identify an optimal strategy.

Methods A probabilistic decision analysis model was developed to estimate the costs and quality-adjusted life years (QALYs) accrued by each of six potential management strategies for minor head injury, including a theoretical ‘zero option’ strategy of discharging all patients home without investigation. The model took a lifetime horizon and the perspective of the National Health Service.

Results The optimal strategy was based on the Children's Head injury Algorithm for the prediction of Important Clinical Events (CHALICE) rule, although the costs and outcomes associated with each strategy were broadly similar.

Conclusions Liberal use of CT scanning based on a high sensitivity decision rule is not only effective but also cost saving, with the CHALICE rule being the optimal strategy, although there is some uncertainty in the results. Incremental changes in the costs and QALYs are very small when all selective CT strategies are compared. The estimated cost of caring for patients with brain injury worsened by delayed treatment is very high compared with the cost of CT scanning. This analysis suggests that all hospitals receiving children with minor head injury should have unrestricted access to CT scanning for use in conjunction with evidence-based guidelines.

  • Accident & Emergency
  • Health Economics
  • Neurosurgery

Statistics from Altmetric.com

What is already known on this topic

  • The risk of missing patients requiring neurosurgery has been reduced due to increased CT scanning, but this raises concerns about increased radiation exposure and costs.

  • Clinical decision rules have been developed in an attempt to limit the use of CT scanning without increasing the risk of missed pathology.

What this study adds

  • This study confirms that CT scanning, as determined by a clinical decision rule, is a cost-effective use of healthcare resources.

  • This study demonstrates that using a decision rule to determine the need for a CT scan is more cost-effective than scanning all children.

  • This study shows that admission of children with a normal CT scan is not a cost-effective use of NHS resources.

Introduction

Background

Each year around 650 000 children attend hospitals in England and Wales with a history of a recent head injury and it is vital to identify those with significant intracranial injury.1 The risk of missing the pathology ofthese patients has been reduced due to increasing access to CT scanning,but this has raised concerns about increased radiation exposure. Clinical decision rules have been developed in an attempt to limit the use of CT scanning without increasing the risk of missed pathology.2–5 The cost-effectiveness of these decision rules has not been evaluated.

Aim of this investigation

The aim of this investigation was to determine the optimal existing CT scanning diagnostic strategy in terms of cost-effectiveness.

Materials and methods

Diagnostic accuracy of clinical decision rules

A systematic review of the literature was undertaken to evaluate the diagnostic performance of clinical decision rules.6 We only included decision rules where data were available to calculate the sensitivity of the rule for neurosurgical and non-neurosurgical lesions separately, and where there were sufficient numbers of neurosurgical lesions to provide a meaningful estimate of sensitivity. Table 1 shows the sensitivity and specificity of each strategy used as probabilities in the model and the decision nodes these apply to.

Table 1

Sensitivity and specificity of children's CT decision rules

The decision analysis model

We developed a decision analysis model using Simul8 software (Simul8 Corporation) to estimate the costs and quality-adjusted life-years (QALYs) gained for the following diagnostic strategies: discharge all patients without testing (theoretical), CT scan all patients (theoretical), Children's Head injury Algorithm for the prediction of Important Clinical Events (CHALICE),2 Paediatric Emergency Care Applied Research Network (PECARN),3 University of California-Davis4 (UCD) and the rule of Atabaki et al., 2008.5 The analysis was conducted for patients aged 1 and 10 years at index event. The model takes a lifetime horizon with mean life expectancy based on UK interim life tables, averaging the value for males and females.7 The economic perspective is the UK National Health Service (NHS) in England and Wales. A simplified schematic of the model is presented in figure 1. Each strategy was applied to a hypothetical cohort of 10 000 patients attending the emergency department with isolated, closed minor head injury (ie, with no other serious injuries). A proportion of the cohort was assumed to have an intracranial lesion requiring immediate neurosurgery (an intracranial haematoma requiring evacuation); a further proportion was assumed to have an intracranial lesion that did not initially require immediate neurosurgery. For convenience, we refer to the former patients as having a neurosurgical lesion and the latter as having a non-neurosurgical lesion, although it should be recognised that the latter may ultimately receive neurosurgical intervention. The remainder would have no intracranial haemorrhage. These proportions were estimated from Smits et al (neurosurgical injury 0.53%, 95% CI 0.33% to 0.85%; non-neurosurgical injury 7.1%, 95% CI 6.26% to 8.05%).8

We assumed that the decision rules would determine which patients underwent CT scanning and that the probability of detecting a neurosurgical lesion or a non-neurosurgical lesion was determined by the sensitivity of the rules for these lesions. We assumed that patients with a neurosurgical lesion detected on CT would be managed before any deterioration occurred, while those who did not have a CT had treatment after deterioration had occurred. We assumed that a proportion of patients with a non-neurosurgical lesion would deteriorate over the following 48 h and require critical care support and/or neurosurgery, while the remainder would remain well. These proportions were taken from a study by Fabbri et al.9 We assumed that patients without an intracranial lesion remained well and did not deteriorate. The proportion of these patients receiving unnecessary CT was determined by the specificity of each strategy.

Table 1 shows the sensitivity and specificity parameters of each strategy in correctly determining that a CT scan was required, predicated on the assumption that CT scanning has 100% sensitivity and 100% specificity in identifying significant intracranial lesions. Relevant lesions were those related to the head injury (ie, we did not consider incidental findings unrelated to the injury). The strategy of CT for all therefore had 100% sensitivity for both neurosurgical and non-neurosurgical lesions and 100% specificity. The strategy of discharge for all had zero sensitivity and 100% specificity.

Patient outcome

Patient outcome is measured by the Glasgow Outcome Score (GOS), defined as: GOS 1 Dead; GOS 2 Vegetative state; GOS 3 Severe disability; GOS 4 Moderate disability; GOS 5 Good recovery. The model designated each patient to a GOS category according to whether they had an intracranial lesion and how promptly it was treated. This involved estimating the probabilities that patients with neurosurgical and non-neurosurgical lesions would end up in each GOS category, depending upon the extent of treatment delay. For prompt treatment of patients with a neurosurgical injury, we meta-analysed the data from five studies identified from a systematic review of the literature.10–14

For delayed treatment of patients with a neurosurgical injury, we used Bayesian techniques to estimate GOS outcomes using a study by Haselberger et al, which showed the association between outcome and time delay from loss of consciousness to operation.14

For patients with a non-neurosurgical lesion where treatment is timely and appropriate, we based GOS outcomes on a study by Fabbri et al.9

No studies were found that reported the outcomes of delayed treatment on non-neurosurgical lesions; we therefore, assumed that outcomes were similar to those in delayed treatment of neurosurgical lesions and adjusted the GOS outcomes from Fabbri et al accordingly.9 The GOS outcomes can be seen in table 2.

Table 2

Clinical outcomes parameters and distributions used in the PSA

Costs

Costs included were: cost of investigation, CT scanning, hospital admission, neurosurgical treatment, intensive care and long-term nursing and rehabilitation. These were taken from the UK Department of Health National Schedule of Reference Costs 15 and the Personal Social Services Research Unit.16 Patients with outcome GOS 5 did not incur any further costs. The cost of GOS states 3 and 4 were taken from a study by Beecham.17 This study estimated costs for patients aged 18–25, and as cost data for younger people were not available, we assumed that costs are the same as for age 18–25. The above study did not provide cost data for patients in a vegetative state. We have therefore based our estimates on expert opinion (Sophie Duport, Royal Hospital for Neuro Disability, Personal communication. 8 May 2010). The cost of glioma was taken from the only relevant and reliable data source identified.25 Costs used in the model are shown in table 3.

Table 3

Costs used in the PSA

Quality of life utility values

A literature review was conducted to identify studies that estimated health-related utility values for GOS states. Two studies were found.19 ,20 The Smits et al study was considered to comply most closely with the National Institute for Health and Care Excellence (NICE) reference case and utility values from this study were therefore used.19 ,21

The Smits et al study did not report the age distribution of those patients used to estimate quality of life (QoL) utilities.19 We assumed that QoL for GOS 3 and GOS 4 is not age-related. We have also assumed that QoL for GOS 5 is not age-related. The QoL utility values for death and persistent vegetative state were assumed to be 0. Utilities used in the model are shown in table 4.

Table 4

Other parameters and distributions used in the PSA

Costs and utilities were discounted at an annual rate of 3.5% as recommended by the NICE guide to the methods of technology appraisal.21

Radiation risk

The estimated risk of a tumour from a single CT scan is taken from a study by Stein et al, which also provides an estimated QoL decrement risk per CT scan.22 The Stein et al study reports that the latency between radiation exposure and tumour diagnosis is over 5 years in the majority of cases.22 We have assumed a mean latency period of 10 years and the cost of cancer is therefore discounted for this time period. The values estimated by Stein et al and used in the model can be seen in table 4.

Persistent vegetative state

The Multi-Society Task Force on Persistent Vegetative State reported the mean length of survival for children in a vegetative state as 7.4 years.23 Patients in GOS 2 accrued the costs associated with a vegetative state for this length of time and were then assumed to have died.

Definition of cost-effectiveness terms

The incremental cost-effectiveness ratio (ICER) measures the relative value of two strategies and is the difference in mean costs divided by the difference in mean benefits. Where a strategy is less effective and more expensive than its comparator, it is dominated. Extended dominance occurs when a combination of two alternative strategies can produce the same QALYs as a chosen strategy, but at a lower cost.24 Strategies that are neither dominated nor extendedly dominated constitute the cost-effectiveness frontier, and the ICER is reported for these strategies compared with the next least effective strategy.

In a probabilistic sensitivity analysis (PSA), each parameter in the model is assigned a distribution; see tables 24. β distributions were used for probabilities as this distribution is bounded between 0 and 1; Gamma distributions were used for costs as the Gamma can describe the potential skewness of costs; Dirichlet distributions were used for GOS states as the states are correlated and must sum to 1; normal distributions were used for all other parameters. Results are presented as mean and incremental costs, QALYs and ICERs. A cost-effectiveness acceptability curve (CEAC) is calculated using the probability that a strategy has the highest net benefit (NB) for each iteration of the model for a range of willingness to pay (WTP) values.25 Net benefit is defined as: NB=WTP*QALY-cost.

Sensitivity analysis

Univariate sensitivity analysis was undertaken to explore the impact of changing each parameter in the model to its lowest CI, and then, highest CI; or if not available by an arbitrary lower value, then by an arbitrary higher value.

Probabilistic sensitivity analysis

PSA was undertaken to explore the impact of the joint uncertainty surrounding all model parameters. For each model iteration (1000 simulations each with 10 000 patients), each model parameter is randomly sampled from the relevant distribution. Monte Carlo sampling techniques were used to produce information on the likelihood that each intervention produces the greatest amount of NB. ICER is reported for strategies that constitute the cost-effectiveness frontier compared with the next least effective strategy.

Results

Deterministic analysis

For both ages, the CHALICE strategy had lower costs and gained more QALYs than the other strategies, and therefore, dominates the other strategies, although the absolute values for costs and QALYs were similar for all selective CT strategies.

Univariate sensitivity analysis

For both ages, no parameter change altered the optimal strategy decision. Discount rates were varied between 0% and 6% in accordance with the NICE methods guide; these rates had no effect on the optimal strategy decision for both ages.21

Probabilistic sensitivity analysis

Tables S5 and S6 show (see online supplementary files) for children aged 1 and 10 years at presentation, respectively, the mean PSA costs and QALYs per patient. Incremental costs and QALYs are not shown as the CHALICE strategy dominates all other strategies, although the selective CT strategies had broadly similar costs and QALYs.

The CEACs indicate that for children aged 1 and 10 years, the optimal management strategy is the CHALICE rule.2 For WTP thresholds between £0 and £50,000, the probability that this management strategy is cost-effective is 75% to 100% for children aged 1 year and 70% to 100% for children aged 10 years.

Conclusions

Our economic analysis confirms that the recent extension of access to CT scanning for minor head injury is appropriate. Liberal use of CT scanning based on a high-sensitivity decision rule is effective and cost saving. The cost of CT scanning is small compared with the estimated cost of caring for patients with brain injury worsened by delayed treatment. All hospitals receiving patients with minor head injury should have unrestricted access to CT scanning. The most cost-effective rule was the CHALICE criterion; however, incremental changes in the costs and QALYs are very small when all selective CT strategies are compared.

Discussion

Our economic analysis confirms that the use of CT scanning as determined by a clinical decision rule is a cost-effective use of healthcare resources for paediatric patients. This conclusion was also found in a similar study in adults.26

Selective CT use according to a clinical decision rule was also cost-effective compared with CT for all patients. This is because the clinical decision rules are all highly sensitive, so using CT for all patients resulted in a substantial increase in the number of normal CTs being performed for a small benefit in terms of additional cases detected. The disadvantages associated with increased radiation exposure offset the benefit of detecting a few extra cases, and the additional costs rendered the CT for all strategy more expensive than the selective strategies. Our conclusion that selective CT use is cost-effective compared with CT for all may not hold if the strategy used to select patients is not sufficiently sensitive.

Modelling is limited by the need to make assumptions and limitations in the available data. Our estimates of the effect of delayed treatment upon intracranial pathology, in particular, are based on very limited observational data.

We did not use age-related QoL utilities. However, it is likely that the QoL lost through the aging process will be proportionately comparable across all management strategies and the conclusions will be unaltered.

The model assumed that the various strategies are implemented in a consistent way. This may not be the case in practice and studies of the implementation of decision rules are required. In particular, the potential for knock-on consequences needs to be explored during implementation. For example, increased use of CT scanning may have unexpected consequences in terms of interpreting the clinical importance of abnormalities identified. Finally, economic analyses typically assume that resources can be redistributed between budgets, where appropriate, to fund cost-effective interventions. This may be restricted in practice, but economic analysis still has a valuable role in highlighting where increased funding in one area can reduce healthcare costs elsewhere.

The CHALICE decision rule can be seen in online supplementary appendix 1.

Acknowledgments

We would like to thank the following people for their help with this project: Professor Timothy J Coats, Academic Unit of Emergency Medicine, Leicester University, Leicester; Dr David Hughes, Department of Neuroradiology, Hope Hospital, Salford; Dr Fiona Lecky, Trauma Audit and Research Network, University of Manchester, Manchester; Mr Tim Pigott, Walton Centre for Neurology and Neurosurgery, Fazakerley, Liverpool; and Mr. Jake Timothy, Department of Neurosurgery, Leeds General Infirmary, Leeds for providing clinical expertise, and Professor Alex Sutton, Department of Health Sciences, University of Leicester, Leicester for providing statistical advice. Thanks also to Joanne Turner for clerical assistance.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Contributors MH was the lead modeller, taking primary responsibility for the construction of the economic model. MH had a substantial contribution to the design and construction of the model, the acquisition of data and analysis and interpretation of the results. MH was the lead author of the manuscript. SG was the lead investigator for the Health Technology Assessment. SG made a substantial contribution to the clinical design aspects of the model, the acquisition of data and interpretation of the results and drafting the manuscript. MS supervised the modelling work and also made a substantial contribution to the construction of the model, he also had a significant contribution in revising the manuscript. AP made a significant contribution to the acquisition of data, interpretation of the clinical effectiveness results and revising the manuscript. AP made a significant contribution to the acquisition of the data, interpretation of the clinical effectiveness results and revising the manuscript. All authors had final approval of the version to be published. MH takes responsibility for the paper as a whole.

  • Funding This project was commissioned by the NHS R&D HTA Programme Number 07/37/08. The views and opinions expressed are those of the authors and do not necessarily reflect those of the Department of Health.

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles