Article Text

PDF

Trends in paediatric health economic evaluation: 1980 to 1999
  1. W J Ungar1,2,
  2. M T Santos1
  1. 1Department of Population Health Sciences, The Hospital for Sick Children, Toronto, Canada
  2. 2Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
  1. Correspondence to:
    Dr W Ungar
    Scientist, Population Health Sciences, Hospital for Sick Children Research Institute, 555 University Avenue, Toronto, Ontario, Canada M5G 1X8; wendy.ungarsickkids.ca

Abstract

Background: Although standard methods for conducting economic evaluations have evolved, little attention has been paid to their application in paediatrics. The Paediatric Economic Database Evaluation (PEDE) Project was conceived to promote research into paediatric health economic methods.

Aim: To examine trends in paediatric economic evaluation between 1980 and 1999.

Methods: A comprehensive literature database created for the PEDE project was the source of the data. Descriptive statistics were used to summarise trends. Publication volume, study outcome category, analytical technique, and journal type were examined over the study period.

Results: The literature search resulted in 787 full paediatric economic evaluations. The volume of publications increased from 61 to 440 citations per 5 year period. The most common health outcome category was cases of disease/condition/abnormality. Cost-effectiveness analysis (CEA) was the most common technique used, accounting for a majority of evaluations in all time periods. The proportion of studies using CEA increased by 23 percentage points, while the proportion using cost–benefit analysis decreased from 31% in 1980–84 to 12% in 1995–99. Cost-utility analysis was the least common analytical technique. Publication in journals of paediatrics/perinatal medicine was the most common venue for all intervals and increased as a proportion of the total over time.

Conclusions: The growth in publication of paediatric economic evaluations suggests that increasing attention should be paid to the application of health economic methods to a paediatric population to ensure high quality allocation decisions.

  • economic evaluation
  • publication
  • paediatrics
  • trends
  • CBA, cost–benefit analysis
  • CEA, cost-effectiveness analysis
  • CMA, cost-minimisation analysis
  • CUA, cost-utility analysis
  • QALY, quality adjusted life year

Statistics from Altmetric.com

In an era of healthcare delivery that is increasingly constrained by economic considerations, decision makers require high quality economic evaluations of interventions, services, and technology to facilitate allocation decisions. While standard methods for the conduct of economic evaluations have evolved in recent years, especially for pharmaceutical interventions,1,2 the validity of applying these methods to a paediatric population has yet to be examined. Child health differs from adult health in important ways, including the physiological vulnerability of children as they grow and develop, their reliance on parents, teachers, and others to provide access to healthcare, and the unique ways in which they manifest disease and interact with the healthcare system and their immediate environment.3–7 These differences translate into unique challenges for conducting health economic assessment in children. The reliance on parental proxy reporting, the assessment of productivity costs of parents and caregivers, the measurement of future lost productivity of children, the inclusion of community and school based resources devoted to health, the determination of utility for calculating quality adjusted life years, and the assessment of willingness to pay are examples of methodological issues that require further study.

The Paediatric Economic Database Evaluation (PEDE) Project was conceived to study the application of health economic methods to a paediatric population by revealing gaps in knowledge and methodology through a close examination of the literature. The ultimate goal of the project is to develop a framework for research into health economic methods in the paediatric population.8,9 The PEDE Project consists of several phases. In Phase 1, a search strategy was developed and a comprehensive database of paediatric economic evaluations published between 1980 to 1999 was created. Various steps in the search strategy were subjected to validity and reliability assessment, and the PEDE database was compared to other similar databases.10 The objective of Phase 2, reported here, was to examine trends in health economic methods over the 20 year period from 1980 to 1999. Phase 3 involved the development of a valid and reliable instrument for appraising the quality of published paediatric economic evaluations,11 and Phase 4 consisted of a quality appraisal of the literature.

METHODS

Full details of the development of the database, including search strategy, sources, and reliability assessment of the citation selection process, are reported elsewhere.10 A list of website addresses scanned for this study can be found in Appendix 2 of the Technical Report.12

According to Drummond et al, an economic evaluation is defined as “the comparative analysis of alternative courses of action in terms of both their costs and consequences”.13 For the PEDE Project, a publication was accepted as an economic evaluation only when a comparator existed and descriptions of both costs and health outcomes were present. The economic evaluation did not have to be the primary objective of the study to be eligible. The inclusion criteria required that the study contained original analysis and included the evaluation of an intervention such as a medical or surgical treatment, a programme, a service, or a new process; that the intervention was directed at the paediatric population including neonates, infants, children, or adolescents less than 19 years; that if interventions were aimed at pregnant women or mothers, outcomes were measured in offspring; that the comparator was either real or implied, such as in a pre- and post-intervention, or a ‘do nothing’ or ‘usual care’ approach; that a health outcome, intermediate or final, was reported; and that costs were measured and reported. Randomised controlled trials and observational studies were eligible, as were modelling studies and meta-analyses if they included novel data aggregation and new analyses. Studies were excluded if they were cost of illness or cost of prevention studies, if a specific intervention was not evaluated, or if they assessed family planning interventions related to birth control that included outcomes such as couple year of protection or number of averted births; however, if the intervention was related to perinatal screening and the outcome was the number of averted births with a malformation or disease, the study was eligible. Interventions consisting of a guideline, a continuous quality improvement process, or a new operating procedure or policy targeted toward improving practice or efficiency were excluded, as were studies where costs were not quantified. To restrict the database to original, full economic evaluations, abstracts from conference proceedings, methodological papers, papers without original analyses, policy statements, case studies or reports, letters, editorials, or notes were excluded.

The application of the inclusion and exclusion criteria resulted in 787 eligible citations between 1908–1999. The following data were collected from each eligible citation: journal, target population, ICD-9-CM disease class,14 age group, intervention being studied, primary outcome measure, and health economic analytical technique, including cost-effectiveness analysis (CEA), cost-benefit analysis (CBA), cost-utility analysis (CUA), and cost-minimisation analysis (CMA). Where more than one analytical technique was used, a primary technique was designated by the abstractor using the following order of importance: CUA>CBA>CEA>CMA. All interventions described in the citations were categorised as either prevention, treatment, programme (organisation or organisational unit, clinic, department, or health system), surgical, educational, healthcare delivery (process, service, tool, or treatment pathway), detection (tests used to screen for disease or the potential of developing disease), diagnosis (the use of clinical tests to confirm the cause of illness), or psychological. Data entry and data management were performed using Microsoft Access 97. In addition, the citation identification number was used to link the Access database record to its full citation, including MeSH terms and abstract, in an EndNote (version 3.0.1) bibliographic database.

Trends in the paediatric health economic evaluation methodology were investigated by performing one way frequency distributions and two way cross tabulations on the variables in the database, including publication year, outcome category, analytical technique, and journal type.

RESULTS

Volume of publications

While the annual number of publications was fairly constant from 1980 to 1989, the volume of publication grew dramatically during the period of 1990 to 1999. The volume and growth by 5 year period is displayed in table 1. The average 5 year growth rate was 96%. Over half of the papers in the database (440) were published between 1995 and 1999.

Table 1

Publications per 5 year period

Outcome measures

As several hundred different outcome measures were reported in the literature, these were grouped into categories as indicated in table 2. The most common category of outcome measure over the study period remained observed cases of disease/condition/abnormality. The proportions of papers studying cases of death and cases of cures/improvements/healing declined slightly over time, while changes in physiological measures and cases of complications/adverse events became more common as outcome measures.

Table 2

Publications per outcome category by 5 year period

A cross tabulation of intervention type by outcome category demonstrated that 58% of studies of health prevention interventions used an outcome categorised as cases of disease/condition/abnormality. Studies of health treatments commonly used cases of disease/condition/abnormality and cases of cures/improvements as outcome measures. Of the economic evaluations of paediatric health programmes, 74% were fairly evenly distributed across the outcome categories of cases of disease/condition/abnormality, cases of death and changes in physiological measures.

Analytical technique

Depending on the way health outcomes are measured and reported, economic evaluations are typically classified as a CEA, CUA, CBA, or CMA.13 In CEA, consequences are measured in natural units, such as cases of disease detected or life years gained. In CBA, the outcome is monetarised, so that costs associated with resource use and administration of the intervention are subtracted from the monetary value of the health benefit achieved. In CUA, the outcome, traditionally life years gained, is weighted by the patient’s preference for a particular health state, resulting in a quality adjusted life year (QALY). In CMA, the outcomes associated with the intervention and the comparator are comparable and only costs are considered. Marked changes in the type of analytical technique used over time was evident, as seen in table 3. While CEA was the most common technique used, accounting for the majority of evaluations in all time intervals, the proportion of studies using CEA increased by 23 percentage points while the proportion of studies using CBA decreased from 31% to 12% between 1980–84 and 1995–99. CUA remained the least common analytical technique used over all time intervals.

Table 3

Publications per analytical technique by 5 year period

A cross tabulation of intervention type by analytical technique indicated that CEA was the most common technique for all intervention types. CBA tended to be used to evaluate health prevention and detection/diagnostic interventions, while CUA was usually applied to evaluations of health prevention interventions and health treatments.

Journal type

As economic evaluations typically represent multi-disciplinary research, the choice of publication venue is broad. Publication of economic evaluations by journal type is presented in table 4. Publication in journals of paediatrics/perinatal medicine was the most common venue for all time intervals and increased as a proportion of the total over time. Paediatric economic evaluations appearing in other sub-speciality journals also increased over time, while the proportion of publications in public health and general medicine journals decreased over time. There was also an increase in the proportion of publications of paediatric economic evaluations in journals devoted to health economics/health policy/methods.

Table 4

Publications per journal type by 5 year period

A cross tabulation of journal type by analytical technique demonstrated that while CEAs tended to be concentrated in paediatrics/perinatal medicine journals, CBAs and CUAs were more evenly distributed across journals of paediatrics/perinatal medicine, sub-speciality medicine, public health and general medicine.

DISCUSSION

Just as the rate of adult economic evaluations continues to grow rapidly,15,16 the number of paediatric studies published every year is rising. This growth reflects the importance of economic evaluations to assess the economic benefit of interventions, programmes, and services for children in a healthcare climate characterised by economic constraints and difficult allocation decisions. Significant growth in the number of paediatric economic evaluations was observed over the 20 year study period, with the number of publications approximately doubling every 5 years.

By definition, economic evaluations represent a balance of costs against consequences, with health consequences expressed as a particular type of outcome measure. Outcome measures that examined the number of cases of a disease, a condition, or an abnormality were the most frequent during the study period and are consistent with the preponderance of health prevention interventions. These measures represent intermediate outcomes. More final outcomes would include mortality, life years gained, and QALYs, and are preferable for allocation decision making.18 While the use of final outcomes represented only 20% of the total, they did demonstrate a sustained growth in frequency over the study period. Final outcome measures are particularly problematic for the paediatric population, unless the study is focused on a terminal disease of childhood such as a severe musculoskeletal disorder or cancer. Currently no published health economic models exist for measuring life years gained over long time horizons that include periods of maturation, development, and rapid physiological change.

A challenge in building the PEDE database was assigning an analytical technique. Given the high frequency of evaluations of preventive measures for infectious disease, many of the studies were self labelled as CBAs. In addition, the early publications pre-dated the widespread acceptance of the conventional analytical techniques defined as CEA, CBA, CUA, and CMA. These early studies tended to be self labelled as CBAs regardless of the actual analytical technique used. To circumvent the problem of mislabelled studies, the research team ignored the authors’ label and assigned one based on a careful reading of the methods used. Thus, many so-called CBAs that examined the incremental cost of cases prevented were relabelled as CEAs. However, if a so-called CBA monetarised the outcome measure (even if it did so poorly), the study remained a CBA. Our results indicated that true CEAs accounted for a large majority of paediatric health economic evaluations, a finding similar to the adult literature.17 CUAs were rare, even in 1995–99. This is probably due to the difficulty in ascertaining life years gained, as discussed above, as well as the challenge of estimating utilities for health states in children. As paediatric economic evaluation becomes more prevalent, a deeper understanding of the methodology should improve the author’s specification of the analytical technique. Otherwise, persistent misuse of the terms CBA and CEA will continue to hamper the conduct of literature searches for the purpose of methodological research in this field.18

Examining the venue chosen for journal publication can reveal the motivation behind the research, for example to influence clinical decision-making, to advance the methodology in the field, or to influence policy. The increasing rate of publication in paediatric and sub-speciality journals suggests that there is a growing appreciation of the importance of economic evaluation in paediatric medicine among clinical specialists. However, publication in a medical journal alone may be inadequate. As an applied form of research, the results of health economic evaluation must be disseminated in a manner that will best aid policy decision making. While publication in a highly regarded international speciality journal may be desirable from a clinician’s or academician’s perspective, it may fail to reach the responsible decision makers in regional governments or health organisations unless accompanied by a targeted dissemination strategy.

A particular challenge in identifying paediatric economic evaluations of healthcare interventions relates to the way ‘health’ is defined. In the paediatric population, health is intricately connected to behaviour and development. Many health problems in children are manifested as behavioural changes for which social services and educational interventions are advocated. Multiple care settings are involved in the delivery of care to children. These include physicians’ offices, clinics, schools, homes, and community agencies. Our focus on scanning only the health literature resulted in the exclusion of economic evaluations of psychological or social service interventions that may be relevant for child health. A greater understanding of child health that integrates psychological, educational, and social service interventions will necessitate a broader approach to assessing health outcomes in children.

In the expanding field of health economics, many researchers focus on methods research. The PEDE database serves as a valuable tool for exploration of methodological challenges in the conduct of paediatric health economic evaluation. In addition, researchers interested in preparing a systematic review and/or meta-analysis of a particular paediatric disorder or aspect of economic evaluation will derive great benefit from such a resource. Plans are underway to update the PEDE database on an annual basis. A web based, user friendly searchable database with appropriate links to paediatric health services research groups, health economic associations and information synthesis organisations is available at http://pede.bioinfo.sickkids.on.ca/pede.

Acknowledgments

This research was supported by funding from The Canadian Coordinating Office for Health Technology Assessment and The Hospital for Sick Children Research Institute. The authors thank Mrs E Uleryk for assistance with search strategy development and for useful comments on the project. The assistance of Ms M Seyed and Ms R Henderson is gratefully acknowledged.

REFERENCES

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles

  • Atoms
    Howard Bauchner