Article Text
PDF
Abstract
Objective There is a body of published research on the effectiveness of home visiting for the prevention of child maltreatment, but little in the peer reviewed literature on cost-effectiveness or value to society. The authors sought to determine the cost-effectiveness of alternative home visiting programmes to inform policy.
Study design All trials reporting child maltreatment outcomes were identified through systematic review. Information on programme effectiveness and components were taken from identified studies, to which 2010 Australian unit costs were applied. Lifetime cost offsets associated with maltreatment were derived from a recent Australian study. Cost-effectiveness results were estimated as programme cost per case of maltreatment prevented and net benefit estimated by incorporating downstream cost savings. Sensitivity analyses were conducted.
Results 33 home visiting programmes were evaluated and cost-effectiveness estimates derived for the 25 programmes not dominated. The incremental cost of home visiting compared to usual care ranged from A$1800 to A$30 000 (US$1800–US$30 000) per family. Cost-effectiveness estimates ranged from A$22 000 per case of maltreatment prevented to several million. Seven of the 22 programmes (32%) of at least adequate quality were cost saving when including lifetime cost offsets.
Conclusions There is great variation in the cost-effectiveness of home visiting programmes for the prevention of maltreatment. The most cost-effective programmes use professional home visitors in a multi-disciplinary team, target high risk populations and include more than just home visiting. Home visiting programmes must be carefully selected and well targeted if net social benefits are to be realised.
Statistics from Altmetric.com
Introduction
Child maltreatment is a problem affecting children in developed and developing countries alike. It is estimated that approximately 20% of women and 5%–10% of men were sexually abused as children and that between 25% and 50% of all children are physically abused.1 Extensive long term impacts associated with maltreatment include impaired physical and mental health and reduction in the ability to contribute to society economically and socially.1
Arguably the most well researched programme for the prevention of maltreatment is home visiting, with many published randomised controlled trials (RCTs) and systematic reviews.2,–,5 The programmes are diverse in their objectives, components, intensity and target populations. Despite considerable investment in the delivery and evaluation of home visiting programmes, confusion still reigns about whether home visiting is a worthwhile investment for society.
What is already known on this topic
There is research on the effectiveness of home visiting for the prevention of child maltreatment, but little on cost-effectiveness or value to society.
What this study adds
The most cost-effective programmes use professional home visitors in a multi-disciplinary team, target high risk populations and include more than just home visiting.
The components and intensity of home visiting programmes and therefore the costs of implementation vary greatly; there are likewise differences in their demonstrated effectiveness in preventing child maltreatment. In order to inform policy, economic evaluation of individual programmes is important. Cost-effectiveness analysis is a useful tool to determine the relative value of investing in home visiting and other programmes for the prevention of maltreatment or to compare the economic performance of home visiting programmes in preventing maltreatment.
The cost-effectiveness of multiple home visiting programmes has not previously been reported in the peer-reviewed literature; however, it would be useful to have the cost-effectiveness of programmes compared in a single exercise so that consistency in the way costs and outcomes are valued would ensure validity.
The objective of this research was to perform comparable economic evaluations for all relevant individual home visiting programmes in policy relevant units. The purpose is to guide policy makers regarding which types of programmes for which populations are likely to be cost-effective and suitable for funding.
Methods
Study aim
The aim of the study is to determine the cost- effectiveness of home visiting programmes compared to current practice for the prevention of child maltreatment. If a number of home visiting programmes are assessed using a common cost-effectiveness metric, it is then possible to compare their relative economic performance and to investigate which types of programmes are most likely to represent good value. The analysis is presented from a societal perspective.
Selection of programmes
A systematic review was conducted to identify relevant trials of home visiting programmes. The search was performed in January 2010, and included terms for ‘child maltreatment’ and ‘home visiting’ and utilised databases such as Medline, PsychInfo, CINAHL, Embase, Sociological Abstracts and the Cochrane Collaboration. Inclusion was not restricted by publication date and due to the existence of systematic reviews in this area, these were initially searched as the main strategy. In order to ensure more recent trials were also captured, specific searches for clinical trials from 2000 onwards were also conducted. Reference lists, key websites, key journals and key authors were searched. For further details of the search, please refer to appendix A in the companion paper by Segal et al6. (Note that while the same 143 trials were considered in full text for both papers, only 33 programmes are included in this paper of the 52 included in the companion paper. The other 19 programmes were excluded from this paper due to the lack of reported outcomes suitable for economic evaluation.)6
To be eligible for inclusion for economic evaluation, studies required a control group, a core home visiting component, commencement during pregnancy or within 6 months of birth, and at least one of the following outcomes: child maltreatment, out-of-home care, injury or accident or hospitalisation. We adopted ‘must have a control group’ as an inclusion criterion rather than the more stringent randomised control group, to increase the number of studies available to inform our understanding of the cost-effectiveness of neonatal visiting programmes. As we also note that the early home visiting studies often claimed to be randomised but were rather ‘quasi-experimental’, we have conducted an independent assessment of the inherent biases rather than simply determining whether a study described itself as randomised.
Selection of alternatives
The costs and benefits of each home visiting programme are constructed and compared to the control group for each trial, respectively. The control groups received a variation of usual care, which included a minimal number of home visits during pregnancy or following birth (six programmes), developmental screening assessments (five programmes), referrals (three programmes), voluntary group meetings (three programmes) or brief tracking visits (one programme). Seven programmes failed to report details and a further five referred to ‘usual care’ with no description. Three programmes described that the control group received ‘no intervention’, although it is not clear what this means (table 1). Having similar control programmes is important in order to compare performance between programmes.
Description of home visiting programmes, populations and quality
Form of evaluation
Comparative incremental cost-effectiveness was the evaluation method chosen in order to provide information for the policy goal to most efficiently prevent child maltreatment for a designated budget. The published home visiting reports were used to obtain information on programme effectiveness as well as details of the programme components for costings.
Assessing study quality
The study design and rating of potentials for bias for each programme are summarised in table 1. Each included study was formally assessed by one reviewer (KD) for bias using criteria developed from the Cochrane Handbook,7 the Centre for Reviews and Dissemination guidelines,8 and Edgeworth and Carr9 who provide criteria specific to child abuse research. Each study was broadly classified as ‘good quality’ (one or less significant potentials for bias), ‘adequate quality’ (two significant potentials for bias) or ‘poor quality’ (three or more significant potentials for bias). Significant potentials for bias were restricted to quality items most likely to compromise study results, such as unadjusted group differences at baseline, failure to conduct intention to treat analysis, outcome assessors failing to be blinded to group status, provision of intervention and control services by the same nurses and non-random allocation to groups. The quality of the studies was considered good for five (15%) programmes, adequate for 16 (48%) and poor for 12 (36%).
The key effectiveness results used for economic evaluation are summarised in table 2 and are further elaborated below.
Cost-effectiveness results
Effectiveness measurement and valuation
A range of primary outcome measures were reported in the 33 studies selected:
Child maltreatment: death, substantiations/confirmed, reports, self-reported, neglect, domestic violence, out-of-home care placements
Injury and hospitalisation: injury, accident, emergency department attendance, hospital admission.
The above outcome categories were chosen as suitable for economic evaluation due to their close relationship with child maltreatment. They are all accepted indicators of maltreatment. Substantiations, confirmed cases, reports and out-of-home placements are all direct measures of child maltreatment, recognising that they will not fully capture all maltreatment that may occur.10 Witnessing family violence is conventionally classified as a form of emotional abuse or as its own distinct maltreatment category.11 Injuries and hospital attendance especially in small children have been shown to be indicators of maltreatment. A study by O’Donnell et al12 in Australia found that children with maltreatment allegations and substantiations had higher prior admission rates to hospital within the preceding year (OR 1.49 (95% CI 1.44 to 1.53) and OR 1.74 (95% CI 1.65 to 1.83)) compared with matched controls. In addition, children with substantiated maltreatment had higher rates of general admissions (0.777 per year compared to 0.216 per year for children without a substantiation) and higher rates of admissions for injuries (0.102 per year compared to 0.016 per year for children without a substantiation).12
Some programmes reported more than one outcome and all relevant results have been utilised. Where an outcome was reported at several time points, the point of longest follow-up was used.
Costs
Programme components were extracted from the published reports, to which Australian 2010 unit costs were applied. All dollar figures presented throughout this paper are Australian (the US dollar is approximately equivalent, January 2010 exchange rate of 0.9924). Unit costs for salaries were sourced from the relevant Australian professional organisations. The common programme costs categories were for salaries for those who deliver the home visiting programme, on-costs (additional costs associated with a salary such as superannuation, leave, payroll tax, etc), programme administration, training and supervision, travel, equipment and other costs such as child care, as appropriate. No discounting was applied to the programme effect or cost within the trial period (ie, 1–2 years).
Cost offsets
Cost offsets for the lifetime cost of maltreatment were applied to each case of maltreatment prevented. Estimated cost offsets were taken from the report by Access Economics on the costs of child maltreatment in Australia.13 The report calculates a lifetime cost of maltreatment for a cohort of first abused children in 2007, composed of healthcare costs (hospitalisation for injuries and treatment of depression and anxiety), additional educational assistance, productivity losses, crime, government expenditure on out-of-home care and protection, deadweight losses (efficiency lost through taxation), and premature death and loss of quality of life (mostly associated with anxiety and depression). The base case estimate of lifetime cost was A$14.4 million.
However, the actual incidence of child maltreatment is uncertain and varies depending on the definition of maltreatment and the data sources used. Access Economics use a lower estimate of incidence of 130 327 (based on the number of new substantiations in 2007 multiplied by 4.8 to adjust for known under-reporting) and an upper estimate of incidence of 489 954 (based on the number of new substantiations in 2007 multiplied by 18.2 to adjust for known under-reporting).13
We take the lower estimate of child maltreatment incidence of 130 327 children and apply the base case lifetime cost estimate of A$14.4 million to give a figure of A$110 253 per child. The upper (A$41.5 million) and lower (A$6.6 million) estimates of costs inclusive of burden of disease are combined with an incidence of maltreatment of 130 327 and are presented as sensitivity analyses (A$50 366 and A$318 760 per child).
Standardising outcomes
Translating diverse child maltreatment outcomes into a single metric, that is, case of maltreatment prevented, was necessary to allow cost-effectiveness estimates to be derived that allow for simple comparison between programmes and clear policy relevant recommendations to be developed.
Surrogate outcomes (injury and hospitalisation) were reported for eight studies (out of the 33 programmes in total) and were translated into substantiated cases of maltreatment using relationships from the published literature (see table 3). (The literature was searched purposefully and in a targeted manner for evidence to model the relationship between child maltreatment and surrogate endpoints. The search was focussed on finding high quality Australian data to match the policy context of these economic evaluations.) We note that hospitalisation does not directly translate into a maltreatment report, rather the baseline rate of maltreatment for all children is adjusted upwards for those children who are hospitalised compared to those who are not (ie, an OR of 1.49 indicates a child who is hospitalised is 1.49 times more likely to also have a maltreatment report compared to a child who was not hospitalised).
Literature used to inform relationship between surrogate outcomes and cases of maltreatment
Eight of the 33 included studies (programmes 23b, 27a, 15, 10, 26, 24b, 24a, 20) reported child abuse reports, which we have translated into child abuse substantiations using the latest Productivity Commission Australian data that there were 6.2 reports per substantiation (286 437 notifications and 46 187 substantiations).14
Allowance for uncertainty
One-way sensitivity analyses were performed as described in tables 3 and 4. The variables selected as upper and lower limits were determined based on a thorough assessment of upper and lower values reported in included studies (eg, for case loads and supervision ratios in accordance with relative level of population risk). We have varied effectiveness as a standard proportion for all programmes, and additionally have adjusted to account for trial quality (see table 4). Values were also discussed with managers of home visiting programmes.
One way-sensitivity analyses variables and limits applied
Results
Description of programmes
The literature search yielded over 1500 results with 143 papers examined in full text. From this we selected for inclusion 28 distinct trials covering 33 separate programme arms for economic evaluation.
The details of the programmes and enrolled populations are summarised in table 1. In summary, 25/33 (76%) of the interventions were evaluated using RCT methodology and 24/33 (73%) were conducted in the USA. The target populations of the programmes were mixed, with the majority delivering services for a combination of populations. Nineteen of the 33 (58%) interventions had the prevention of child abuse and neglect as a key objective. Eleven (33%) of the interventions utilised nurses or midwives as the home visitors, 10 (30%) used other professionals such as social workers or psychologists, six (18%) used paraprofessionals, three (9%) used lay people and five (15%) utilised a multidisciplinary team as home visitors. Sixteen (48%) home visiting programmes commenced prenatally and continued after birth. The mean number of home visits ranged from six to 41. The programme length ranged from 3 months with the longest offering services to children up to 5 years. The scope of the home visiting programmes varied significantly with only four (12%) programmes consisting of home visiting alone or home visiting plus minor additions (phone calls, referral, clinic visit, transport), whereas 11 (31%) programmes were considered to have a ‘whatever it takes’ approach with considerable flexibility to address client needs, with the remainder somewhere in between (for definition of categories refer to the footnote of table 1).
Cost-effectiveness
The effectiveness of the included programmes varied substantially from 0.03 to 200 additional cases of maltreatment prevented per 1000 children in the programme. Programme costs varied from A$1800 to A$30 000 per family, reflecting the large diversity in programme components and intensity. The cost-effectiveness results are presented in table 2. Eight of the 33 programmes were dominated (more expensive and less effective than the comparator). Thus cost-effectiveness results are reported for the remaining 25 intervention arms and are grouped by level of population risk to enable comparison between and within target population categories. In describing results we focus on the 18 programmes defined as meeting at least ‘adequate quality’ criteria and which are not dominated (see table 2).
Analysis of the cost saving programmes
Two programmes of at least adequate quality were found to be cost saving when a base case estimate of lifetime cost offsets associated with maltreatment prevented (A$110 253) was incorporated. These were program 13 (the Australian home visiting programme) and program 19 (Child and Youth Program in Baltimore). Using the upper estimate of lifetime cost offsets of A$318 760, a further five programmes become cost saving (programs 16a, 1, 24a, 24b, 14). These seven programmes are presented in table 5, where details are provided of the included populations and programme characteristics.
Programmes with at least adequate quality study design that are cost saving: population and programme characteristics
The seven best performing programmes were (in order of cost-effectiveness): the Child and Youth Program conducted in Baltimore with low income women involving lay visits (with professional support) until age 2 (program 19),15 an Australian home visiting programme for high risk young teenage mothers aged less than 18 years (program 13),16 the Special Families Care Project Minnesota, an intensive early intervention programme for mothers at high risk of abusing their infants (program 1),17 ,18 the Nurse Home Visiting Program in Denver Colorado that used nurse visitors (program 16a),19 ,20 the New Zealand Early Start programme for low income, welfare dependent women using nurses or social workers (program 14)21 ,22 and the Olds Nurse Family Partnership programme conducted in Elmira, New York involving nurses visiting prenatally and until age 2 years (programs 24a, 24b).23,–,30
All of the programmes except one (program 19) utilised professional visitors including nurses (24a, 24b, 13, 16a, 16b, 14), social workers/psychologists (14) or a multidisciplinary team (1). Three programmes included pre and postnatal visiting (1, 16a, 24a). All except two programmes (19, 14) enrolled largely, and some exclusively, first time mothers. One programme only enrolled adolescent mothers (13) and adolescent mothers comprised nearly half the sample for another three programmes (16a, 24a, 24b). The population risk level was elevated for all programmes. The scope of most programmes included home visiting plus many more services or a fully flexible approach. Three of the programmes (16a, 24a, 1) had a high number of home visits (>20), whereas two had a lower number (<10) of home visits (13, 26b), although for program 13 visits could be up to 4 h in duration. Five programmes continued until the child was aged 2 years (19, 16a, 1, 24a, 14), while the other two ran for 6 months only (24b, 13). The programmes had control groups with generally low intensities ranging from standard care only (n=2) through to minimal intervention (n=3), with a further two programmes not reporting what the control group received.
Given the difference in visit intensity, these seven programmes varied in their costs per family (see table 5). All programmes had a match between programme theory, components and population, suggesting integrity in programme design that should support better performance, which was found (see Segal et al for further details6).
Least cost-effective and dominated programmes
There were a number of programmes with very high incremental cost-effectiveness ratios (ICERs; estimated cost per case of maltreatment avoided) or where costs were greater than the comparator for a lesser outcome (dominated programmes: 28, 22, 25, 21, 8, 17, 4, 5). The programmes demonstrated to be of poor value with very high ICERs (eg, >A$1 million per case of abuse prevented or dominated) and of adequate quality included programmes 23b, 26, 20, 15, 10, 8, 4 and 5. These eight programmes were delivered by a variety of home visitors ranging from lay/paraprofessional visitors (programs 5, 8, 15) and nurses (programs 4, 10, 20, 26) to other professionals (program 23b). Some programmes commenced prenatally and extended postnatally (programs 8, 10, 15, 20, 26) and some were postnatal only (programs 4, 5, 23b). The target population groups were mixed, with three considered medium to low risk (programs 20, 23b, 26), three classified as mixed risk (programs 8, 10, 15) and only two considered high risk (programs 4 and 5). The programme scope was focused on just the home visiting service plus minimal additions for the majority of these programmes (programs 26, 15, 5, 4). Three programmes used less than 10 home visits (programs 4, 5, 26) and only two used more than 20 (programs 20, 23b). The majority of programmes ran for 12 months or less (programs 4, 5, 8, 26). The majority of these programmes (programs 4, 5, 8, 15, 26) did not have a match between programme theory, components and population, suggesting lack of integrity in programme design (see Segal et al for further details6). The programmes tended to have control groups that received low intensity standard care or minimal intervention.
Key differences between cost saving and least cost-effective programmes
The key differences evident for the cost saving programmes were the nature of the high risk populations and programmes that delivered much more than just home visiting. They tended to include more home visits for a longer period and all included a match between programme theory, components and population.6
Sensitivity analysis
The range in cost-effectiveness results derived from one-way sensitivity analyses are also presented in table 2. The cost-effectiveness results were most sensitive to the wage rates of all staff, the mean caseload of visitors (which were only varied where this was not reported), size of effect and the data used to translate intermediate outcomes (such as hospitalisation or injury) into child protection outcomes (reports).
Discussion
This research provides for the first time, economic evaluation of all published home visiting programmes presented in a format that allows for comparison of cost-effectiveness across diverse home visiting programmes. Results are reported as cost per case of child maltreatment avoided (with and without cost offsets). This was chosen as the performance measure of greatest policy relevance that would best support decisions that are in the best interests of society. It also allows for the comparison of cost-effectiveness across distinct programmes according to their particular characteristics, such as target population and type of visitor. While we have provided cost offsets associated with maltreatment estimated for Australia, the results can be readily adjusted to other countries and systems, as alternative country specific cost offsets can be applied to the estimated programme cost per case of maltreatment prevented, greatly assisting with policy relevance.
That the most cost-effective programmes were targeted at higher risk families is not surprising. Home visiting is an expensive service, with some programmes reporting that visitors may see only two to four families per day. Such a high cost programme is unlikely to represent a worthwhile investment from a societal perspective unless there is commensurate opportunity for a high level of benefit for each family visited. This suggests a programme that is sufficiently intensive and appropriately tailored to make a difference with the more vulnerable and complex families.
Relation to previous research
Economic evaluations of a small number of individual home visiting programmes have been reported in the published literature and were identified through a previous systematic literature review (and more recent update).31 Some of the existing published economic evaluations report non-policy relevant outcomes that cannot be interpreted, such as cost per point increase in maternal sensitivity.32 ,33 In contrast, our economic evaluations cover multiple programmes, and use cost per case of maltreatment prevented, which has the considerable advantage of being much easier to interpret and allows direct comparison between diverse programmes. Three cost–benefit studies were identified covering 17 programmes where all outcomes are assigned a dollar value and combined with costs to generate a net present value or benefit to cost ratio (see Drummond et al34 for an explanation of cost–benefit analysis), but do not allow comparison between programmes. They include a meta-analysis of 15 diverse programmes,35 conflicting with recognised protocols for the conduct of meta-analyses only where programme homogeneity exists,7 and two cost–benefit analyses of individual programmes. Their findings suggest that only some programmes are likely to lead to cost savings.
This research represents the largest volume and scope of economic evaluations performed of infant home visiting programmes in the published literature to date covering 33 distinct programmes/target populations. The broad scope has allowed investigation of programme and population components and their relationship to cost-effectiveness, which has not been previously possible.
Limitations
The economic evaluations rely on published study reports, which do not always provide detailed descriptions of programme components and resources to inform programme costs. We have, for instance, added programme administration, training and supervision costs for each programme, even when not explicitly identified.
The economic evaluations were limited to within the trial periods which differed for each programme. However, programme length is a core design feature and ideally should align with expectations about the achievement of programme goals. Thus programmes should be cost-effective within their ‘own’ time frame.
Economic evaluations are always performed for a specific context (Australian in this case) which is necessary in order to inform policy decisions. This analysis uses Australian cost data, and Australian data on the relationships between child protection reports and substantiations, and reports and hospitalisation. The cost-effectiveness results are not overly sensitive to reasonable variations in these relationships, and thus given the large differences in cost-effectiveness estimates, we suggest that the overall conclusions are still likely to apply.
Some programmes designed to prevent maltreatment find that child protection reports actually increase. This was the case for some of the programmes included in our analyses. This may or may not be an indication that the programme did not work and the impact of potential surveillance bias should be considered.
Conclusions
Important lessons are available from this economic evaluation of home visiting for infants for programme designers, implementers and funders. The extreme diversity in cost-effectiveness means that home visiting is not always a worthwhile societal investment. While evidence of the importance of the early years is undeniable, it is also the case that not everything that addresses this goal is cost-effective. In general, it appears that the more cost-effective programmes employ professional visitors and target high risk populations (often including young, low income, first time mothers with multiple disadvantage), and use a comprehensive approach (including more than just the home visits), designed to meet the specific needs of the target population.
In conclusion, if potentially large resources are to be spent on home visiting, careful analysis should be made of the home visiting programme objective, the specified target population, the planned programme components and the theoretical underpinning, which should all be consistent. On-going evaluation is also crucial to ensure that expected benefits to children and families are being realised.
Acknowledgments
The authors would like to acknowledge Rachelle Opie who systematically searched, selected and data extracted information on the home visiting programmes.
References
Footnotes
-
Funding The Australian Research Council provided funding for this study.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.