METHODS

In brief, all parents of 2–8 year old children registered with three general practices in a sociodemographically mixed area of Oxford were invited to participate in a survey (69.4% response rate) which included a validated child mental health/behaviour inventory²⁷ (Eyberg Child Behaviour Inventory). One parent in each family where inventory scores for at least one child in the family fell above the median (that is, worse behaviour) were invited to participate. In families with more than one child in the age range, the index case was the child with the highest score. Parents of children with a diagnosed learning difficulty or previous treatment for behaviour problems were excluded. A total of 116 parents (30% of those invited) consented to enter the trial. These parents were no different from those who refused in terms of their social class, but they were more likely to have a child whose behaviour scores were in the clinical range on the Eyberg inventory (39.4% versus 29.5%). Consenting parents were allocated to one of two groups on the basis of their availability, preferences for day and time of attendance, and need for crèche provision. Key demographic factors including single parenthood, ethnic group, occupational status, and child’s ECBI score and age were also taken into account in group allocation in order to balance the groups on key confounding variables. The two groups, one comprising 60 parents and one comprising 56 parents, were then randomly allocated to receive the intervention or to be part of a control group.

Data were collected using questionnaires comprising a number of sociodemographic questions and the following validated mental health inventories: Eyberg Child Behaviour Inventory²⁷ (ECBI); Goodman Strengths and Difficulties questionnaire²⁸ (SDQ); Parenting Stress Index²⁹ (PSI); 28 item General Health Questionnaire³⁰ (GHQ); and Rosenberg Self Esteem Scale³¹ (RSE). The questionnaire was administered pre-intervention, immediately post-intervention, and at 6 and 12 months follow up.

Analysis was conducted on an intention to treat basis using paired t tests and Wilcoxon matched pairs signed rank test (as appropriate for the distribution of scores) to calculate the significance of differences between baseline and follow up scores within groups. Grouped t tests and Mann-Whitney U tests were used to calculate the significance of between group differences.

All parents from the first three of six parenting groups and the equivalent control group parents were invited to participate in a semi-structured interview at 2–4 months post-intervention. Twenty six intervention group (23 who completed more that 50% of the sessions and three who did not) and 15 control group parents agreed to take part in an interview. Both groups were asked about their approach to parenting and any difficulties they had experienced as parents. Intervention group parents were asked about their views of the parenting programme, the programme leaders, and personal outcomes. Control group parents were asked about the impact of taking part in the study. Interviews were conducted in the parents’ home, audiotaped with the parents’ permission, and transcribed in full. Data were coded in three stages: open coding in which the data were broken into the smallest meaningful unit; axial coding in which the initial codes were compared and contrasted and grouped together in larger categories; and selective coding focusing on higher order categories or themes and the relationship between them to develop a conceptual framework. When this process was completed the findings from our study were compared with those from other published studies to identify the extent to which they confirmed or refuted the findings of other qualitative researchers.

Both 6 and 12 month questionnaires included open ended questions for parents to record their views about the impact of taking part in the study, the value of the parenting programme, what was most helpful or difficult, and whether the parent would recommend the programme to a friend. Forty six intervention group parents and the same number of control group parents completed questionnaires at 6 months. At 12 months questionnaires were returned by 44 and 43 parents respectively in the two groups. These data were analysed thematically with the aim of validating the results of the interview data.

The methods and results of the qualitative aspects of the study are described in full elsewhere.^32,33 Only the results with a bearing on interpretation of the quantitative results of the trial are presented here.

The intervention

The Incredible Years programme from the Parent and Child Series²⁵ is primarily a behavioural intervention, which uses videotape modelling and experiential learning covering child play, praise, rewards, ignoring undesired behaviour, limit setting, positive discipline such as time out, and logical consequences of behaviour. Parents set themselves goals, undertake homework each week, and report back on progress. Sessions are of two hours duration and take place weekly over a 10 week period. Five health visitors and two nursery nurses received three days training in facilitating the programme, from a local voluntary organisation, the Family Nurturing Network, prior to the start of the trial. They also attended supervision sessions on a weekly basis during the course of the programmes. These focused on addressing problems arising from the delivery of the programme (that is, pacing of the presentation of materials, group dynamics, use of role-play, etc), in addition to ensuring the integrity of its delivery. Four of six parenting groups met at a local medical centre and two groups met at a local community centre. Four groups met in the evening and two in the daytime with crèche facilities available.

RESULTS

Quantitative

Groups were well balanced in terms of key demographic and social factors.²⁶

Just over half of the index children were boys. The age range of the index children was 2–8 years with a mean of 4.6 (SD 2.0) years. Thirty four (31 mothers and three fathers) of the 60 parents in the intervention group attended 50% or more of the sessions. They were more likely than those who dropped out to have index children under the age of 4 years (47.19% versus 16.7%). Loss to follow up was 23% (n = 13) at 12 months in the control group and 28% (n = 16) in the intervention group (28 of the 34 attenders and 16 of the 26 non-attenders provided data) (fig 1). Parents lost to follow up did not differ significantly from those remaining in the study with respect to age or sex of child, ethnicity, social class, single parent status, child behaviour scores, whether scores fell in the clinical range, parent scores on the GHQ, PSI, or the self-esteem scale, whether the parent attended the group, or whether they originally expressed interest in attending or had attended another parenting group in the past.

Table 1 reports the mean score and standard deviation for all outcomes pre- intervention and at 6 and 12 months follow up in control and treatment groups. The scores of all parents who returned questionnaires at each stage are included in each mean.

At 12 months significant change (p < 0.05) in a positive direction was observed for intervention group children on the intensity scale of the ECBI, and the total, conduct, and hyperactivity scales of the SDQ. Significant change in a positive direction was also observed for intervention group parents on all scales of the GHQ, with the exception of the anxiety subscale; on all scales of the PSI, with the exception of parent-child interaction subscale; and on the RSE scale. However, control group children also showed significant change on both scales of the ECBI and on two scales of the SDQ (total and conduct). Control group parents showed significant improvement on all scales of the PSI, with the exception of the parent-child interaction subscale, and on the RSE.

Table 2 shows between group differences (those showing differences in mean change between groups) for all outcomes. At 12 months there were no significant differences between the control and intervention group as regards any of the scales measuring children’s emotional and behavioural adjustment. There was one significant difference on one of the scales measuring parental mental health—the GHQ depression scale.

Figures 2–4 show the plotted change over time in mean scores for the outcome measures that showed a significantly greater change in the intervention than control group at either 6 or 12 months: ECBI intensity scale, SDQ conduct subscale, and GHQ depression scale. These figures suggest that, although there was a small increase in ECBI intensity scale scores at 12 months, improvements in mental health and behaviour in intervention group children were largely maintained over time. The loss of statistical significance between groups was predominantly due to a continuing fall in control group scores between 6 and 12 months. In contrast GHQ depression scores for parents were different at the start of the trial with those in the intervention group being much higher. The significant reduction in depression scores in this group is attributable to these unexplained high scores at the start of the trial. Intervention group scores on this measure were similar to those in the control group at all three post-intervention measurements.

Qualitative results

Both intervention and control group parents reported experiencing similar difficulties with parenting before the trial started, including feeling isolated, frustrated, stressed, and out of control at times. They also reported difficulty dealing with their children’s problem behaviour.

Impact on intervention group parents

One of the clearest themes to emerge from the data was that parents had found the programme helpful. Ways in which it had helped included: enabling them to feel more able to cope, more confident, more supported, and more skilled in dealing with their children’s problem behaviour. With regard to specific behaviours, some parents reported shouting at their children less and using more positive discipline strategies.

“This helped me to be calm and to deal with things as they crop up instead of letting them go on ... time out has really helped—it’s not a cure ... but I am much calmer—I can just do my task calmly and not have the anger.”

Some parents described specific improvements in their children’s behaviour; for example, reductions in biting others, or in night waking. Others reported an improvement in their relationship with their child, citing “child led” play as important in this respect.

Reports of problems or difficulties with the programme included: doubts about “time out” as a technique, and difficulty in implementing changes because of lack of support from spouses/partners. Some parents, but not all, expressed a preference for attending programmes with their spouses.

“I do wish in a way that I could have dragged my husband along! ... it would have been difficult to arrange but one ... of the key things to come out of it was that it was important that you agreed on how you were going to deal with certain issues before they arose so that you could sort of present a united front.”

Several parents also reported difficulty sustaining the changes that they had made while attending the group. They reported finding it hard to change the habits of a lifetime and suggested additional sessions at a later date to reinforce what they had learnt.

“It’s so difficult ... because ... for years and years you’ve ... been ... the way I’ve been brought up and ... then in ... 8 or 10 weeks ... they totally change your way of ... doing things and then after that you’re left to your own devices ... it’s so easy to ... go backwards.”

Reports that the programme was, in general, unhelpful were rare even among those who dropped out. Common reasons for dropping out included increased work commitments, geographical move away from the area, and holidays.

Impact of the trial on the control group

One of the themes to emerge from the data collected from the control group was that the completion of the trial questionnaires had encouraged these parents to reflect on their parenting and their children’s behaviour. Some reported feeling more confident that they and their children were normal.

“It is nice to know that we all encounter the same/similar problems with our children. Sometimes when I shout at my child I feel guilty and the questions made you realise that we all do it sometimes.”“I think that over the course of the study I have seen my responses become more positive both in relation to child’s behaviour and my attitudes. I might not have been so aware of this had I not answered the same questions on a regular basis. Thank you!”

At the 12 month follow up, one intervention group parent (a non-attender) and four control group parents said that they had attended a community based parenting programme.

DISCUSSION

The aim of this trial was to test the potential effectiveness of a universal open-access programme. The block randomised design was chosen because it is a more practical design to use for trials of health promoting interventions delivered in groups.²⁶ Only parents of children whose behaviour fell in the worst half of the distribution were invited to take part because of concerns about documenting change in population samples using instruments designed to identify and measure change in clinical populations. Thirty per cent of those invited agreed to take part and these were more likely to be parents whose children had problem behaviour. Although in terms of a clinical intervention this uptake rate is low, in terms of a public health intervention it is very good, and the fact that it particularly attracted those whose children had problem behaviour means that this universal approach does reach those in need. Although only 57% of the participants attended at least half of the sessions, this figure compares well with the attendance rate in parenting programmes in both high risk or clinically indicated samples. It is useful to note that the drop out rate was higher among parents of older children, confirming that the optimum age for invitation to this programme is likely to be when the child is 2–3 years of age.⁴⁰ A follow up rate of 75% was achieved at 12 months and the follow up covered attending and non-attending parents. The intention to treat analyses therefore includes drop outs. No differences were apparent between those who were lost to follow up and those who contributed data, so there is no reason to suppose that loss to follow up will have biased the results.

It has been possible to show a positive short term (up to six months) impact of this parenting programme on the mental health and behaviour of children. However, while within group differences showed that improvement in children’s mental health was largely maintained at 12 months, between group analyses showed no significant differences. The absence of significant findings at 12 months was mainly due to the fact that between the 6 and 12 month follow up periods, the scores of the control group improved, while the scores of the intervention group remained similar to those obtained at 6 months. Had this trial been conducted using waiting list controls, as have most earlier trials of parenting programmes, these results would have been interpreted as showing long term effectiveness.

Our results might be partly attributable to regression to the mean and/or to improvement in scores resulting from expected maturational change.²⁶ Such effects, however, would have been expected to be maximal during the first six months of the trial, and the further improvement in control group scores after six months was therefore unexpected. While the Eyberg behaviour inventory is validated for children age 2–12 years, the Goodman Strengths and Difficulties questionnaire has been validated only down to the age of 4 years. The preponderance of preschool children in the trial may have contributed to the negative findings on this instrument. Behaviour scores of preschool children are less stable over time than those in older children and changes may therefore be more difficult to detect.

The one significant change in favour of the intervention group at 12 months related to parental depression. This result was likely to be attributable to unexplained high depression scores in the intervention group parents at baseline. In this analyses of 19 outcome measures, it could also have occurred by chance.

In the face of these negative findings with respect to the quantitative data, the qualitative data are important. What emerged from our qualitative data was that the great majority of parents found the programme helpful. The effects they described—feeling more supported, less stressed, more skilful, and more able to cope with problems—are important components of positive mental health,⁴¹ and are recognised as important in terms of caring for children. Standard mental health inventories, focusing as they do on mental illness, may not be especially sensitive to improvement in these aspects of mental health. Likewise, validated child behaviour inventories may be insensitive to the improvements in the quality of parent-child relationships and the atmosphere at home, but both of these are likely to be important for social and emotional development.

While some parents clearly reported achieving change in their children’s problem behaviour and in unhelpful parenting styles, some described difficulty implementing changes because of lack of support at home. Others reported problems sustaining changes they had initially made. It is possible that changes achieved and sustained by the intervention group overall were not sufficient to produce a large enough difference in quantitative scores to achieve statistical significance.

Interviews with control group parents provided some indication that there may have been a Hawthorne effect operating as result of taking part in the trial. Parents reported taking more interest in their children’s behaviour than they had previously done. As a result they may well have been more open to information about positive parenting widely available in the media and from other parents than they would have been if they had not been taking part in the trial. Nine per cent (4/44) of control group parents had taken the initiative to find and attend a community based parenting programme between the six month and one year follow up. Some of the improvement in control group scores may therefore have been due to real improvements in parenting inadvertently brought about by the trial (contamination).

The qualitative data showed that some parents thought the programme would have been more effective if they had attended with their partners. This finding is consistent with the results of other studies,³⁴ and many service providers are trying to find ways to encourage fathers to attend parenting programmes.³⁵ Reports of difficulty sustaining behaviour change over time and the desire for further support have also been reported previously.^9,33 Continuing support to the group from both volunteers and professionals has been proposed as well as flexible, open access, non-structured sources of support.³⁷ Previous studies have also suggested that 30–50% of families who take part in parenting programmes are likely to show no benefit because of other events in their lives, and may therefore need more than a single intervention.³⁶

Behaviour change is widely acknowledged to be difficult to achieve by those working in health promotion, and models of behaviour change developed to describe and support changes in health related lifestyles (smoking cessation, exercise participation) may be helpful in interpreting the results of this trial.^38,39 In particular, positive change followed by relapse is not uncommon. Neither is it uncommon to have several attempts before finally changing for good. It may therefore be necessary to provide additional support and/or repeat programmes, at least to some families, to achieve maximum effectiveness.

Overall, the 6 month results of this trial are consistent with the findings of other studies which show that parenting programmes can have a positive impact on parent and child mental health, and that this improvement can be maintained.^10,12 However, the non-significant findings at 12 months, although possibly attributable to methodological problems, suggest that the beneficial impact reduces over time. They illustrate the importance of conducting follow up with both intervention and control groups. Collection of qualitative data enabled a better understanding of the results.

Conclusions

While this study does not provide the evidence base to show long term effectiveness of this parenting programme in a non-clinical population in the UK, there are reasons to believe that the results may have been partly attributable to Hawthorne effects, to contamination of the control group and to inadequacy of outcome measures rather than to lack of intervention effectiveness. The study has provided evidence that the great majority of parents valued the intervention and that they perceived it to enable them to make beneficial changes to their parenting and to improve their mental health. It has also shown that some parents found the changes difficult to sustain and has pointed to ways of improving the long term effectiveness of these programmes.

The 30% uptake and 57% attendance rates, while at first sight disappointing, are impressive for a health promoting intervention in which those invited did not have acknowledged health problems. They suggest a level of acceptance of the potential value of such programmes which indicates that universal coverage could be a reasonable long term goal.

If this goal is to be pursued, further development of parenting programmes is necessary—particularly with regard to attracting fathers as well as mothers, and to providing reinforcement sessions after programmes have finished. Further UK trials with long term follow up will also be needed, in particular larger trials, including all families, which use outcome measures capable of measuring change at the positive end of the mental health spectrum. Such trials will benefit from incorporating qualitative as well as quantitative methods of data collection.