Background and aims: There are few standardised questionnaires for the assessment of respiratory symptoms in preschool children. We have developed and tested the short-term repeatability of a postal questionnaire on respiratory symptoms for 1-year-old children.
Methods: A newly developed postal questionnaire for the assessment of wheeze and other respiratory symptoms was sent to parents of a population-based random sample of 4300 children aged 12–24 months. After an interval of 3 months, a random sample of 800 respondents received the questionnaire a second time. The responses were compared using Cohen’s kappa (κ) to assess agreement corrected for chance.
Results: The first questionnaire was returned by 3194 (74%) families, the second one by 460/800 (58%). Repeatability was excellent (κ 0.80–0.96) for questions on household characteristics, environmental exposures and family history, good (κ 0.61–0.80) for questions on prevalence, severity and treatment of wheeze, and moderate (κ 0.39–0.66) for chronic cough and upper respiratory symptoms.
Conclusions: This short postal questionnaire designed for use in population-based studies has excellent repeatability for family and household characteristics and good repeatability for questions on wheeze. Short-term changes in symptom status might be responsible for variable answers on recent chronic cough and upper respiratory symptoms. Overall, the questionnaire is a valuable instrument for community-based research on respiratory symptoms in 1 to 2-year-old children.
Statistics from Altmetric.com
In epidemiological surveys, the prevalence of asthma and other wheezing disorders is usually assessed by questionnaires. Because objective tests are difficult to perform, especially in young children, and their sensitivity and specificity for the diagnosis of asthma are questionable,1 the design of appropriate questionnaires is crucial. Basically, a new instrument needs to demonstrate repeatability and validity. This paper focuses on test–retest repeatability, a measure of the consistency of the performance of a questionnaire when used under similar circumstances.
Standard respiratory questionnaires for schoolchildren have shown good or satisfactory repeatability2–6 but may not be appropriate for infants and preschool children, where symptoms may differ due to developmental changes and rely exclusively on proxy reports. For instance, limitation of speech with severe wheeze and exercise-induced symptoms cannot be assessed in infants unable to talk and run. Moreover, different clinical phenotypes are thought to co-exist within the large group of preschool children suffering from wheeze.7–9 To distinguish these phenotypes, symptoms must be assessed in detail. Therefore, standard questionnaires developed for schoolchildren cannot be applied automatically for very young children. Few targeted questionnaires exist. Their repeatability has been tested in small numbers of children recruited in neonatal units or outpatient clinics10 or excluding ethnic minority groups.11 They are therefore not necessarily generalisable.
We developed a new postal questionnaire for use in a large population-based cohort study of respiratory symptoms in preschool children in Leicestershire.12–16 This paper describes the short-term repeatability of this questionnaire when it was reapplied to parents of 1-year-old children after a 3-month interval.
Setting and study population
In 1998, we used the Leicestershire Child Health Database, which includes the birth notification with mother’s self-reported ethnic origin, religion, country of birth and language, to select a random sample of 3500 white (mother self-identified as British Isles or European) and 800 south Asian (mother self-identified as Indian, Pakistani or Bangladeshi) children aged 1 year (ie, in their second year of life at recruitment) with complete birth records. South Asians, the largest ethnic minority group, accounted for 14% of 1-year-old children resident in the county at that time.
Questionnaire and mailings
We developed a short four-page questionnaire that could be posted to families with young children, for self-administration by parents (the questionnaire is available on the ADC website at http://adc.bmj.com/supplemental). It contained sections on upper and lower respiratory symptoms and diagnoses, healthcare utilisation and treatments for wheeze, environmental exposures including indoor air pollutants, breastfeeding, pets, nursery care, number of siblings and other household members, parental history of atopic diseases and ethnicity, language and social conditions. Some questions were derived from an earlier questionnaire used locally11 or from the International Study of Asthma and Allergy in Childhood (ISAAC).6 Others were newly developed. The sources of the different questions are summarised in the online table 1 (available at http://adc.bmj.com/supplemental). All questionnaires were printed in English, with an accompanying letter translated into the four main local south Asian languages offering translation services. We sent this questionnaire to all 4300 families, with a reminder letter to non-responders 6 weeks later (first mailing in April, reminder in May, baseline survey). Within 3 months, 3194 (74%) questionnaires had been returned. An identical questionnaire with an explanatory letter was then sent exactly 3 months later (in July, repeat survey) to a random sample of responders stratified by ethnic group (600 white and 200 south Asian families).
Statistical analyses were performed with Stata, version 8.2 (Stata, Austin, TX). Symptom prevalence at baseline was calculated separately for participants and non-participants of the repeat survey, and comparison between these groups was assessed by χ2 tests (dichotomous variables) and tests for trend (categorical variables). To assess repeatability, Cohen’s kappa (κ) coefficients were calculated.17 κ compares the observed agreement between two assessments made on two different occasions, with the agreement that would be expected simply by chance. Because the κ coefficient is sensitive to the population prevalence of responses and also to asymmetrical imbalance in marginal totals, we also present the percentage of observed total agreement (number of positive and negative answers to both questionnaires divided by the total), and the separate proportions of positive and negative agreement (number of answers in positive agreement divided by the average number of positive answers; number of answers in negative agreement divided by the average number of negative answers). These values help in understanding individual results.18 19 For ordinal and quantitative variables, agreement was assessed as intraclass correlation coefficients (ICC). Landis and Koch20 have suggested that κ coefficients ⩽0.4 indicate poor agreement, values of 0.41–0.60 moderate agreement, 0.61–0.80 good agreement and >0.8 excellent agreement. To examine which factors might influence agreement, we stratified analyses by household language (English, other), ethnicity (white, south Asian), the Townsend local area deprivation score (low, medium, high), interval between baseline and repeat survey (<3 months, ⩾3 months), domicile (urban, rural) and respondent (same respondent, different respondent to repeat questionnaire, and mother vs father). The equality of subgroup agreement measurements was tested using the methods of Donner et al.21 22 Missing values were coded as “no” because sensitivity analyses showed no difference in results.
The response rates were 74% (3194/4300) in the baseline survey and 58% (460/800) in the repeat survey, and higher for white (n = 368/600, 61%) than for south Asian children (n = 92/200, 46%, p<0.001). Most baseline questionnaires were completed in April or May (n = 2691/3194, 84%), repeat questionnaires were mostly completed in July and August (n = 451/460, 98%). The median (interquartile range) age of the children was 17.7 (14.8–20.7) months at the baseline survey and 20.5 (17.7–23.2) months at the repeat survey. For test–retest analyses, we included the 413 children with the same respondent (mother or father) in both occasions, as conventional.2 However, results were very similar in a sensitivity analysis including all 460 questionnaires.
Prevalence of respiratory symptoms
The 413 study participants who replied to the repeat questionnaire had a lower prevalence of wheeze, shortness of breath and cough in the baseline survey than the children who failed to respond (table 1). Among 413 participants in the repeat survey, the overall prevalence of respiratory symptoms was similar on the two occasions.
The repeatability of the most important symptoms are shown in table 2 (see also online tables 2 and 3 at http://adc.bmj.com/supplemental which give detailed results for all symptoms, family history and environmental exposures). In general, agreement was excellent (κ >0.8) for family history of atopic diseases and environmental exposures (smoking, breastfeeding, pets, cooking fuel). Questions on the frequency and severity of wheeze, asthma diagnosis, inhaler treatment and healthcare utilisation had good repeatability (κ 0.61–0.80). Repeatability was moderate for questions on triggers of wheeze and cough (κ 0.49–0.72). In contrast, agreement for questions on chronic cough, upper respiratory symptoms and skin problems were (with the exception of a diagnosis of eczema) only moderate (κ 0.39–0.68). Stratified analyses did not show any significant differences in agreement between subgroups, with one exception: repeatability of several questions was poorer when both questionnaires had been completed by the fathers (n = 39) compared to mothers (n = 374). Low p Values were found for questions on night cough (κ 0.10 vs 0.44 for fathers and mothers, respectively, p = 0.003) and cough apart from colds (κ 0.28 vs 0.56, p = 0.09), snoring (κ 0.51 vs 0.67, p = 0.08), eczema (κ 0.22 vs 0.70, p = 0.003), paternal smoking (κ 0.53 vs 0.88, p = 0.001), number of physician visits (ICC 0.55 vs 0.76, p = 0.02), posseting (ICC 0.36 vs 0.70, p = 0.002) and duration of breastfeeding (ICC 0.85 vs 0.96, p<0.001).
This paper presents the repeatability of a short respiratory questionnaire designed for self-completion by parents of 1-year-old children. Repeatability was excellent for sections on family history and environmental exposures, good for questions on wheeze, asthma, treatment and healthcare utilisation over the past 12 months, and moderate for upper respiratory symptoms and cough.
One strength of our study compared to most others is the relatively large sample size, providing increased precision (reasonable confidence intervals around κ coefficients). We had a well-defined population-based study group, including British south Asians, the main local ethnic minority group, in a proportion similar to that of the general population (14%). The only other published repeatability studies for preschool questionnaires of which we are aware included few and/or selected children: 72 hospital-based children10 and <100 (exact number not quoted) children of white ethnicity.11 The response rate in the repeat survey (58%) compares well with response rates of 47–50% in other repeatability studies (table 3).
As in other surveys,2 respondents in the repeatability study were less symptomatic than non-respondents. κ depends on the marginal observed prevalence.18 With a fixed agreement rate, κ is maximal for a prevalence of 0.5 and decreases if the prevalence approaches 0 or 1.23 In our study, the prevalence of wheeze in the last 12 months in the participants of the repeatability study was 25%, with a resulting κ of 0.69 and an agreement rate (PO) of 88%. For a prevalence of 35%, as in the total study population, κ would have been 0.74 (assuming the same agreement rate). Thus we may have underestimated κ values. On the other hand, respondents may be more reliable than non-respondents, leading to an overestimation of κ. Overall, the resulting bias is likely to be small. Other characteristics of the study, such as the age of the children, the time period over which the questionnaire extends, who responded to the questionnaire and the interval between repeat surveys, will have a bigger impact on κ values.23 These differences in methodology have to be taken into account when comparing different studies (table 3). For instance, short intervals between the two measurements4 10 24 give the parents less time to forget previous answers and the children less opportunity to change their true symptom status. Also, we need to be cautious when extrapolating our results from 1-year-old children to older toddlers. For instance, repeatability of the question on duration of breastfeeding is likely to be poorer if the question is asked at an older age.
It is reassuring that repeatability did not differ much between subgroups defined by language, ethnicity or social class, although power for these comparisons was limited. Where fathers responded, significantly lower repeatability was found for questions on current symptoms and infant care, confirming clinical observations. In most families, mothers spend more time at home with young children than fathers and are more likely to take time off work when the children are ill. Therefore, mothers usually provide more accurate reports on children’s health status. The relatively poor repeatability we found for the questionnaires completed by fathers might, however, not have a large impact for the interpretation of questionnaire surveys, because most questionnaires are usually completed by mothers.
Although the absence of objective measurements of atopy and environmental exposures such as parental smoking in epidemiological studies is often criticised, the high repeatability of parental answers gives confidence in these questionnaires. When interpreting the results for symptoms over the past 12 months, it has to be kept in mind that perfect repeatability can never be obtained because the time windows do not completely overlap. We sent the second questionnaire 3 months after the first one, so that new symptoms will have developed in some of the children and remitted in others.
This is not the only cause for imperfect agreement, as shown for the symptom “wheeze ever” which, illogically, was reported in the first but not the repeat questionnaire by about 5% of parents (as also in Brunekreef et al2) (table 2). Poorer repeatability for cough and upper respiratory symptoms compared to wheeze has also been noted in other studies. This may be partly explained by high short-term variability in these symptoms, making it likely that symptoms are reported if they have occurred recently, but not if the last episode happened many months previously. As all questionnaires were sent at the same time of the year, we could not compare repeatability between different seasons of the year. In contrast, wheeze, shortness and breath and inhaler use might be recalled more consistently because parents are more concerned. For skin problems, repeatability was significantly better for the question on diagnosis (“eczema”) than for ISAAC questions on symptoms (“itchy rash”) (table 2 and online table 2).
Finally, we want to stress that having shown a good repeatability does not allow the conclusion that our questionnaire has also a good validity. While repeatability refers to the reproducibility of a measurement, validity refers to whether the questionnaire measures what it intends to measure. The best way to assess validity is to compare answers to the questionnaire with objective measurements. These might include repeated respiratory sound recordings to validate reports of wheeze and cough, comparison with hospitalisation or GP records to validate health care utilisation and inhaler use, or measurement of urinary cotinine to validate exposure to environmental tobacco smoke. It was not possible for us to do this.
In conclusion, this short postal respiratory questionnaire developed for cross-sectional and longitudinal studies of preschool wheeze and other respiratory symptoms has a repeatability, in 1-year-old children, similar to standard respiratory questionnaires for older children, and could therefore be recommended for further use in community-based studies in this age group.
What is already known on this topic
Standardised questionnaires on asthma and wheeze exist for schoolchildren.
Similar instruments for preschool children are scarce despite significant age-related differences in the features of many respiratory symptoms.
What this study adds
We developed a parent-completed questionnaire for assessment of respiratory symptoms in 1-year-old children which showed good to excellent repeatability.
This instrument can be used in other community-based surveys of this age group.
We thank the parents of the children for completing the questionnaires, and Tony Davis, Business Manager, Children’s Directorate, Leicester City West Primary Care Trust, for his assistance.
Funding: The work presented in this paper was funded by the Swiss National Science Foundation (PROSPER grant 3233-069348 and 3200-069349, and SNF grant 823B-046481) and the UK Department of Health (grant no. 0020014). Initial data collection was supported by a research grant from Trent NHS Executive (Trent Research Scheme, RBF no. 98XX3).
Competing interests: None.
The Leicestershire Health Authority Research Ethics Committee approved the study.
- intraclass correlation coefficients
- International Study of Asthma and Allergy in Childhood
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.