Aim: To develop a standardised and validated respiratory symptom questionnaire for use in epidemiological or follow up studies in infants and preschool children.
Methodology: After initial design and development, the questionnaire was administered to two cohorts of subjects, one recruited from a respiratory clinic and the other from a postnatal ward. The two cohorts then repeated the questionnaire, two weeks apart. The qualities of the questionnaire were assessed.
Results: Response rate to the initial questionnaire was 100% for the clinic based cohort and 64% for postnatally recruited families (total number of subjects 114). Questions showed good to moderate short term reliability (weighted kappa scores 0.47–0.7; average correct classification rates 0.74–0.91). Four domain concept scores showed excellent internal consistency (Cronbach alpha scores 0.87–0.95). Using principal component factor analysis, four new domains were devised showing acceptable construct validity and internal consistency. Criterion validity was assessed using a respiratory physician based diagnosis of asthma (RPBDA) as the gold standard for comparison. All eight scales in the questionnaire could significantly distinguish between infants with RPBDA and well or mildly symptomatic subjects.
Conclusion: We have developed a practical, acceptable questionnaire with eight concept domains for use in infants and preschool children. The questionnaire has strong construct validity and internal consistency with good short term reliability of questions. More detailed study of criterion validity and the responsiveness of the questionnaire is required using a larger population and including children with the different phenotypes of wheezy illness.
- respiratory questionnaire
- RPBDA, respiratory physician based diagnosis of asthma
- URTI, upper respiratory tract infection
Statistics from Altmetric.com
Over the past decade there has been more understanding of the epidemiology of wheeze in infants.1 It appears that there may be a number of subgroups of wheezing illness in infants, each having different aetiological factors and natural history.2 Although the epidemiological characteristics of the different subgroups are beginning to be described,3–5 detailed descriptions of symptom patterns have not yet been fully explored. Currently there is no validated or standardised questionnaire available to explore the prevalence and natural history of respiratory symptoms in this age group. This has partly been caused by problems with the definitions and terms used to describe wheezing illness in infants. The international study for asthma and allergies in childhood (ISAAC) questionnaire is now the most commonly used questionnaire to study the prevalence of respiratory symptoms in children. This has been designed for parental completion for their 8–10 year olds or for self completion for the 13–14 year old age group.6,7 Other questionnaires are available but are all designed for older children.8–11
Luyt et al designed their own questionnaire for use in 1–5 year olds and the questions focused on wheezing and wheeze attacks with some emphasis on running induced wheezing.12,13 They defined severity of wheezes, and number of attacks of wheeze in 12 months, but did not examine the impact of the wheeze attacks on the child and the family. A questionnaire designed for 5–14 year olds has been examined for validity in Sheffield, UK and this explores not only the frequency of respiratory symptoms but also the impact that those symptoms have on the child and family.14 The aim of this study was to design and validate a respiratory questionnaire particularly for infants and preschool children, which not only examines the frequency and pattern of wheezing and other symptoms but also focuses on the impact that these symptoms have on the child and family. The resulting questionnaire may be useful for future epidemiological studies and as a follow up tool for use in neonatal studies where often the outcome at 2–3 years of age is of interest.
Design of the questionnaire
Using the standard questionnaires already in use, mainly for older populations,4–10 a questionnaire was designed. The following features common to these were used: wheezing, cough and breathlessness frequency, within four concept domains of daytime symptoms, night time symptoms, impact of the symptoms on the child, and impact on the family. The suggestions of six respiratory paediatricians were used to help finalise the content. It was felt that long term parental recall would be unreliable, therefore most of the questions refer to the child’s symptoms over the preceding three months. A template based on a previous questionnaire was employed where each question has a stem and five possible graded responses.14 All questions were phrased so as to avoid the use of technical terms, leading questions, composite questions, double negatives, presuming questions, prestige questions, imprecise questions, and use of words open to interpretation.15
Assessment of questionnaire characteristics
This is a qualitative process that assesses the ease of administration and acceptability of the questionnaire. A number of colleagues and parents of children attending clinics were asked to read the questionnaire; it was then refined further, taking into account the views of these individuals.
This is the number of individuals returning the questionnaire and was determined by administering it to parents of children who were recruited at birth from the maternity unit, and also parents of children under 3 years of age referred to the outpatient respiratory clinic with chronic respiratory symptoms (questionnaires were sent prior to the visit for the parents to complete before the consultation).
The two aspects of reliability that were studied were:
The test–retest reliability is the stability of response, which is short term reliability over time (usually 1–2 weeks). This is assessed using Cohen’s kappa, weighted kappa score, and average correct classification scores for analysis. Kappa scores >0.41 are considered to show moderate agreement; those >0.61, good agreement; and those >0.81, very good agreement.16,17
The parents of the infants recruited from birth were sent a further questionnaire two weeks after the first one was returned. The parents of the children referred to the respiratory clinics were asked to complete the questionnaire again while attending clinic prior to their consultation. Having two questionnaires per child enabled us to examine the short term reliability of questions.
The internal consistency is the extent to which the questions in each underlying concept domain interrelate with each other (using Cronbach’s alpha and factor analysis). An alpha score >0.7 is acceptable, >0.81 is good, and >0.91 is excellent.18
The internal consistency of each of the four domains was examined from the responses to all the initial questionnaires collected. Using factor analysis, alternative domain concepts were developed and further exploration of construct validity was completed.19
Face and content validity
The assessment of face validity and content validity (whether the questionnaire covers all relevant aspects of the topic unambiguously) was made informally by the respiratory paediatricians involved in the study (NJS and CVP).
There is no “gold standard” objective test that can be used to categorise wheezy infants in order to compare the questionnaire against. There are many clinical phenotypes described in the literature but the different possible diagnoses are still to be clearly defined. Gold standards could be airway hyperresponsiveness (which have difficulty with sensitivity, specificity, and reliability in their own right20–22), breath sound analysis,23 or the consultant respiratory specialist opinion. We have used consultant opinion as a gold standard (although there is certain to be variability among them), to examine the initial qualities of the scales derived from the previous section of analysis. To determine whether the questionnaire could differentiate a child with no or minimal symptoms from a child with enough symptoms to be labelled as having asthma, scores for infants from the newborn cohort and those who had been seen by a respiratory physician and labelled as having asthma (RPBDA) were compared using the two sample t test.
Previous questionnaire studies have calculated reliability, internal consistency, and aspects of validity using between 40 and 160 subjects.8–10,12,14,24,25 Factor analysis needs to have a minimum of 100 subjects for adequate assessment.19 We thus aimed to recruit at least 50 patients with symptoms from the outpatient clinic and at least 50 patients from a newborn cohort. For more detailed assessment of criterion validity, many more subjects would be required.
The study had the approval of the local research ethics committee and the parents gave written consent prior to completion of the questionnaires.
Response rates and demographics
The initial response from the newborns recruited was 128/200 (64%). Repeat questionnaires were received from 72/128 (56%). All the 42 outpatients completed an initial and repeat questionnaire. Thus 114 pairs of questionnaires were analysed. Twenty of the outpatients had been diagnosed as having asthma. The mean age of the newborn cohort at the time of completion of the questionnaire was10 months and that for the outpatients was 13 months (range of the whole group was 6–35 months). There were 60 (53%) males.
Short term reliability
Most questions have a kappa score of greater then 0.4, which suggests moderate agreement; only one question on “noisy breathing from the throat”, a rather ill defined symptom, had a kappa score of 0.39 (95%CI 0.35 to 0.5), indicating only fair agreement.
Cronbach alpha scores for the four domains are as follows: daytime symptom score (19 components) 0.95, night time symptom score (five components) 0.80, impact of symptoms on family score (four components) 0.91, and impact of symptoms on child score (four components) 0.87.
For more detailed examination of construct validity, using factor analysis on the 32 questions involving a scaled response, six initial principle components were identified. Question 24, about snoring, was the sixth component (eigen value 1.2) accounting for 3.7% of the variance; questions 17, 18, and 19, about fast breathing and noises coming from the throat, was the fifth component (eigen value 1.33) and accounted for 4% of the variance. These questions were excluded prior to further factor analysis on the remaining 28 questions.
On examination of the questions four new scales are suggested with higher Cronbach alpha scores. Scale one refers mainly to cough and questions associated with impact of symptoms on the family and affect on feeding and waking in the child (cough and impact score). Scale two is mainly symptoms of a rattly chest and wheezing, possibly activity related (activity wheeze and rattle score). Scale three refers mainly to shortness of breath with symptoms reducing the child’s activity (short of breath score). Scale four, although the weakest correlations, suggests another wheezing scale associated with upper respiratory tract infections (URTI) and nocturnal wheezing with symptoms making the child tired (wheezing with URTI score).
Comparison of the replies from the 20 subjects who had been diagnosed as asthmatic by a respiratory specialist (RPBDA) was made with those from the 72 subjects identified from the neonatal population with no, or minimal symptoms. The table shows that all eight of the scales in the questionnaire will significantly differentiate a child with a physician diagnosis of asthma from a well child. Sensitivity and specificity for cut off scores for each of the scales, for differentiation between RPBDA and well children can also be calculated. Further detailed examination of the questionnaire with respect to criterion validity will require larger numbers of infants with different diagnoses.
We have developed a practical, acceptable respiratory symptom questionnaire for use in infants and preschool children. This had satisfactory response rates when administered to a cohort of postnatally recruited subjects and was easy to use in an outpatient population. Face and content validity were considered acceptable.
Short term reliability
We have shown that most questions have moderate to good short term reliability. We have calculated both weighted kappa scores (measure of agreement which allows for the possibility of chance agreement, but is affected by the prevalence of the positive responses) and average correct classification (a measure of repeatability which is relative to the prevalence of responses). These measures should both be calculated, to give complete information on repeatability; when the prevalence of a symptom is low, the kappa score may be low and the average correct classification is considered to be a more accurate estimate of the reliability.18 Only one question (about noisy breathing from the throat) had a kappa score of <0.41. This probably should not be used and indeed has been identified by the factor analysis as being non-contributory to the scales we have developed. The kappa levels obtained are similar to those in other questionnaires designed for older children, questions on wheeze usually being more reliable than questions on cough.24,26 As has also been noted previously questions concerning symptoms with colds and upper respiratory tract infections were less reliable than questions on symptoms without colds.24 All questions in our questionnaire had an average correct classification score above 0.7.
Four initial concept domains were designed and the internal consistencies for their scores were acceptable, with the “impact on family” and “daytime symptom” scores having excellent internal consistency. Essentially this means that each score has items, which are closely related and have good within-scale, between-item correlations. The level of internal consistency for these four domains is similar to that reported by Usherwood et al in their questionnaire designed for older children.14
Factor analysis and construct validity
To further refine these scores, principal component factor analysis was used. This is a powerful technique for exploring the item correlation during score validation.19 Varimax rotation was used to minimise the number of variables that have high loadings on each factor, thereby simplifying the overall structure.19 Initial analysis suggested that within the 32 items there were six possible areas of correlation. The question (number 24) on snoring was an outlier and appeared to have little correlation with the other factors. The questions about the noise from the throat (numbers 17 and 18) and fast breathing (number 19) were also outliers and were excluded from analysis. These exclusions resulted in four new domains.
The first new domain mainly concerns coughing symptoms (“cough and impact score”). It is interesting that most of the “impact on the family” questions closely correlated with these coughing questions. This suggests that parents may be more concerned about coughing as a symptom (particularly at night) than wheezing or shortness of breath. The second new domain focuses on wheeze associated with activity and having a rattly chest (“activity wheeze and rattle score”), and the third domain was characterised by shortness of breath (“shortness of breath score”). All three of these new domains had Cronbach alpha scores of 0.95 or greater which suggests excellent, within-scale, between-item correlations. The fourth new domain is characterised by wheeze associated with colds or upper respiratory tact infection (“wheeze with URTI score”). This had a lower Cronbach alpha score of 0.79, which is still acceptable. One could hypothesise that these four new domains represent four phenotypes of respiratory symptoms, in the infant and preschool child with wheeze and cough. The next stage is to use these four new scales in a larger cohort of symptomatic subjects to examine whether different diagnostic phenotypes show different scores within these domains. As far as our present study goes, the questionnaire has strong construct validity and internal consistency with the eight concept domains used.
Criterion validity was the hardest quality to assess as there is no objective measure or “gold standard” to compare with. We have coarsely assessed it using respiratory physicians based diagnoses of asthma (RPBDA) in an outpatient cohort. The eight questionnaire scales can significantly distinguish children with RPBDA from mildly symptomatic and well infants. Subjects with RPBDA had significantly higher scores on the eight scales compared to those who had minimal or no symptoms (no diagnosis of asthma; see table 1). We used ROC curves to examine the performance of each for the eight scales. All the scores had acceptable sensitivity (between 88.9% and 96.7%) for identifying RPBDA but some had unacceptably low specificity (such as the wheeze with URTI; 31.3%).
We have only studied a small cohort for the initial exploration of the questionnaire characteristics with a few subjects labelled as having asthma. Other questionnaires have used similar or smaller numbers for validation exercises.24,25 However we have been able to describe differences between those with asthma and those without. The major problem for any asthma and respiratory symptom questionnaire is not having a gold standard for examining criterion validity. Pragmatically we have used respiratory physician based diagnosis of asthma as the gold standard. Bronchial challenge tests could be used as the external measure for criterion validity. However, these tests have many problems with their own reliability and relation to clinical symptoms.20–22 More detailed study of criterion validity against a different gold standard, together with the responsiveness of the questionnaire is required, using a larger population and including children with the different phenotypes of wheezy illness.
Other than using the Flesh score, we have not attempted to examine the readability of the questionnaire in detail. Neither have we looked at the factors which may affect responses such as cultural background, language spoken at home, age, or gender of the person completing the questionnaire, relationship to the child, and level of education. These are clearly extremely important and need to be explored further.
We have developed a respiratory symptom questionnaire, for use in the infant and preschool child. It examines the impact these symptoms have on the child and family as well as exploring different symptom patterns. We have shown it to be acceptable and easily completed with good response rates. Face and content validity are acceptable. Most questions have good test–retest reliability. Eight symptom scores have been developed with high internal consistency and construct validity. These scores may reflect different phenotypes of respiratory symptoms within the infant and preschool child population. Using a respiratory physician’s clinical diagnosis of asthma as a gold standard we have shown good criterion validity for the eight scores. The next challenge in the development of the questionnaire is to use it on larger populations, to further explore the possible phenotypes suggested in this initial assessment, and to assess the responsiveness of the questionnaire to change in symptoms and clinical state within individuals.
We acknowledge all the families who have completed the questionnaires and the medical, nursing, and midwifery staff involved in their distribution and development.
A parent completed questionnaire to describe the patterns of wheezing
and other respiratory symptoms in infants and preschool children
C V E Powell, P McNamara, A Solis, and N J Shaw
Full tabular summaries of the questionnaire results
The Tables are available as downloadable PDFs (printer friendly files).
If you do not have Adobe Reader installed on your computer,
you can download this free-of-charge, please Click here
Table 1 Short-term reliability results for the 20 daytime symptom questions. Except for �ever wheezed�, questions refer to the last 3 months
Table 2 Short-term reliability results for the 5 �night �time� symptom, 4 �impact on child� and 4 �impact on family� questions.
Questions refer to the last 3 months
Table 3 Results of principle component analysis with varimax rotation on data from 28 questions (questions 17,18,19 & 24 excluded). Internal consistency of four domains presented as Cronbach alpha scores
Table 4 Main contributing items to the four identified components with corresponding correlation from factor analysis of 28 questions
Table 5 Sensitivity and specificity of scores for each of the original four domains. Figures in bold are the best cut off score to discriminate between doctor diagnosed asthma (n=20) and the neonatally recruited unlabelled infants (n=72)
Table 6 Sensitivity and specificity of scores for each of the four newly created symptom domains. Figures in bold are the best cut off score to discriminate between doctor diagnosed asthma (n=20) and the neonatally recruited unlabelled infants (n=72)