Screening for autism in preterm children: diagnostic utility of the Social Communication Questionnaire
- 1Research Department of Academic Neonatology, Institute for Women's Health, University College London, London, UK
- 2Division of Psychiatry, School of Community Health Sciences, University of Nottingham, Nottingham, UK
- 3Wolfson Institute of Preventive Medicine, Barts and The London School of Medicine and Dentistry, Queen Mary, University of London, London, UK
- 4Department of Psychology and Health Sciences Research Institute, Warwick Medical School, University of Warwick, Coventry, UK
- Correspondence to Dr Samantha Johnson, Institute for Women's Health, UCL, 86–96 Chenies Mews, London WC1E 6HX, UK;
- Accepted 30 August 2010
- Published Online First 27 October 2010
Objective Preterm survivors are at high risk for autism spectrum disorders (ASD). The diagnostic utility of the Social Communication Questionnaire (SCQ) in screening for ASD was assessed in extremely preterm children at 11 years of age.
Design All babies born at <26 weeks gestation in UK and Ireland from March through December 1995 were recruited to the EPICure Study. Of 307 survivors, 219 (71%) were assessed at 11 years. Parents of 173 children completed the SCQ to screen for autistic features and the Development and Well Being Assessment (DAWBA) psychiatric interview. A consensus diagnosis of ASD was assigned by two child psychiatrists following review of the DAWBA parental interview and corresponding DAWBA teacher questionnaire.
Setting Community-based follow-up.
Results Using the established SCQ cut-off (scores ≥15), 28 (16%) extremely preterm children screened positive for ASD. Eleven (6%) were assigned a diagnosis of ASD. Using this cut-off, the SCQ had 82% sensitivity and 88% specificity for identifying ASD in this population. Using a receiver operating characteristic curve, SCQ scores ≥14 had optimal diagnostic utility (area under curve: 0.94; sensitivity: 91%; specificity: 86%). Positive predictive value was relatively low (31%) resulting in numerous over-referrals. However, children with false positive screens had significantly worse neuro-developmental, cognitive and behavioural outcomes than those with true negative screens.
Conclusion The SCQ has good diagnostic utility for identifying ASD in extremely preterm children and is a useful screening tool in this population. Children with false positive screens represent a high-risk group in whom further diagnostic assessment would be beneficial.
Extremely preterm birth is associated with a high risk of functional disability later in life.1 While neuro-sensory impairments contribute to the range of residual disabilities observed, the most common adverse outcomes are cognitive impairment, behavioural problems and executive dysfunction.2,–,4 Awareness has also increased of a high prevalence of social and communication difficulties and autism spectrum disorders (ASD) in this population.5 6 Contemporary studies of very preterm survivors have reported positive screening for autistic features in 21–25% in infancy.7 8 Most recently, we have reported an increased prevalence of autism spectrum symptoms and 8% prevalence of ASD diagnoses in extremely preterm children at 11 years of age.9
What is already known on this topic
▶ Extremely preterm children are at high risk for autism spectrum disorders (ASD) in middle childhood.
▶ The specificity of screening for autism may be confounded by the high prevalence of neuro-developmental disability in this population.
What this study adds
▶ The Social Communication Questionnaire has good diagnostic utility and is a useful first line screening tool for ASD in extremely preterm children.
▶ Children with false positive screens are at high risk for other neuro-cognitive and behavioural impairments and further neuropsychological assessment would be beneficial.
The importance of screening for ASD in extremely preterm children is increasingly recognised for both clinical and research purposes. Although early intervention improves outcome, screening in infancy is confounded by the high prevalence of neuro-developmental delay in this population.7 10 Screening in middle childhood has greater discriminative validity and many extremely preterm children may present with emerging social and behavioural difficulties prompting the need for screening and assessment at this age. The prevalence of autistic features is also increasingly included as an outcome measure in many epidemiological studies for which cost and time efficient measures are required.
The Social Communication Questionnaire (SCQ)11 has been validated for use in middle childhood and has good discriminative validity for identifying children with ASD in clinical samples,12 13 those with special educational needs14,–,16 and the normal population.14 However, the efficacy of the SCQ in identifying children at risk for ASD has not been investigated in an extremely preterm population in which the high rate of neuro-developmental disability may diminish the predictive validity of such measures. Our aim was therefore to investigate the diagnostic utility of the SCQ in identifying ASD in extremely preterm children in middle childhood.
All babies born at ≤25 weeks gestational age in the UK and Ireland from March through December 1995 were recruited to the EPICure Study, a prospective whole-population study of outcome following extremely preterm birth. Of 307 survivors at 11 years of age, 11 (4%) lived outside the study area, the parents of 57 (19%) declined consent and 18 (6%) did not respond to study invitations to participate. The remaining 219 children (71% of survivors) were formally assessed at 11 years of age (median 131 months; range 121–145 months). Drop-out analyses revealed that children not assessed (n=89) at 11 years were more likely to be born at 25 weeks to unemployed parents of non-white ethnic origin and have more frequent cognitive impairment at 2.5 and 6 years than those assessed (n=219). Detailed drop-out analyses have been published previously.3
The SCQ (lifetime form)11 is a 40-item parent report based on the Autism Diagnostic Interview-Revised. Item scores are summed to yield a total SCQ score (range 0–39 for verbal and 0–33 for non-verbal children) with higher scores indicating a greater frequency of symptoms. Missing values were pro-rated (if ≤3 items were missing on the Social Interaction and Communication subscales and ≤2 items on the Repetitive Behaviour subscale; n=16, n=23, n=8, respectively). SCQ scores were compared to established cut-offs for screening for ASD (scores ≥15) and for more narrowly defined autistic disorder (scores ≥22).11 12
To identify children with ASD diagnoses, parents completed the Development and Well Being Assessment (DAWBA),17 a diagnostic interview for childhood psychiatric disorders. Parents were interviewed over the telephone (88%) or completed the DAWBA online (12%). Supplemental information was provided by teachers who completed a corresponding questionnaire-based version of the DAWBA, an approach previously evaluated in a large community study.18 Data were entered into an electronic database and scoring algorithms were used to yield computer-generated diagnoses. These computer-generated summary sheets and detailed transcripts of parental descriptions of the child's past and current levels of social and communicative behaviours and play/activities/routines were reviewed by two child and adolescent psychiatrists who assigned clinical consensus diagnoses based on all available information. The following DSM-IV-TR19 diagnoses in the broad category of ASD were assigned: autistic disorder, Asperger disorder, Rett syndrome, childhood disintegrative disorder and pervasive developmental disorder-not otherwise specified.
Children were also assessed using the Kaufman-Assessment Battery for Children,20 a standardised IQ test that yields a Mental Processing Composite (MPC) score (mean 100; SD 15). Cognitive impairment (scores <−2 SD) was classified using the mean (SD) of a comparison group of 153 classmates also assessed as part of the EPICure Study3 in order to account for the secular increase in IQ scores over time.21 Functional neuromotor, hearing and visual impairment was also classified using a standard paediatric evaluation, and overall functional disability was classified using the child's rating in each of the four domains (cognition, hearing, vision, motor). Definitions of functional disability and related outcomes are detailed elsewhere.1
Parents completed the Strengths and Difficulties Questionnaire (SDQ)22 to screen for other behavioural and emotional disorders. SDQ scores >90th percentile of the comparison group were used to identify clinically significant emotional symptoms, conduct problems, attention/hyperactivity, peer problems and total difficulties.
Parents and children received study information sheets and informed consent was provided by parents. Children were assessed by a paediatrician and psychologist. Parents completed questionnaires approximately 1 week prior to the child's assessment. The DAWBA was completed subsequently. Psychologists simultaneously scored standardised tests from which excellent inter-rater reliability was achieved: >95% agreement in test items. The study was approved by the Southampton and South West Hampshire Research Ethics Committee.
Data were double-entered and analysed using SPSS and Stata. Differences between children with and without complete SCQ and DAWBA data were analysed using independent samples t tests for continuous outcomes and Fisher's exact tests for categorical outcomes. To determine diagnostic utility of the SCQ, rates of ASD diagnoses and positive SCQ screens were cross-tabulated and agreement assessed using Cohen's κ. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) with 95% CIs were calculated and a receiver operating characteristic (ROC) curve was constructed to identify an SCQ score with optimal diagnostic utility.
Both SCQ and DAWBA data were obtained for 173 (79%) children assessed at 11 years (mean age 130 months; range 121–144 months). Of the 46 children with no or incomplete data (non-responders), the parents of 28 completed the DAWBA only, 10 the SCQ only, and eight did not complete either measure. Non-responders were significantly more likely to have serious neuromotor (OR 3.26; 95% CI 1.28 to 8.32) or visual (OR 3.10; 1.17 to 8.24) impairment, functional disability including cognitive impairment (OR 2.04; 1.06 to 3.95) and lower IQ scores (mean difference −9.6 points; 95% CI −16.4 to −2.9; table 1).
The mean total SCQ score was 7.99 (SD 7.51; range 0–35; n=173). Using the established cut-off score (≥15), 28 (16.2%) children screened positive for ASD (figure 1). Boys were significantly more likely than girls to screen positive for ASD (23.1% vs 10.5%; OR 2.55; 95% CI 1.10 to 5.91), and children with cognitive (28.6% vs 9.1%; OR 4.00; 95% CI 1.71 to 9.35), neuromotor (41.7% vs 14.3%; OR 4.29; 95% CI 1.25 to 14.66) and visual impairment (48.5% vs 14.2%; OR 5.04; 95% CI 1.42 to 17.87) were significantly more likely to screen positive than those without functional impairment.
Overall, 11 (6.4%) extremely preterm children received an ASD diagnosis. Of these, nine were diagnosed with autistic disorder and two with atypical autism. No children were diagnosed with Asperger syndrome or other ASD. Prevalence and correlates of ASD diagnoses for the whole population are published elsewhere.9 23
Mean SCQ scores were significantly higher for children with ASD diagnoses (n=11; mean 23.27; SD 8.32) than those without (n=162; mean 6.85; SD 6.22; mean difference 16.43 points; 95% CI 12.51 to 20.34; figure 1). Agreement between rates of positive SCQ screens and ASD diagnoses were cross-tabulated and predictive values calculated using various SCQ cut-off scores. Using the established SCQ cut-off (≥15), there was a significant association between positive SCQ screens and ASD diagnoses (κ=0.41; 95% CI 0.21 to 0.50; p<0.001). While sensitivity (0.82; 95% CI 0.48 to 0.98) and specificity (0.88; 95% CI 0.82 to 0.93) was high, PPV was relatively low (0.32; 95% CI 0.16 to 0.52) indicating a large number of false positives: 11% of all children (table 2).
We examined the predictive value of using a cut-off score of ≥22, usually applied for discriminating between autistic disorder and other ASD, as the default cut-off in this population. Using this cut-off, there was significant agreement between positive SCQ screens and ASD diagnoses (κ=0.53; 95% CI 0.25 to 0.74; p<0.001). However, while PPV was marginally improved (0.50; 95% CI 0.23 to 0.77) this was at the expense of test sensitivity (0.64; 95% CI 0.31 to 0.89; table 2). The number of false positive classifications was reduced by almost two-thirds, but four of the 11 children with ASD diagnoses (57%) now had false negative SCQ screens and would therefore not be identified as at-risk on screening alone.
An ROC curve was constructed to identify an SCQ cut-off score with maximal predictive value in this population. The area under the curve (AUC) was 0.94 and a cut-off score ≥14 had optimum diagnostic utility (κ=0.41; sensitivity 0.91, 95% CI 0.59 to 0.99; specificity 0.86, 95% CI 0.80 to 0.91). PPV remained relatively low (PPV 0.31; 95% CI 0.16 to 0.50), but sensitivity was maximised and all but one child with an ASD diagnosis screened positive on the SCQ.
In high-risk populations, children with false positive scores may be at increased risk for other neuro-developmental impairments, behavioural problems and socio-communication difficulties. Therefore, neuro-developmental outcomes were compared between children with false positive and true negative screens using the ROC-determined cut-off of ≥14 (table 3). Children with false positive screens had significantly lower MPC scores than children with true negative screens (−19.46 points; 95% CI −28.32 to −10.59; p<0.001) and were significantly more likely to have functional disabilities (OR 4.11; 95% CI 1.57 to 10.76) including neuromotor, visual and cognitive impairments (table 3). Children with false positive screens were also significantly more likely to have parent-reported behaviour problems overall (OR 8.00; 95% CI 2.90 to 22.04) and in each of the four domains of emotional, conduct, attention and peer problems (table 3).
Excluding children with serious functional disabilities, those with false positive screens had significantly lower MPC scores (mean difference −7.55 points; 95% CI −14.55 to −0.55) and were more likely to have emotional (OR 20.50; 95% CI 3.51 to 119.92; p=0.001), conduct (OR 13.05; 95% CI 2.27 to 74.90; p=0.011), attention/hyperactivity (OR 11.88; 95% CI 2.11 to 66.73; p=0.004) and peer problems (OR 26.47; 95% CI 2.99 to 234.52; p=0.001) than those with true negative screens.
Recent reports of a high prevalence of ASD in extremely preterm children9 highlight the need for screening for social and communication difficulties in these survivors. This study shows that the SCQ has good diagnostic utility for identifying ASD in extremely preterm children.
The mean SCQ score (mean 8; SD 8) and rate of positive screens in this study were higher than those reported for normal population samples14 24 and reflect the generally higher level of autism spectrum symptoms in extremely preterm survivors.9 25 Using the established cut-off score for identifying ASD (≥15), the SCQ had high sensitivity (82%) and specificity (88%) exceeding standards required for screening tests.26 These values are comparable with or exceed estimates reported for diagnostic utility of the SCQ in clinical samples12 13 and in children with special educational needs14,–,16 in middle childhood, and far exceed values reported for diagnostic utility of the SCQ in younger children.27,–,29 Construction of a ROC curve produced a high AUC (0.94) and a total SCQ cut-off score of ≥14 was found to have optimal diagnostic utility maximising both sensitivity (91%) and specificity (86%) in this population.
Assessing sensitivity and specificity alone can be misleading. In practice, professionals may have screening test results alone on which to base decisions for referral and thus SCQ predictive values should be considered. In this population, NPV were consistently high (≥98%), thus assuring the clinician or researcher that almost all children with negative screens would fail to meet diagnostic criteria for ASD. However, PPV were relatively low: 31–32% of children with positive screens received an ASD diagnosis resulting in a high rate of false positive screens (13% of all children screened). A high rate of false positives is not uncommon in behavioural screening in which PPVs of 30–50% are often reported.26 30 SCQ scores are elevated in children with learning disabilities and behavioural problems in the absence of diagnosed ASD,12 31 and such disorders are among the most prevalent adverse outcomes associated with extreme prematurity.2 3 This study shows that children with functional disabilities are four times more likely to screen positive on the SCQ than their extremely preterm peers. Parents who lack a conceptual understanding of the nature and aetiology of ASD may be unable to discriminate these symptoms and may rate SCQ items as positive based on behaviours associated with other neuro-developmental sequelae.15
Over-referrals are considered a negative consequence of screening if the cost of further diagnostic assessment is high and confers no benefit. However, routine screening is likely to be cost effective in this population as those with false positive scores were four to six times more likely to have cognitive or neuro-sensory impairment, and eight times more likely to have parent-reported behaviour problems than those with true negative screens. Even after exclusion of children with serious neuro-sensory disabilities, those with false positive screens were over 20 times more likely to have anxiety or peer problems. Thus those with false positive screens are a group in whom further diagnostic assessment and psychiatric referral would be beneficial.
Our analysis of the characteristics of non-responders showed that parents of children with functional disabilities, particularly physical and neuro-sensory impairments, were less likely to complete the SCQ. Such parents may feel that some items are not applicable or are difficult to answer when trying to isolate autistic features from other neuro-developmental sequelae. Professionals using the SCQ with extremely preterm cohorts should be aware of this response bias.
The strengths of the present study lie in the investigation of the efficacy of the SCQ in a whole-population-based cohort of extremely preterm children including rigorous assessments of neuro-psychological outcomes. Psychiatric interviews were carried out for all children in the study using the DAWBA and consensus diagnoses were made by two experienced child psychiatrists using information gained in the DAWBA including parent and teacher descriptions of behaviour. The DAWBA is a well-established diagnostic interview that was used as the principal measure of childhood psychopathology in the British Mental Health Surveys32 and as a specific diagnostic measure in population-based prevalence studies of ASD,33 34 generating prevalence estimates comparable to other ASD diagnostic measures. The validity of the DAWBA diagnosis of ASD is also supported by evidence of good agreement (κ=0.75) between DAWBA diagnosis and the Diagnostic Interview for Social and Communicative Disorders.35 Parents completed the SCQ prior to the DAWBA so that that questionnaire responses would be unaffected by participation in the interview. SCQ results were not affected by chronological age as reflected by the lack of a significant correlation between SCQ scores and age at assessment (r=0.029; p=0.705; n=173). Given the association between age and SCQ scores,13 the diagnostic utility of the SCQ in this population may differ if used at other ages. This warrants further investigation.
The SCQ has good diagnostic utility for identifying extremely preterm children with ASD and is an effective method of first-level screening in this population. Children with false positive screens are at high risk for other neuro-behavioural sequelae and thus further diagnostic assessment and educational planning is warranted. The SCQ is a useful screening tool and a cost effective outcome measure for assessing autistic spectrum symptomatology in extremely preterm children.
The authors are indebted to the EPICure Study Group, which includes paediatricians in 276 maternity units in the UK and Ireland who identified the cohort, contributed perinatal data and whose help was invaluable. We are also indebted to the children and parents for their continued participation in the EPICure Study. Data collection was conducted by Joe Fawke, Sue Thomas, Vicky Rowell, Rebecca Smith, Rebecca Trikic and Samantha Johnson. Support for DAWBA assessments was provided by Robert Goodman (London). Co-investigators for the EPICure Studies were Neil Marlow (UCL; Principal Investigator), Kate Costeloe (Queen Mary, University of London), Enid Hennessy (Queen Mary, University of London), Janet Stocks (UCL) and Elizabeth Draper (University of Leicester). The EPICure study website can be viewed at www.epicure.ac.uk.
Funding This study was funded by the Medical Research Council (MRC), UK. Neil Marlow receives a proportion of funding from the Department of Health's NIHR Biomedical Research Centres funding scheme at UCLH/UCL.
Competing interests None.
Ethics approval This study was conducted with the approval of the Southampton and South West Hampshire Research Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.