Aim: To quantitatively examine the influence of study methodology and population characteristics on prevalence estimates of autism spectrum disorders.
Methods: Electronic databases and bibliographies were searched and identified papers evaluated against inclusion criteria. Two groups of studies estimated the prevalence of typical autism and all autism spectrum disorders (ASD). The extent of variation among studies and overall prevalence were estimated using meta-analysis. The influence of methodological factors and population characteristics on estimated prevalence was investigated using meta-regression and summarised as odds ratios (OR).
Results: Forty studies met inclusion criteria, of which 37 estimated the prevalence of typical autism, and 23 the prevalence of all ASD. A high degree of heterogeneity among studies was observed. The overall random effects estimate of prevalence across studies of typical autism was 7.1 per 10 000 (95% CI 1.6 to 30.6) and of all ASD was 20.0 per 10 000 (95% CI 4.9 to 82.1). Diagnostic criteria used (ICD-10 or DSM-IV versus other; OR = 3.36, 95% CI 2.07 to 5.46), age of the children screened (OR = 0.91 per year, 95% CI 0.83 to 0.99), and study location (e.g. Japan versus North America; OR = 3.60, 95% CI 1.73 to 7.46) were all significantly associated with prevalence of typical autism. Diagnostic criteria, age of the sample, and urban or rural location were associated with estimated prevalence of all ASD.
Conclusions: Sixty one per cent of the variation in prevalence estimates of typical autism was explained by these models. Diagnostic criteria used, age of children screened, and study location may be acting as proxies for other study characteristics and require further investigation.
- pervasive developmental disorders
- systematic review
Statistics from Altmetric.com
The prevalence of autistic disorder is now considered to be around 10 per 10 000, and the prevalence of pervasive developmental disorders, 27.5 per 10 000. These are derived from studies which have estimated prevalences of autistic disorder ranging from 0.7 to 72.6 per 10 000.1 An increase in prevalence estimates has been observed over time, the reasons for which are not clear and may include: changes in study methodology; a genuine rise in autism risk factors; increase in services available, including diagnostic; increased awareness among educational and clinical professionals; and growing acceptance that autism can coexist with a range of other conditions.1–4
True variation in prevalence could generate aetiological hypotheses for autism and it is vital to understand what underpins the variation. Accurate estimates of the true prevalence are of value in planning diagnostic and intervention services.
Several narrative reviews have been conducted. This paper uses systematic and quantitative methods to examine reasons for variation in prevalence estimates. The aims are to assess the degree of variation among prevalence studies of autism, and to provide an overall summary of prevalence diversity taking into account among-study variance using meta-analysis. Aspects of study methodology and population characteristics are then examined using meta-regression to investigate their influence on prevalence estimates.
Two databases, MEDLINE and EMBASE, were systematically searched by the first author (box 1). In addition, bibliographies of previous reviews1,5,6 were examined to identify published prevalence studies.
Box 1: Search strategy for identifying prevalence studies
MEDLINE (PubMed) (searched 13/04/04)
(“Autistic-Disorder”/all subheadings [MeSH†] OR “Asperger-Syndrome”/all subheadings [MeSH] OR “Schizophrenia-Childhood”/all subheadings [MeSH] and (PY = 1966–1970) OR autis* (free text term)) AND (“Prevalence-“/all subheadings [MeSH] OR “Cross-Sectional-Studies”/all subheadings [MeSH] OR “Mass-Screening”/all subheadings [MeSH] OR “Multiphasic-Screening”/all subheadings [MeSH]).
EMBASE (Excerpta Medica Database) (searched 13/04/04)
(BIDS EMBASE, via Ovid, copyright 2003)
(exp‡ autism/OR exp infantile autism/ OR exp Asperger syndrome/ OR autism.mp§ (as keyword) OR Asperger.mp (as keyword)) AND (exp prevalence/ OR exp mass screening/ OR exp screening/ OR cross-sectional.mp (as keyword)) NOT (genetic screening/exp OR genetic screen.mp (as keyword).
Identified papers were examined against criteria for inclusion (box 2). The paper itself was examined if the abstract was insufficiently clear. Where there was more than one paper published on a particular study, the most recent was included in the review.
Box 2: Inclusion criteria
A geographically and temporally defined population
Cross-sectional study or data, or first phase of a longitudinal study
Defined diagnostic criteria stated for autism or autism spectrum disorder
Includes individuals under 18 years old
Initial selection in a wide range of children in the general population, or in a clinical setting
Final identification of cases based on clinical or other diagnostic assessment of selected children
Published in English, or with detailed summaries in English
Peer reviewed paper or conference presentation
Includes prevalence data
(criteria adapted from Wing7)
Methods and population characteristics reported across most studies were selected for data extraction. The first author extracted and coded the data. In studies using different diagnostic criteria, prevalence data based on the more recently published diagnostic criteria were extracted. The studies formed two groups: those that assessed the prevalence of classic autism, or autistic disorder, known here as “typical autism”; and those that assessed the prevalence of autism spectrum disorders (ASD) or all pervasive developmental disorders, known here as “all ASD”. Assessments of risk of bias included reporting of refusal rates and the reliability of screen and assessment procedures.
In the basic tables, crude prevalence estimates (number of cases/sample size) were presented, along with standard errors. For all meta-analyses and meta-regressions, prevalence estimates were transformed to logits to improve their statistical properties. These were later back-transformed to prevalences and expressed as cases per 10 000 people.
Description of heterogeneity among studies and summary of prevalence
Forest plots8 were used to visualise the extent of heterogeneity among studies. Two statistical methods were used to quantify the variation. A standard test for heterogeneity examined the null hypothesis that the true prevalences are identical in every study. Since heterogeneity was expected a priori, this was supplemented with a measure of the degree of inconsistency across studies, I2.9 I2 describes the proportion of variation in prevalence estimates that is due to genuine variation in prevalences rather than sampling error. It is expressed as a percentage, with 0% indicating consistency.
The random effects model assumes the study prevalences follow a normal distribution, allowing for among-study variation.10 The usual confidence interval for the mean in the random effects model does not take among-study variance into account, so is deceptively narrow when there is substantial variation across studies. Instead, a 95% interval for the true prevalence was calculated as the mean of logits ± 1.96 τ, where τ is the among-study standard deviation.11
Investigation of sources of heterogeneity
The potential influence of covariates on the prevalence estimates was investigated using a random effects regression model, thus taking account of among-study variance, using the metareg command in STATA.12 The regression coefficients represent log odds ratios, which are presented as odds ratios with 95% confidence intervals.
A multivariate meta-regression model was constructed to investigate which covariates were associated with prevalence estimates if there was adjustment for other study covariates. The fit of each model was assessed using the percentage of among-study variance explained ((1−(τ2 in model/τ2 in model with no covariates))×100), together with a significance test for each introduced variable (T = coefficient/SE, related to the t-distribution). The models were constructed using a forward stepwise procedure as described in the results section. For each model a maximum number of covariates was set at n/10 where n was the number of studies, following standard recommendations for model size relative to sample size.13
Literature searches identified 670 papers (including duplicates). After exclusion through comparison of titles and abstracts against inclusion criteria, 77 papers were identified for detailed examination. Thirty seven papers were excluded, including ten on the basis of inclusion criterion 2 (box 2), ten on the basis of criterion 4, nine on the basis of 6, and one on the basis of 7. In addition, four papers did not have detailed English summaries, one was not peer reviewed, and two were untraceable. Of these seven potentially eligible studies, four were conducted in Japan, one in the USA, one in France, and one in Sweden.
Forty papers met inclusion criteria, of which 37 gave estimates for typical autism and 23 for all ASD (table 1). The study sample sizes ranged from 826 to 4 590 333 (median = 48 705). Only 17 (40%) studies reported the refusal rate at the screen phase of the study, and 13 (33%) at the assessment stage. Six (15%) studies reported investigating the reliability of their screen method, and 11 (26%) studies stated that the inter-rater reliability for the diagnostic assessment had been investigated. Many studies did not report refusal rates and reliability, so these covariates could not be included in further analyses.
Description of heterogeneity among studies and summary of prevalence
There was clearly wide variation in the prevalence estimates of typical autism (fig 1) and an increase in prevalence estimates over time. The Q statistic was very large (Q = 1947.6, df = 36, p < 0.001; I2 = 98.2%), showing that there was a great deal of variation among the studies. There was also a high degree of heterogeneity among estimates of all ASD (fig 2) (Q = 1577.7, df = 22, p < 0.001; I2 = 98.6%).
The back-transformed mean of the random effects distribution for studies of typical autism was 7.1 per 10 000 (95% interval for true prevalences: 1.6 to 30.6) and for studies of all ASD was 20.0 per 10 000 (95% interval for true prevalences: 4.9 to 82.1).
Investigation of sources of heterogeneity
Two studies were excluded from the regression analyses: either the published paper was not available,16 or insufficient information on study methodology was included in the English abstract of a Swedish paper.35
Studies of typical autism
The associations between study covariates and prevalence estimates of typical autism from univariate meta-regression analyses are shown in table 2. Taking account of the age of the children, for example, explained 23% of the among-studies variance.
Diagnostic criteria and decade of publication were the covariates that explained the most variance among studies in the univariate analyses. These two covariates are collinear and it was not possible to include both in a multivariate model. The diagnostic criteria used were entered first into the multivariate model since this was considered to be more directly related to variation in prevalence estimates than decade of publication, which is a proxy for all time varying covariates. The binary categorisation of diagnostic criteria was used, as it was not possible to use multiple categories of diagnostic criteria in a multivariate analysis with so few studies. Age of the children screened also explained much among-study variance, and was entered next into the model.
Models with three covariates were constructed which included age, diagnostic criteria, and each remaining covariate in turn. Screening method used, and whether the study was on a population or clinic based sample, were not significantly associated with prevalence. Urban location gave rise to higher prevalence estimates than studies carried out in rural or mixed locations (OR = 1.90, 95% CI 1.10 to 3.25; among-study variance explained = 53%). Studies that drew on records of previous diagnostic assessments resulted in lower prevalence estimates than those which included a prospective diagnostic assessment (OR = 0.57, 95% CI 0.33 to 0.96; variance explained = 53%). Including region of study provided the model that explained the most among-study variance (variance explained = 61%) (table 3). In this final model, using ICD-10 or DSM-IV led to prevalence estimates three times those using other diagnostic criteria. The odds ratio for age was 0.91 (95% CI 0.83 to 0.99), showing that an increase of one year in the age of the children screened led to a significant reduction in prevalence estimates. For example, when the odds ratio is taken to approximate a relative risk, if prevalence was estimated to be 10 per 10 000 in a sample of 5 year olds, it would be expected to be around 9.1 per 10 000 in a sample of 6 year olds. Studies in Japan gave rise to prevalence estimates that were 3.6 times those in North America.
Studies of all ASD
The associations between study covariates and prevalence estimates of all ASD from univariate meta-regression analyses are shown in table 4. Only three covariates were significantly associated with the prevalence estimates: age of the children screened, urban or rural study location, and the diagnostic criteria used. The screen method used was of borderline significance. Of these, diagnostic criteria explained most among-study variance, and was therefore included in the multivariate analyses. Each of the other covariates was introduced into the model in turn to form models with two covariates. As in the analyses of studies of typical autism, as decade and diagnostic criteria were collinear, only the covariate for diagnostic criteria was included in further analyses. When adjusting for diagnostic criteria, the only covariates that were significantly associated with the prevalence estimates were the age of the children screened (variance explained = 50%) and urban or rural study location (variance explained = 53%). Both these models are presented (table 5). Using ICD-10 or DSM-IV gave rise to prevalence estimates that were over twice those in studies using other diagnostic criteria. When including age in the model, an increase in the age of the sample by one year was associated with a fall in prevalence by a factor of approximately 0.85, taking the odds ratio as an approximation of a relative risk. Alternatively, when including study location, studies in urban areas gave rise to prevalence estimates over 2.5 times those in rural or mixed urban and rural areas.
As expected, a large amount of variation in prevalence across studies was found by graphical representation of estimates and by indices of heterogeneity. Despite this wide variation, pooled estimates are useful to indicate the public health burden of the disorder. The study variation is reflected in the very large intervals on the summaries of overall prevalence. The estimates of around 7.1 per 10 000 for typical autism, and 20.0 per 10 000 for all ASD are slightly lower than those estimated previously at 8.7–10.0 per 10 000 and 27.5 per 10 000 respectively.1,3
The covariate most strongly associated with prevalence estimates for typical autism and all ASD was the diagnostic criteria used. This association has been recognised previously.2–4 The time variation in prevalence is so closely linked to changes in diagnostic criteria, the two could not be examined separately. Furthermore, it was not possible to account entirely for the effect of the diagnostic criteria on the prevalence estimates as the ICD-10 and DSM-IV diagnostic schema leave some scope for variation in their interpretation and application.
The age of the children screened was strongly associated with the prevalence estimates. Manifestations of ASD may be more obvious in younger children. Alternatively, some screening methods may be more sensitive for younger children. Methods of screening were found to be significantly associated with the prevalence estimates in the univariate analyses of typical autism, but not after adjusting for the age of the children screened.
The multivariate model that explained most among-study variance in studies of typical autism included the region studied, with studies from Japan having significantly higher estimates than North American studies. This could be due to other study factors. For example, a higher proportion of the Japanese studies were from urban areas (4/7 (57%) studies) compared to those in North America (1/6 (17%) studies). All the Japanese studies used prospective diagnostic assessments, and all but one drew on whole population rather than clinical samples. Due to the imposed limit of three covariates in the model, it was not possible to adjust for further potential effect modifiers. Countries differ in their diagnostic practice both in their theoretical background and their training procedures for healthcare workers. This may, in part, account for between-region variation in prevalence.
In an alternative model for typical autism, when adjusting for age and diagnostic criteria, studies including prospective diagnostic assessments gave rise to higher prevalence estimates than those using retrospective records. This may be linked to the use of different diagnostic methodology at different times. Alternatively, an assessor taking part in prospective research studies might observe children more closely for symptoms of ASD.
When adjusting for diagnostic criteria, urban location was also observed to be associated with higher prevalence estimates for both typical autism and all ASD. If the screen method relied on records, these may have been more complete in urban locations. If the screen method used referrals from clinicians, it is possible that a higher proportion of children were known to services in urban locations. There may have been different diagnostic practices in urban locations where staff were more likely to be employed at specialist healthcare centres than in rural locations. It is easier to access the population in urban locations, and response rates may have been higher, but data on response were too limited to investigate this.
Limitations and recommendations for future research
Publication bias was not investigated in this review, as funnel plots were not considered appropriate due to the large degree of variation across studies. It is unlikely that the set of papers published is biased with respect to prevalence reported. However, it is possible that some studies were not identified in the searches if they were not published in mainstream journals. There may have been some time lag bias, with smaller studies, or studies with unremarkable results, coming through to publication slower than larger studies.
Of the papers identified for detailed examination, five potentially eligible studies were excluded as they did not have a detailed English summary or were not peer reviewed. There is no reason to suspect that the lack of availability of data from these studies is a direct consequence of the prevalences they might have observed.
The choice of coding of the covariates may have affected the model, such as using the midpoint of the age range or grouping diverse diagnostic criteria. Furthermore, it was only possible to assess the impact of reported covariates, or easily quantifiable covariates. Qualitative influences on prevalence such as awareness of autism in each population could not be included. As more studies are published, it may be possible to include new covariates or more precise coding of existing covariates in such a model. It would be valuable to have even more thorough recording of study characteristics in future studies to facilitate meta-analyses of studies.
It is unlikely that it would ever be possible to measure and record all potentially important covariates. An alternative approach to investigating trends in prevalence, through ongoing monitoring of defined school aged populations using standard methodology, has been recommended.1 This would enable researchers to investigate changes in prevalence over time, and geographical variations while controlling for study methodology.
This review has contributed to explaining some of the influences on variation among prevalence estimates. Over half of the variation among study estimates can be explained by the age of the children screened, the diagnostic criteria used, and the country studied. Other important factors were whether the study was in a rural or urban location and whether cases were assessed prospectively or retrospectively. The impact of these identified factors on prevalence estimates should now be further investigated as they may be acting as proxies for other influences on prevalence. For example, the effect of geographical location on prevalence may be due to the services available, or variation in awareness of the disorder. By taking this quantitative approach, this review has shown that using meta-analytic techniques can be a valuable additional tool in deepening our understanding of the influences of study and population characteristics on variation in prevalence estimates in autism spectrum disorders.
This work forms part of a PhD thesis entitled “Screening for autism spectrum disorders” (University of Cambridge, 2003). Jo Williams (née Johnson) held an MRC studentship, and subsequently received funding from the Shirley Foundation. We wish to thank Prof Patrick Bolton and Prof Simon Baron-Cohen for their comments on an earlier draft of this paper.