Tuesday, October 27, 2009

Unemployment Among Adult Autistics in the UK

I've previously criticized estimates of the "costs of autism" to society. The whole rationale of coming up with such estimates is objectionable to begin with, but I've also criticized what appear to be exaggerations in the figures that form the basis of these analyses.

One key component of such estimates is "lost productivity" due to lack of employment and related metrics. I've pointed out that old data on the employment rate of adult autistics no longer applies. If you want to come up with cost estimates based on an ASD prevalence of 0.6% or 1%, you have to know the employment rate that corresponds to the ASD criteria that results in said prevalence. You can't just look at diagnosed adult autistics, and hope they are representative of all adult autistics, especially considering autism has been very much under-recognized in the past.

Of course, data that addresses this objection was not available until recently. That did not stop me from coming up with a guesstimate, though. I suggested that perhaps 70% of autistic children diagnosed today might end up being employable a generation from now.

I think it was clear in the blog post that this was a rough estimate extrapolated from other trends. The estimate, predictably, was met with accusations that I trivialize the challenges of autistics and so forth, specifically from Jon Mitchell. It seems that whenever I do some math or look up figures in the literature, I'm automatically engaging in trivialization.

Jon was of course "mystified" as to how I came up with the 70% figure, and implied that it's unbelievably high. Whenever he wants to snipe at me, this is the one thing Jon likes to bring up. He also proclaims I'm "one of the more prominent ND bloggers" so he gets to criticize not just me, but he probably thinks he's criticizing the entire neurodiversity movement in the process.

Readers are probably aware by now that the UK National Health Service recently released its first report on a prevalence study of autism in adults living in private households. The report includes some data on employment, but all the report says about it is this:
No significant variation in rate of ASD by economic activity status was found.

That's interesting enough by itself, but let's look at data from Table 2.4. There are 3 groups of persons considered in the study: Those in employment, those who are unemployed, and those who are economically inactive. The following is the prevalence of ASD in each group:

In employment: 0.9%
Unemployed: 1.6%
Economically inactive: 1.5%

The report also gives a "base" population, which consists of all persons from phase 1 who had been assigned a near-zero probability of ASD. You may recall that 7461 respondents were interviewed in phase 1. The base population consists of only 5998 persons, divided as follows:

In employment: 4492
Unemployed: 291
Economically inactive: 1215

The base population is not exactly representative of the general population (it's more "non-autistic" than normal if you will) but it will have to do. I will use it to estimate rates of employment and unemployment in autistics and non-autistics, as shown in the following table.

In EmploymentUnemployedEconomically Inactive

There are some differences, but the study authors didn't find them to be statistically significant. In any case, it would seem that adult autistics who live in private households across the UK are largely productive individuals who contribute to the economy in a manner similar to their non-autistic peers.

Of course, this particular report doesn't tell us about autistics who live in "communal establishments." I've suggested that when all is said and done, pooled prevalence of ASD might turn out to be around 1.3%. Even if all autistics who live in communal establishments are considered unemployed, total unemployment among autistics might be around 35%.

Thursday, October 08, 2009

Is It More Like 1.2% to 1.5%?

Over two years ago I wrote a post titled Moving Toward a New Consensus Prevalence of 1% or Higher. At the time the prevalence of ASD was generally considered to be 0.6%. If you Google it, you'll find this figure is still the one that's cited most frequently. At present, no one has come out and precisely said the consensus prevalence has been revised to 1%, but I think that's pretty much where we're at. Consider what Roy Richard Grinker said recently to Time Magazine.
"It provides what scientists call convergent validity: no matter how you shake the bushes, you come up with this 1%," says Richard Roy Grinker, an autism researcher at George Washington University who has worked to determine ASD prevalence in South Korea.

Two years ago, and even long before this – as suggested by Lorna Wing – there were already some indications in the literature that 1% might be a more accurate figure. It was not a lucky guess or anything like that. My concern, which came to pass, was that as study methodologies evolved and awareness improved, the public would be frightened by what might appear to be a real increase in the prevalence of autism, with no end in sight. Ms. Clark (you remember her, right?) speculated that the process might be very gradual, with people getting increasingly surprised, and finally collapsing from exhaustion. It turned out to be a little more quick and relatively painless, at least so far.

I believe the question at this point is whether we've reached a plateau. Is prevalence going to stabilize at 1%? Have identification methods been pushed to their limits? Is the level of awareness (important in phone surveys and passive systems) the highest it's ever going to be?

I think that's possible, provided the same criteria and diagnostic tools continue to be used. But I gotta tell you, I have some reasons to believe we haven't seen the end of this just yet – not very many reasons, admittedly, but some. We might find that a more accurate number for prevalence is a little bit higher, perhaps 1.2%, or even 1.5%.

I'm not going to discuss cultural trends or the inherent subjectivity of psychiatric diagnoses, simply because this would be difficult, if not impossible, to quantify. But I do want to discuss case finding in this post.

What brings this about is the recent NHS prevalence study of autism among UK adults. The study has already been discussed by Anthony Cox, Kev Leitch, Sullivan, Catherina, and others. It found that 1% of adults living in private households had ASD. This result should come as no surprise to regular readers of this blog, particularly those who have read High Prevalence of Autism in Adults.

The NHS study did not look for autistic adults living in "communal establishments" (institutions.) According to the NAS, the report is "the first part of a much more detailed research project" (source) so I have little doubt there will be a follow-up that looks for adults in institutions. What will happen then? The NHS report says that 2% of all adults live in communal establishments. I believe pooled prevalence from both studies might turn out to be around 1.3%. It will easily be 1.2%.

Of course, there's some room for statistical uncertainty in these figures, and we'll have to wait for replications that use roughly the same methodology to see how the figures converge. But taking them at face value, 1.2% or 1.3% are a little higher than might be expected. Shouldn't the prevalence among children in the UK be even higher than this? Presumably, diagnostic stability to adulthood is not 100%, and autistics might have a slightly lower life expectancy than normal.

It's important to understand that the NHS study is not only the first of its kind, but also one with a methodology that is both novel and innovative. The methodology is designed so that the prevalence of ASD can be estimated even if the screening tools available to the researchers are unreliable and not well researched. The method does not require knowing how a screening tool performs a priori. It's preferable that the tools perform well – it helps with statistical power – but this is not a requirement per se.

I realize most people can't seem to make heads or tails of the methodology of the NHS study. I've tried to explain it with examples (e.g. here). You have my assurance that it's probabilistically sound. This can be proven mathematically.

That's not to say the methodology doesn't have any problems. Like any new methodology, it will be scrutinized in the scientific literature. Potential issues will be identified. Guidelines on how the selection probabilities should be defined will be proposed, etc. This is generally acknowledged by the authors:
The present methods are therefore felt by the authors to be to the highest standard achievable at present. However this is a first methodological development of its kind in the autism field and it is to be hoped that future surveys could build and improve on the present procedures.

Don't be surprised, though, if this methodology becomes a sort of standard that is subsequently used in studies of children, and not just those dealing with autism prevalence. Don't be surprised either if this in turn results in slightly higher prevalence figures.

The methodology of the NHS study is such that it doesn't leave any autistics behind, so to speak. It produces only an estimate, but it's an estimate that includes all autistics in principle (if we discount participation refusal biases.) A study that relies on a conventional screening process, on the other hand, can easily leave out autistics who did not pass the screening.

It's not easy to know a priori how many autistics a screening tool might miss. Suppose you evaluate 1000 random persons with a reliable diagnostic instrument and also a screening tool you want to test. You might identify 10 autistic people by means of this undoubtedly expensive endeavor. Suppose 5 of them also passed the screening. Can you assume 50% of autistics are missed by the screening tool? Not really. With statistical confidence, you can only say that anywhere from 20% to 80% would pass a screening.

Sure, you could test a much larger group of previously identified autistics. But how do you know this group is representative of all the autistics you intend to identify in a subset of the general population? This is clearly a limitation of pre-NHS methodology.

In the old days, they didn't even take into account the likely unreliability of screening methods. In Lotter (1966), for example, they just provided a questionnaire for teachers to fill out and only considered children reported as having "certain types of behavior." At various other levels of screening, it was always assumed the screening was perfect.

This brings me to a relatively recent study, Kadesjö et al (1999). It reported the prevalence of ASD was 1.21% in the small Swedish town of Karlstad. All children involved were born in 1985. What's interesting about this study is its intensity. The primary author personally evaluated 50% of 818 children attending normal classrooms. Unfortunately, it was a small study. They found 10 autistic children out of 826. If they had only found 9 autistic children, prevalence would be 1.1%. If they had found 11 children, prevalence would be 1.3%. I can't help but wonder, though, what might have happened if 100% of the children had been evaluated. Is it possible that at least one more autistic child would be found among the remaining 409 children who were not assessed for autism past the initial screening?

To summarize, diagnostic criteria and diagnostic methods are not the only thing that matter in a prevalence study. Case finding is also crucial, but now the NHS has introduced a methodology that could make case finding practically a non-issue.