A Word About Hertz-Picciotto & Delwiche (2009)

The recent Chicago Tribune Articles have resulted in a flurry of reactions, particularly from AoA. I'd like to discuss just one of them in this post. Kent Heckenlively was complaining that the Chicago Tribune reporters did not give weight to any of the documentation he had provided them. Specifically, to support the assertion that we are in the midst of an autism epidemic, Heckenlively cited a press release by the MIND Institute.

Why cite the press release and not the paper? Well, that's because the conclusions of the paper said the following.

Other artifacts have yet to be quantified, and as a result, the extent to which the continued rise represents a true increase in the occurrence of autism remains unclear.

A true increase... remains unclear. Is that clear enough?

The paper is titled "The rise in autism and the role of age at diagnosis." I'll refer to it as Hertz-Picciotto & Delwiche (2009). I've already criticized multiple aspects of the paper (last time here) but I'd like to say a few more things about it.

The conclusions of the paper seem refreshingly honest, but I'm guessing they are that way simply to get through peer-review. The missing artifacts are not just any artifacts, either. One of the artifacts that did not enter the calculations is key: awareness. It would be nonsensical to assume that awareness of autism has not changed since the early 1990s.

But what did the press release say?

“It’s time to start looking for the environmental culprits responsible for the remarkable increase in the rate of autism in California,” said UC Davis M.I.N.D. Institute researcher Irva Hertz-Picciotto, a professor of environmental and occupational health and epidemiology, and an internationally respected autism researcher.

Hertz-Picciotto said that many researchers, state officials and advocacy organizations have viewed the rise in autism's incidence in California with skepticism.

(source)

This is completely at odds with the conclusions of the paper, and I find it quite dishonest. To make it perfectly clear, yes, I'm accusing Dr. Hertz-Picciotto of intellectual dishonesty.

Dr. H-P further states that:

“These are fairly small percentages compared to the size of the increase that we’ve seen in the state,” Hertz-Picciotto said.

Is that even true? The paper finds that a 2.2-fold increase may be explained by changes in diagnostic criteria, 1.56-fold due to inclusion of "milder" cases, and 1.24-fold due to changes in age at diagnosis. If you multiply these factors, you come up with a 4.26-fold increase that may be explained by just these 3 artifacts.

That's 62% of the entire rise the authors were attempting to explain. Maybe 62% is a "fairly small percentage." In this case I'm going to give Dr. H-P the benefit of the doubt and say that she likely didn't know the factors needed to be multiplied. There are no indications in the paper that a calculation of the overall contribution of the 3 artifacts combined was even attempted. I won't even get into statistical uncertainty.

The flaws of the analysis are not what bother me the most, however. Consider this. What is the value of this study? Why was it carried out? If you're trying to determine whether artifacts can explain the rise in autism service classifications in California, and you cannot estimate the contribution of all relevant artifacts, what is the point of the analysis?

Here's an analogy. Suppose you wanted to determine if global warming can be explained by the greenhouse effect. In order to do this, you estimate the contribution of methane, water vapor and nitrous oxide to recent temperature increases, but you leave out CO₂. You conclude that those 3 gases alone cannot explain the entire rise in temperatures, but perhaps that's because you did not consider the biggest contributor to the greenhouse effect: CO₂. Then you tell the media that greenhouse gases cannot fully explain the rise in temperatures.

Wouldn't such a study be better understood as a propaganda effort, rather than a contribution to scientific knowledge?

Someone might complain that Hertz-Picciotto & Delwiche (2009) does contribute to scientific knowledge, because it tells us about the impact of certain artifacts. But does it actually do that?

Take what is perhaps the most important artifact the paper does take into account: changes in diagnostic criteria. What would you do if you wanted to determine the impact of changes in criteria? You might carry out a prevalence study with good case-finding that uses two different criteria on the same population: DSM-IV and DSM-III (or perhaps Kanner criteria.) Hertz-Picciotto & Delwiche (2009) does not do anything that even resembles this. They use data from a separate study, so they didn't even contribute new data. We can't even be sure how well the data from Finland might apply to California. The case-finding of the Finnish study is not necessarily very good either. Plus it's just one data point, with all the uncertainties implied by that.

To determine the impact of inclusion of "milder" cases (i.e. anything that is not "autistic disoder"), what would you do? I think you could evaluate a random sample of CalDDS autistic children and diagnose them with either autistic disorder, PDD-NOS or Asperger's. What the researchers did instead was use data from a separate MIND Institute study where CalDDS children had been evaluated with the Autism Diagnostic Observation Schedule (ADOS) and the Autism Diagnostic Inventory (ADI.) Are these diagnostic tools even able to accurately distinguish between autism spectrum diagnoses? I'm not aware of any evidence that they do.

In other words, the MIND Institute study is not even informative about the impact of the artifacts it did take into account. I frankly can't see this study as a contribution to scientific knowledge at all. It gives the appearance of being part of a propaganda effort.

Chicago Tribune Articles

I'm just going to link to the Chicago Tribune Articles below.

• Autism treatments: Risky alternative therapies have little basis in science
• Autism treatment: Success stories more persuasive to some than hard data
• Questionable treatments for children with autism

What do readers think?

You can probably tell I "stole" those links from Orac. I'll also link to his post:

• The Chicago Tribune: Telling it like it is about the antivaccine autism "biomed" movement

I noticed that "biomed" pushers have shown up in comments, and their primary counter-attack seems to be roughly this: "FDA-approved drugs also have serious side-effects!"

They do. Anti-psychotics, especially, can have nasty albeit rare side effects like neuroleptic malignant syndrome. But the biomed people are just trying to change the subject, aren't they?

Feel free to use the comments section to discuss the ethics of FDA-approved drugs for autism. Do the benefits documented in trials outweigh their more serious side-effects? Are there concerns about the evidence of their efficacy? Do we even understand how they work?

False Despair

What would be the value of telling a parent of a recently diagnosed autistic child that the child will never be able to hold a job, if this is unlikely to be true for a good majority of autistic children diagnosed at present? Is there any? I just don't see it. In fact, I think this would be at least as harmful as telling a parent that their child will grow up to be just like Dr. Temple Grandin.

I was recently told that I shouldn't divulge data that could be interpreted as painting a "rosy picture" of autism. It's not only rude to tell people what they should or shouldn't write about, but this kind of suggestion is quite unreasonable in my view.

Beyond the merits of the idea that it's best to presume competence, I believe one should have realistic expectations based on actual data, rather than vague assumptions based on stereotypes and outdated information of unclear origin.

I also think it's highly questionable to dismiss new findings solely on the basis that they "don't feel right" or because they fail to confirm one's preconceptions and personal experience. This is the very definition of closed-mindedness. (A lot of people seem to think "closed-mindedness" means you are not willing to accept pseudo-scientific claims, but this is simply not true.)

It's important that data is current too. It would be completely dishonest to pretend that autistic children diagnosed today are just like the children who were called autistic in, say, the 1960s. That's simply not a realistic view. I have discussed adult outcome studies in the past, and I think they are relevant, but they need to be understood in their proper context. For example, 61% of autistics born in 1983-1985 receiving services under the autism category in California were identified as having intellectual impairments. Meanwhile, only 27% of those born in 1993-1995 are identified the same say (Gernsbacher et al. 2005; The MIND Institute 2002.) This is despite the fact that California DDS has eligibility restrictions, sometimes interpreted to mean that only persons with "full syndrome" autism are eligible (which is probably an untrue assumption, but it's also the case that not all persons with an ASD diagnosis are necessarily eligible.)

I say this knowing full well that my own son is different to most autistic children diagnosed today, and perhaps more like children who were called autistic in the 1960s. I expect my readers to be similarly capable of distinguishing personal realities from population-level realities. If I divulge data applicable to autistics who don't have intellectual disability, and you have an autistic child with intellectual disability, then the data does not apply to your situation. I think this should go without saying.

If you look at really good paper critiques, you will find a common characteristic: They don't just point out limitations and errors; they explain how the limitations and errors could make the results what they are, in the event that the hypothesis the authors have advanced is incorrect. This typically involves pointing out details the authors have apparently overlooked. For examples of remarkably good paper critiques, see the Photon in the Darkness Blog.

Then you have really poor paper critiques. These come in various flavors. Perhaps the lowest form of critique is the ad hominem circumstantial, otherwise known as an appeal to motive: "I suspect the author is biased, therefore the results of the paper are probably wrong." This is essentially a useless critique, since all authors of all papers could be suspected of having a bias. Some additional criteria is required to make it worthwhile. For example, if a researcher is known to have engaged in substantial scientific misconduct (e.g. plagiarism), then it is reasonable to hold all subsequent works by the same author to be suspect a priori. It would also help to explain how the bias might alter the results. In a hypothetical case, it could be reasonable to speculate along these lines: "Since the study was not blinded, it's probable that researcher bias could have inadvertently altered the results of the assessments." This is a plausible explanation, as opposed to a simplistic insinuation.

Another precarious form of critique is what I'll call the critique by anecdote or personal experience. Example # 1: "How could this global warming theory be true? It was really cold this winter." Example # 2: "I have never seen any autistic adults, and I know 'em when I see 'em, so this really expensive and exhaustive prevalence study just cannot be right."

BTW, how is it that David Kirby knows 'em when he sees 'em, if he's never seen them at all?

There are some reasonable forms of critique that don't require much effort. Appealing to the unreplicated status of a result is one example. This is a good convention that is part of the scientific method. "The result could be of interest, but let's reserve judgment until it's replicated."

Some people have said they don't yet buy the prevalence result from the NHS study because the study is unreplicated. In this particular case, I believe they are mistaken. For one, there's considerable prior evidence that autism can be found in various populations of adults where previously it was largely unrecognized. Furthermore, Kadesjö et al (1999) found a prevalence of 1.21% among children born in 1985 in the small town of Karlstad, Sweden. The "children" happen to be 24 now. If the study were done today, on the same exact cohort, it would be considered an adult prevalence study. In this sense, the NHS study may be considered a replication.

Let me make this clear. There's simply no excuse to continue to claim that autistic adults are a non-significant population compared to the population of autistic children. Doing so is just another form of denialism.

Jonathan's Completely Dishonest Attack on My Latest Posts

Dear reader: Please go read Jon Mitchell's utterly dishonest attack on my two latest posts. It's titled Some neurodiversity potpurri.

First of all, I'd like to inform Jon that even though I've said this blog is pro-neurodiversity, I'm not a leader of neurodiversity proponents or anything of the sort. All my opinions are personal and should in no way be seen as opinions put forth by something called neurodiversity. It's very uncool to try to use what I say (or what other bloggers say for that matter) as a way to attack the neurodiversity philosophy as a whole.

Now, the most outrageous misrepresentation of what I said is the following:

I see that Joseph of the autism natural variation blog has decided based on one published report based on the statuses of only 19 people that adults with autism don't have problems with employment nor with marriage.

I could've predicted this was going to be Jon's response, and it's not the first time he's tried to misrepresent what I've said. I'll leave statistics for the end of the post. I want to discuss the misrepresentations first.

What I said about lack of employment is that (1) figures used in prior estimates appear to be exaggerated; (2) that it would seem adult autistics who live in private households across the UK are largely productive individuals who contribute to the economy in a manner similar to their non-autistic peers.

I never said autistic people don't have problems with employment. All I said is that autistics are largely employed. There's a big difference. The data can't tell me anything about specific problems autistic people have with employment, and I have no doubt there are a variety of problems.

I stated that there appear to be some differences in the employment rates of autistics and non-autistics, but the authors didn't find them statistically significant. This is what the authors said:

No significant variation in rate of ASD by economic activity status was found.

That's a true statement. If you want to lash out at the authors for daring to say that, be my guest.

The misrepresentation of my claims about marriage is even more outrageous, since my post clearly said in bold that autistic people in the UK are apparently about half as likely to get married than non-autistic people.

In the past Joseph presented some statistic saying that 25% of autistics were employed. As far as I can tell he neglects to mention any source or reference or link for this statistic in any post on his blog.

OK, that one is my fault. I failed to cite a reference in the old post, and I don't even remember the exact reference. I'm usually careful about that sort of thing. If you're interested, Howlin et al. (2004) reports that almost one third of the adult autistics in that study had some form of employment. Szatmari et al. (1989), a follow-up of adults without intellectual disability, reports a rate of employment of 50%.

So Joseph seems to imply that autistic persons are making just as much money as an NT.

I never talked about salaries, and if you made inferences from my post in regards to salaries, that's really not my fault. The study does have data on salaries. The summary of the findings was the following.

While the likelihood of having ASD appeared to increase among men as household income decreased, this was not significant (when analysis was run using household income
grouped into tertiles).

In this case, again, there might have been some differences, and if you look at the numbers in Table 2.3 it would be difficult to deny that autistics are making somewhat less money than non-autistics. But in a statistical sense, the numbers cannot tell us for sure that there's a difference.

So again, we have ND trivialization of an autistics inability to get married or make a living.

To Jon Mitchell, any figures and facts that don't agree with his personal views and experiences with autism constitute "trivialization." Reality doesn't exactly matter if it doesn't jive with what he sees as the necessity of continuing to have a grim worldview.

The Stats

Jon doesn't like the stats of the study because:

The authors of the reports extrapolated this number 19 to the greater population claiming that 1% of adults in the UK in private households have autism. However this was just a guess based on mathematical projections.

For starters, saying that the result is a "guess" is clearly an inaccurate characterization, and I've explained this previously. Many of the 19 autistic people identified must have been assigned a probability of selection that is less than 1.0 in phase 1. For example, if 10 of the 19 autistic people had been assigned a probability of 0.25, then clearly there must have been about 40 autistic people in the original group who were assigned a probability of 0.25. This is a probabilistically sound projection, not a "guess." (It's a bit more complicated than I explained, because there's also some weighing due to participation refusal based on some household variables.)

Now, no study can prove a negative, i.e. that there's absolutely no difference between the characteristics of two groups (this is the "null hypothesis.") All you can say is that you found the null hypothesis cannot be rejected. Of course, some studies are better able to rule out small differences as opposed to big differences, with statistical confidence.

It does matter that only 19 autistic people were found, but what mostly matters is the overall sample size. In Table 3.6 they indicate the confidence interval for ASD prevalence is 0.5% to 2.0%. They list the sample size as 2854 and the "weighed" sample size as 7358.

There are prevalence studies smaller than this, and existing data on the characteristics of autistic adults are usually based on comparable or smaller samples of autistic people.

The confidence interval is actually what you would get if you had found 7 autistic people out of 700 in a standard sampling (and I have some thoughts about implications of this, but I won't go into them here.)

Prevalence of ASD among adults who are employed was 0.9%. The confidence interval would be roughly equivalent to what you'd get if you had found 5 autistics out of 555 people, or 0.4% to 2.1%. (About 75% of all adults are employed in the UK.)

So on the one hand you have a prevalence of [0.5% - 2.0%] and on the other hand you have a prevalence of [0.4% - 2.1%]. Is there a difference? We just can't say there is.

Marriage Among Autistics, Or Why the NHS Study Obliterates CADD

In the previous post I discussed data that contradicts the common belief that autistic adults are largely unemployed.

In this post I want to discuss marriage. Marriage, again, is something that is considered very rare among autistics. It's a stereotype like any other stereotype, of course, but it's not an entirely unjustified one.

Of all the autistics Leo Kanner wrote about, I believe only one (Robert F) is known to have married. Said Kanner:

The contacts thus established led to the discovery that the boy-meets-girl issue was paramount in the talks of the companions. Again, there was a vaguely felt obligation to "conform." Those attempts were sporadic and short-lived. The "explanations" offered indicated that there was not too much displeasure with the absence of any real involvement.

Henry C. reported that he was single, that several girls "had hoped to change that" but that he had "no desire to get tied down for a good long time." Thomas G. declared categorically that girls "cost too much money." Clarence B., who "socialized" with a girl for a short time in college, stated that he "ought to get married but can't waste money on a girl who is not serious." Bernard S. was said to have approached a girl once for a date "in a very negative way" (inviting rebuff). Fred G. "experimented" once with a double date arrangement (never repeated).

George W. made things easy for himself by deciding a priori that girls were not interested in him. Sally S., the only girl in our group, once asked seriously at 23 years of age what she ought to do if ever she fell in love with someone, an experience she had never had before. She said: "I have never had the interest in boys most girls my age have." At 30 years, she dated a man for a few months but gave this up because she was "frightened by any intimacy."

(Kanner et al. 1972)

In several outcome studies we see the same pattern. Of the 16 autistic adults without intellectual disability from Szatmari et al. (1989), only one was married, even though 4 were dating regularly. Average age at follow-up was 26.1. From a recent outcome study out of Utah involving 41 autistic adults, we learn that only "a few" were married.

One problem with these findings is that adults in outcome studies are young adults. Additionally, we should not assume that diagnosed autistic adults, first identified when they were children, are representative of all autistics from the general population, known and unknown. Current understanding of autism is different to what it was in the past.

Once again, the NHS adult prevalence study proves useful in addressing these sorts of limitations in available data. The report tells us that the prevalence of ASD in three marital status groups is as follows.

Married/cohabitating: 0.5%
Single: 2.5%
Widowed/divorced/separated: 0.6%

There's also information on a "base" population that is not exactly representative of the general population, but it's probably close enough to being representative that it can be used to come up with estimated proportions of autistics and non-autistics in each of the categories.

	Married/Cohabitating	Single	Widowed/divorced/separated
Autistics	33%	57%	10%
Non-Autistics	61%	22%	15%

Data for men only:

	Married/Cohabitating	Single	Widowed/divorced/separated
Autistic Men	30%	63%	7%
Non-Autistic Men	61%	26%	14%

There are some differences. Autistic people in the UK are apparently about half as likely to get married than non-autistic people. I would not say the characteristic is diagnostic, however.

I should point out once more that the study only looked for autistic adults who live in private households. The data will likely change somewhat when autistics who live in "communal establishments" are considered.

Implications For CADD

I've previously criticized the concept of Cassandra Affective Deprivation Disorder (CADD). It's a pseudo-scientific, made-up and damaging idea to the effect that autistic people make for terrible significant-others and spouses to such an extent that they can cause their partners to develop psychological problems and health issues, even cancer.

If CADD had an ounce of truth to it, you'd expect divorce to be rampant among autistics. Heck, you'd expect autistics to widow more often, if the claim about effects on health were true.

Yet, the divoced-to-married ratio for autistics is 0.29, whereas the same ratio is 0.24 for non-autistics. (It's 0.24 vs. 0.22 for men only.) That's a minor difference, not significant by any stretch of the imagination.

It's clear that the reason a lot of autistic people in the UK are single is not because they fail at relationships. It's because they are not good at starting relationships.

The claim that autistic marriages fail much more often than expected is mythical, evidently. Nevertheless, many web pages state the following: "Dutch research suggests that the divorce rate for people with Asperger syndrome is around 80 per cent." Interestingly, this exact phrasing appears throughout the web, as if it had been copy-pasted. I have tried to locate the original source of the claim, to no avail. One person says they "lost the link."

The claim appears in a book by Ashley Stanford for people whose significant-other has Asperger's: "Preliminary research performed in Holland suggests that the divorce rate for couples in which one partner has AS may be as high as 80 percent (Relate leaflet)."

The Relate leaflet in question appears to be this webpage by none other than FAAAS. FAAAS, for those who don't know, is an organization of people who moan about their "frustrations" with their adult autistic relatives. FAAAS, not surprisingly, is connected to Maxine Aston of CADD infamy.

The claim that appears in the Relate leaflet is exactly the following:

For the partner of a person with AS, their bewilderment at the puzzling behaviour of their spouse can cause deep distress and a breakdown in their health. Inevitably this means that there is a high divorce rate amongst marriages where one partner has AS. Research in Holland suggests that this is as high as 80%, and research recently done in the UK is due to be published next year.

It's entirely unsourced. I'm currently skeptical as the the existence of the famous Dutch research.