Thursday, January 08, 2009

The MIND Institute's Second Attempt: More of the Same Type of Reasoning

The MIND Institute has published a new study titled The Rise in Autism and the Role of Age at Diagnosis (hereby referred to as H-P et al.) Among its conclusions: "Younger ages at diagnosis, differential migration, changes in diagnostic criteria, and inclusion of milder cases do not fully explain the observed increases." As you can imagine, the paper is being cited uncritically in the usual places.

I would like to discuss what it is the paper finds, how it finds it, and whether the findings are accurately characterized. Before I do that, however, I think some background is in order.

Back in 2002, a MIND Institute report to the California Legislature concluded basically the same thing: That a loosening of the criteria had not contributed to the rise in autism diagnoses in California. Back then the MIND Institute report was considered evidence of the "autism epidemic." But it contained a significant error in reasoning, first noted in Gernsbacher, Dawson & Goldsmith (2005).

Basically, the 2002 report found that an earlier cohort of children met DSM-IV criteria at about the same rate as a more recent cohort of children. As Gernsbacher et al. patiently explain, it's not surprising that the more recent DSM-IV criteria is met by nearly all the children, the younger ones and the older ones. What would be more relevant to find out is whether the older children meet some narrower criteria that the younger children do not meet. Gernsbacher et al. illustrate the fallacy using a height analogy and call the researchers' conclusions "imprudent."

There are two main areas of the new paper that I want to discuss: Changes in diagnostic criteria, and inclusion of milder cases (birth cohorts 1990-2006). The paper looks at other areas, but I'd like to focus on the two that are key.

Changes in diagnostic criteria

Finding: Changes in diagnostic criteria have contributed a 2.2-fold increase in the rise of autism incidence in California.

How is this determined? This being the most important area of the paper, my expectation was that I would find new data produced by H-P et al. showing that only a 2.2-fold increase may result from changes in diagnostic criteria. This is not the case at all. H-P et al. rely on prior work.

Specifically, they rely on Kielinen et al. (2000), a study out of Finland based on data "collected from hospital records and the records of the central institutions for the intellectually disabled in the Provinces of Oulu and Lapland in 1996–1997."

The 2.2-fold increase is the difference found by Kielinen et al. when comparing autistics diagnosed with Kanner's criteria vs. those diagnosed with ICD-10 or DSM-IV.

So what's wrong with this result? You need to ask yourself if the results from Kielinen et al. are applicable to changes in the California DDS autism population.

You see, even if we assume that all California autistics from the birth cohort 1990 were diagnosed with Kanner's criteria, why should we assume that all persons matching this criteria were identified and registered with California DDS? Isn't it more likely that only a small minority of such persons were recognized at the time? Furthermore, Kielinen et al. only look at the population with autism or other psychiatric conditions as recorded in known databases. California DDS has a much broader population pool to draw from, which would be more than relevant at the present time.

Also note that the DSM-IV prevalence found by Kielinen et al. is low compared to that of other DSM-IV studies: 20.7 in 10,000.

In essence, I do not believe there is any basis for comparison of Finnish ascertainment done in a 2000 study vs. California DDS ascertainment, either in 1990 or in 2006.

Inclusion of milder cases

Finding: Inclusion of milder cases contributes a 1.5-fold increase in the rise of autism incidence in California.

How is the result determined? H-P et al. again rely on work from a separate study: The Childhood Autism Risks from Genetics and Environment study. In this study, 64% of California cases were confirmed to meet criteria for autism (excluding Asperger's and PDD-NOS) when evaluated using two separate diagnostic instruments.

I was initially confused as to why inclusion of "milder" cases is necessary to take into account, if the paper already claims to have looked at changes in diagnostic criteria. The reason is probably that Kielinen et al. would not have studied PPD-NOS or Asperger's cases.

So what's wrong with the result? If you're trying to account for cases that Kielinen et al. would have missed, I'm not sure that PDD-NOS and Asperger's are enough. What about high functioning autism in general? That's not the same thing.

There's also the question of why only "milder" cases should be considered. Why not more severe cases? Should we assume that Kielinen et al. would've detected all of those? Then there are cases that are neither more severe nor milder but just not diagnosed as autism traditionally. For example, would Kielinen et al. have recorded cases of autism in Down Syndrome or Cerebral Palsy, where California DDS obviously does?

In the end, this particular result does not tell us much. What it says is that if you exclude all autistic children who should not be eligible for California DDS services, the impact on the number of cases is not too great. I could've told you that.

Did They Miss Anything?

One question that comes to mind is why the researchers did not discuss diagnostic substitution, particularly from mental retardation.

Would this be part of the "changes in diagnostic criteria" analysis? Yes and no. To the extent that children previously diagnosed with mental retardation are currently diagnosed with autism because DSM-IV criteria says so, then yes, the analysis by H-P et al. would suffice (assuming that analysis were valid, which I don't believe it was). If there are reasons for the shift that go beyond criteria, then no, it is not enough.

Diagnostic substitution in California is interesting for other reasons. I think there's sufficient data there to do a proper analysis. For example, you can look at cases of mental retardation without autism over time and see if they decline. I have reason to believe such an analysis will be available in the near future.

Comment

H-P et al. is a surprisingly poor paper. It does not produce any new data in order to support its two main results. It makes an apples-to-oranges comparison between a Finnish epidemiological study and California DDS ascertainment over time. It tells us the obvious about "milder" cases. In the end, I don't think this is an improvement over the 2002 MIND Institute report to the California Legislature. In fact, it could very well be worse.

The way H-P et al. have gone about trying to show there's a real rise in autism incidence over time is not a very good way to go about doing things, in my view. There are other ways. For example, I've suggested trying to replicate Lotter (1967) in detail. This would not be as easily challenged.

6 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. I refer you to my most recent post on Mike Stantons blog.

    It applies to the mind institute as well as SBC, and also critiques the reasoning of Michelle Dawson who is no less prone to the scientific fallacy which can be seen if one follows up the Duhem-Quine Thesis and Pierre Duhem's Undetermination Hypothesis.

    In my argument below one can happily substitute the word "cause" for "gene" as the two words are linguistically related anyhow through etymology and use.

    Self Quote follows:-

    "There is no single gene for the categories "Artist" or "Scientist" either each condition is contingent upon multiple factors because they are a human category not a natural one, in essence the reason why a single gene or cause will not be found is because "Autism" is also a human category, and as such is not "watertight" it leaks all over the place because that which is called autism is the confluence of many rivers and depending where you stand in the lake you might feel the influence and currents of any one of them more than another.

    Therefore those who pin the argument for autistics rights on the outcome of a science which can describe us in a positive way are on a hiding to nothing as the rememdies are societal and to hope for a scientific justification is to bend science according to ones will, which of course is what all scientists and philosophers do anyway.

    Autism finds itself wholly within the social model of disability for an explanation of how it is studied, valued, devalued or otherwise debated.

    ReplyDelete
  3. I'm someone who does get your arguments about the way autism is socially constructed, Larry, and I basically agree.

    I don't believe, however, anyone has suggested that autistic rights hinge on the outcome of science. I don't believe Michelle Dawson or anyone else has said this. There are obviously other aspects to autistic rights.

    But there is a problem when the standards of science and ethics applied to autistics are poor.

    For example, I would ask if calling autistics an epidemic is warranted or appropriate, based on a study that doesn't consider something as basic as awareness; a study that equates Finnish epidemiology with California DDS ascertainment. This is quite a poor standard, in my view, and the use of such poor standards is relevant to autistic rights.

    ReplyDelete
  4. Southeast and main Asian pandora jewelry countries have twisted rubies for centuries, cheap pandora bracelets but research as to where, and how to find more deposits is Pandora charms spare, and production has figured out how and mining companies,” Pandora beads Giuliani says, to look at exactly the right time and place.” pandora set Farther investigation of claret formation, based on tectonic scenery, cheap pandora geochemistry, fluid inclusions and isotopic ratios, allowed discount pandora Giuliani’s lineup to remodel a new prototype for the French Institute pandora 2010 of Research for Development (IRD) and the National Scientific pandora sale Center of Research, two government-sponsored knowledge Pandora Bangles and technology research institutes that aim to aid in the sustainable cheap pandora bracelets development of developing countries. Before the collision pandora bracelets prices of the Eurasian and Indian plates, lagoons or deltas sat in the regions where marble is giant, pandora bracelets and charms he says, “and there is the brains to expect that the new pandora bracelets sale thoughts should help development of the artless capital.” discount pandora bracelets Virginie Garnier, Gaston Giuliani and Daniel Pandora necklace Ohnenstetter urban the shape to do just that. They work for the garnet cheap pandora charms genesis. While studying the bedrock in Vietnam in 1998, the discount pandora charms French players found rubies, which detained traces of aluminum, chromium pandora charms sale and vanadium from universities, international corporations, governments pandora charms 2010 and why the rubies got there, and has created a paradigm Pandora beads to help these evaporites, Garnier says, when the Eurasian cheap pandora beads and Indian plates collided, raising the Himalaya Mountains.

    ReplyDelete