Saturday, January 20, 2007

David Kirby Toys With IDEA

David Kirby is now admitting that autistic children can in fact be missed in significant numbers, and that administrative prevalence increase can be attributed to increasing recognition of the condition. This is from his latest post at HuffPo:

But the national autism rate within IDEA is only 25 per 10,000 among 3-5 year olds, far below the estimated rate of 60 per 10,000. Ohio has only 9 cases per 10,000, Puerto Rico just 7, and Oklahoma just 4 (about 1/20th the rate of Maine).

Clearly, we need to let the diagnosing and reporting catch up before drawing any solid conclusions about autism...

I think that's a step in the right direction; a far cry from what he said just one month ago (which I won't address in this post):

With so many holes shot through their "five large studies" defense, the government lawyers will be left to argue that autism is purely genetic, that there is no environmental component, and that the rates of illness have not "really" gone up. We are simply better at recognizing and diagnosing the disorder, that's all.

My guess is that a month ago David Kirby did not know much about some of the details in IDEA, such as the fact that Maine has a special education autism prevalence of 77 in 10,000 among 3-5 year olds, whereas Oklahoma's prevalence is 4 in 10,000. I'm willing to bet this difference is not due to Maine having more crematoriums. (Oklahoma is catching up quickly too, considering that its autism prevalence has just about doubled between 2004 and 2005).

It follows that David Kirby must now also believe in the mechanism of diagnostic substitution, as documented by Shattuck (2006). Most children with autistic disorder or PDD-NOS "missed" in Oklahoma, for example, are likely in the special education system, except reported as part of other categories. These other categories are probably Mental Retardation, Specific Learning Disability, and Speech and Language Impairments. As the autism caseload rises, the caseload of some or all of these others categories can drop, and it has.

Kirby's look at IDEA comes after his prediction about the California DDS 3-5 cohort failed to materialize (a fact which he, JB Handley and others have failed to acknowledge). [ERRATA 01/22: In his recent debate with Arthur Allen, David Kirby did amit that his prediction failed and that this was in fact a blow to the thimerosal hypotheis]. Administrative prevalence of autism in the 3-5 IDEA cohort, nevertheless, is also on the rise as of the latest report. It went from 22 in 10,000 to 25 in 10,000 between 2004 and 2005. As a result, David Kirby has decided to try something new:

Even more extraordinary was the tumbling rate of 3-5 year olds entering school with speech or language impairments, which make up nearly half of ALL disabilities in the IDEA program.

In 2003 the rate was 286/10,000. In 2004 it fell to 282/10,000 and in 2005 it was 272/10,000 - a nearly 5% decline in just two years.

My readers probably know how I feel about claims by David Kirby. So I decided it was prudent to verify the claims from his latest post using data provided by ideadata.org. And I found some information of interest. Some of the numbers are mistaken, but that is not the main problem. What I have found to be wrong and misleading is the implication that these drops are unprecedented and thus possibly explained by a drop in thimerosal exposure.

Before I go on, readers should be aware that there is a caveat. Special education data are not reliable for tracking autism prevalence, as Laidler (2005) explains. The same should be true to an extent about other categories. I will do my best to cite everything so there's little room for doubt.

First, let me get the correct rate drops from the data. In the category "Speech or language impairments" (3-5), special education prevalence went from 2.77% (source) in 2004 to 2.72% (source) in 2005; not as dramatic a drop as Kirby claimed. Either way, there is a drop, and I think it's important to figure out if such a drop is expected or unexpected.

So I went and checked if there's earlier data to compare with. I found table B2A on disability prevalence for the 6-11 cohort from 1991 to 2005. This data clearly shows that (1) the administrative prevalence of speech and language impairments in the 6-11 cohort has been fairly stable throughout the 1990s and beyond, and that (2) there can be random variations in this prevalence from one year to the next. The prevalence peaked at 1.74% in 1993 and again in 2005. It had a low of 1.65% in 2001. These numbers do not support either an epidemic of speech and language impairments in the 1990s nor do they support David Kirby's claim about small drops meaning something in particular.

There is also a table of population served from 1997 to 2001, where we can see drops in the whole population prevalence of "Speech and language impairments" all throughout that period. It is easy to infer diagnostic substitution from that table, particularly from mental retardation over to autism. But it just so happens that between 2004 and 2005, the 3-5 mental retardation prevalence did not drop. I think the increase in the autism category had to have come from elsewhere.

Diagnostic substitution cannot explain the following claim by David Kirby, however:

In 2003, the rate of enrollment among 3-5 year olds in the US IDEA program was 587 per 10,000 kids, continuing a rising trend each year for over a decade. In 2004, the rate went up again, to 594/10,000 children.

But then, in December, 2005 (last year's data will be out in June) the rate fell, for the first time ever, to 580/10,000: a decline of 2.4%.


Again, let me get the correct numbers first. (I suspect whoever helped David Kirby used their own population tables, which is not necessary since the reports give prevalence). The number of 3-5 year olds served by special education system in 2004 was 5.87% of the population (source). The number served in 2005 was 5.84% in 2005 (source). The difference is 0.03% of the population. Although small, it is a drop, and it does seem to be unprecendented in the 3-5 cohort in recent times.

A reasonable explanation for this tiny drop is that the total number of children in the 3-5 cohort is starting to level off, and small fluctuations should be thus expected from one year to the next. How do I know this? Again, let's look at the 1997 to 2001 table. You will note that while the percentage of 3-5 year olds served increased steadily throughout that period, the precentage of 6-21 year olds served was fairly stable. This suggests there is probably an earlier age of diagnosis factor in 3-5 trends. You will also note the percentage of children served does drop from time to time, as it did between 1999 and 2000 in the older cohort. So there's probably no significance to the drop from 5.87% to 5.84% of the population.

A more noteworthy change is one that occurred in Maine, as David Kirby explains:

In Maine, the Special Ed rate among 3-5 year olds fell from 1,178/10,000 to 1,080/10,000 - down 3.3% in a single year.


The numbers are actually correct in this case, and the drop is pretty signicant in my opinion. An expected 1.7% of the state's 3-5 year olds are missing from the IDEA system in 2005. Special education budget cuts are a likely explanation for this drop, but let's determine why Maine might be an unusual state.

Remember how David Kirby suggested that the reason autism rates are not dropping is because we need to let diagnosing and reporting catch up to the epidemiology? Well, we don't have that problem with Maine. In 2004, the administrative prevalence of autism in the state was 66 in 10,000. That is even a little higher than the epidemiology. So it must have dropped by 2005, right? Well, no, it increased to 77 in 10,000. That is not a tiny change.

Maine is the US state with the highest administrative prevalence of autism. They must diagnose autism liberally compared to other states. But that's not all. They also have the 3rd highest proportion of 3-5 year olds served by the IDEA system in the US (after Wyoming and Kentucky). It would not be surprising if they tried (or had) to move in the direction of the rest of the country.

Maine's increase of 0.11% in the "Autism" category is matched exactly by a drop of 0.11% in the "Speech and language impairment" category. Most of the whole population drop is apparently in the "Developmental delay" category. It appears that the definition of "Developmental delay" is created by each state (source).

In conclusion, I do not believe David Kirby's latest post has a leg to stand on. But to his credit, he really does know how to muddy the waters.

12 comments:

  1. I recently attended a legislative committee meeting in my state (one of the ones with a VERY strong developmental preschool program). There was a mention of the drop in my state's IDEA numbers (which Kirby missed in his reporting, BTW).

    The legislators were obviously curious. They asked, "Why?" The response was that special education enrollment drops with increased early intervention.

    ReplyDelete
  2. Hmmm, but I thought it was impossible to miss an autistic person. I guess they didn't exist until Kirby started his book.

    http://www.thoughtfulhouse.org/0405-conf-dkirby.htm

    "They're going to say I had some kind of personal motivation, but I had never met a child with autism in my life until I started this book. I had never heard of Thimerosal until the Homeland Security Bill was passed. I'm a journalist: I just actually thought this was a really hot story."

    ReplyDelete
  3. The response was that special education enrollment drops with increased early intervention.

    I can believe that about older cohorts, but it's probably not a major factor in the 3-5 one. I think increases and drops can happen for the same reason: diagnose more or less.

    ReplyDelete
  4. I imagine that there will always be statistical errors and 'bad counts' but I think we should be glad that Mr.Kirby is looking at the facts available. I hope it helps him.
    Best wishes

    ReplyDelete
  5. I kind of want to know who helped Kirby with his calculations and what specific method they used.

    ReplyDelete
  6. This is found at the end of Kirby's post:


    NOTE: Much of this data can be found at www.ideadata.org.
    Figures on special ed grants is at http://www.ed.gov/about/overview/fed/10facts/edlite-chart.html
    Special thanks to Laura Kasemodel, of Minnesota, for her math-wiz help.

    ReplyDelete
  7. BTW, it's not clear if he's looking at different reports than those for 2004 and 2005 at ideadata.org; or if they calculated prevalences independently instead of taking those of the reports.

    I posted a request for clarification last night at HuffPo, but my message has not appeared as of yet.

    ReplyDelete
  8. I just noticed an error in my post. I'll only correct it here. Where it refers to the prevalence of Speech and language impairments in 6-11 year olds between 1991 and 2005, that is actually the prevalence for 6-21 year olds.

    The observation still applies to 6-11 year olds, which appear in the same table (third down). The 6-11 prevalences (as percentages) in that period are:

    4.01 4.00 4.00 3.98 3.94 3.93 3.89 3.85 3.83 3.81 3.81 3.88 3.97 4.07 4.13

    The peak is 4.13% and occurred in 2005. The low was 3.81% and occurred in 2000 and 2001. The fluctuations have a periodic pattern, but it's clear drops in the category are not unusual.

    ReplyDelete
  9. Mcewen wrote:
    "At 1/21/2007 9:27 AM, mcewen said...
    I imagine that there will always be statistical errors and 'bad counts' but I think we should be glad that Mr.Kirby is looking at the facts available. I hope it helps him."

    Giving someone the benefit of the doubt, is usually a good idea. But Kirby ONLY looks at facts to see if he can warp them to fit his agenda. If you think he's sincere in some way, you are dead wrong.

    ReplyDelete
  10. Having seen DK on stage and in action, I must say he appears to have been thoroughly coached in spin-doctoring.

    ReplyDelete
  11. Note: I just added errata, since I just found out that in the debate with Arthur Allen, David Kirby did admit that his CDDS prediction failed and this did constitute a severe blow to the thimerosal hypothesis.

    ReplyDelete
  12. Lesser Bigot2/01/2007 10:18 PM

    Hi Jo

    I was just reading your intro about yourself. Great! Now Autism is hereditary - You get it from your children!!!!
    Love ya dude

    Yours
    Lesser Bigot

    ReplyDelete