Thursday, March 16, 2006

Geiers Retraction a Done Deal IMHO

I would like to ask bloggers on this side of the debate to refrain from disparaging JPANDS for the time being. Let's give them the benefit of the doubt and see how they handle this matter.

In response to my last communication sent to the Editor-in-Chief of JPANDS, Dr. Huntoon, he replied indicating that he would forward my concerns to the authors as it is fair to allow them to respond.

Meanwhile, I had received a reply from CDDS:


Thanks Joseph – let us know if you need supporting statements or other information.



Paul Choate
DDS Data Extraction
[phone removed]


I took them up on that and asked if they have data on newly reported clients (true new cases) and whether they could confirm if these numbers are dropping in reality. I got a reply from Paul Choate containing an Excel file with numbers going back to 1992. This is still the conventionally available data, but it will be very helpful in posts I plan for the future. I must say that the staff at CDDS Data Extraction obviously do a great job and are very responsive. Paul's reply follows:


Joseph –



Right now we are in contact with David, (not Mark) Geier. He has requested a two datasets, one showing “new consumers with autism” by quarter and the other showing number autistic consumers by quarter of their assessment date. Once they are produced and released our policy is that we will release it again to other requesters if asked. The only caveat is that if there is confidential information we must either encrypt or delete the confidential parts, or require Human Subjects review through our Agency, CA Health and Human Services. All of our information is subject to HIPPA requirements, and any new requests outside other state entities are charged at around $80 per hour necessary to produce.



Allow me to make a few points:



CA DDS has client evaluations (CDERs) on all clients over age three in our service delivery system. There is no requirement for California residents with autism or other developmental disabilities to be served by our system, participation is voluntary. Thus we do not have information on the prevalence of autism in the general population, we only have information on those who come to us for our services. It is unknown how our population reflects trends in the larger population, as the number of clients seeking our services is a function of a number of internal and external factors. You may surmise that as our services to a certain population become more widely known there may be a change in population of those requesting services. We pay service providers to provide services to eligible consumers, so service systems may have incentive to promote who comes to us for services. We are an entitlement system, so there is no other requirement, such as income level, only that a person meet our disability requirements:



http://www.dds.ca.gov/statutes/
laws_main.cfm



Lanterman Act
California Welfare and Institutions Code
Division 4.5. Services for the Developmentally Disabled
Chapter 1. General Provisions
4512. As used in this division:
(a) "Developmental disability" means a disability that originates before an individual attains age 18 years, continues, or can be expected to continue, indefinitely, and constitutes a substantial disability for that individual. As defined by the Director of Developmental Services, in consultation with the Superintendent of Public Instruction, this term shall include mental retardation, cerebral palsy, epilepsy, and autism. This term shall also include disabling conditions found to be closely related to mental retardation or to require treatment similar to that required for individuals with mental retardation, but shall not include other handicapping conditions that are solely physical in nature.


(l) "Substantial disability" means the existence of significant functional limitations in three or more of the following areas of major life activity, as determined by a regional center, and as appropriate to the age of the person:
(1) Self-care.
(2) Receptive and expressive language.
(3) Learning.
(4) Mobility.
(5) Self-direction.
(6) Capacity for independent living.
(7) Economic self-sufficiency.


This was changed from “at least one” of the major areas of life activity about two years ago in an effort to stem some of the rapid population growth to those with the highest need for service.



As for those who do come to us for service we have a date of determination of autism, there is such a date on over 98% of the autism consumer records. See page three of the CDER:

http://www.dds.ca.gov/FactsStats/pdf/
DS3753_CDER.pdf



I’m sure you have seen the “limitations” document

http://www.dds.ca.gov/factsStats/pdf/
CDER_QtrlyReport_Consideration_
Limitations.pdf



As to whether a client is on active caseload we have a Client Master File that has a status type. We maintain monthly copies of all CMF and CDER records back to 1992.

'0'= 'Diag and Evaluation'

'1'= 'High-Risk Infant'

'2'= 'Active Client'

'3'= 'at Risk Person (parents of potential clients)'

'4'= 'Inactive'

'5'= 'Closed Transfer'

'6'= 'Closed not DD'

'7'= 'Closed Deceased'

'8'= 'Dev Center Client'

'9'= 'Closed Other'

'S'= 'Closed - Out of State'

'D'= 'Closed - Not Determined'



So we have the status of every client by month from 1992 forward.



The majority of our clients come to us in age 0-2 through our High-Risk Infant program (Early Start), but autism diagnoses are usually made in the clients 3-5th year. Thus clients come to us often before they are assessed as autistic. This creates a problem with saying how the current population is growing; on the quarterly report we calculate net change of consumers with autism assessments. The numbers in our quarterly report reflect changes in active population who are assessed, so it is a mix of net inflow and outflow, combined with change in assessment with age.



Two interesting facts that have been documented are that 1) the nature of the served autistic population as reflected in the CDER has changed, and 2) that much of the growth is localized geographically, chiefly in a handful of Los Angeles area centers. It used to be that MR was highly co-morbid with Autism, but the rate of MR in the newer population is dropping. In LA area centers the Autistic population has grown to around 25% of the caseload, in central and Northern California the proportion averages around 10%. I’ll attach a chart showing the rate at the centers. This information is the same as what is presented in the quarterly report. The quarterly report is available online for the past few years, but is available in hard copy back to the early 90’s. We maintain a dataset that has the data from the quarterly report table 1 “Statewide” and table 34 “Autism” from 7/92 forward. I’ll attach the data and the MR proportion and geographic charts.



To summarize, we don’t serve the full population and client assessments change over time. When we send Geier the next round of information we would be willing to also send it to you.



Paul Choate
DDS Data Extraction
[phone removed]


At this point it is clear that Dr. Geier is aware of the terminology error, and is requesting the data on true new cases in order to at least be able to claim that even though the paper is mistaken, the conclusions are still valid.

I replied to CDDS with the following:


Thank you Paul. I'd certainly like to have a look at that data myself. Will it include number of drop-outs per quarter as well? Are there limitations on the use of that data? For example, could I post a table on a website with that data?

I have been analyzing some of the publicly available CDDS data, and I have noticed the declines in autistic characteristics over time that you mention, particularly mental retardation (all types), epilepsy and severe behaviors. At the same time, the proportion of lack of mental retardation is increasing every quarter. Additionally, prevalence in the Westside (West LA) regional center is about 500% higher than that of the Central Valley (Fresno) regional center, using epilepsy as a baseline, whereas prevalence of mental retardation is the same. This is all consistent with a 'broadening criteria/awarness' theory and I have written about it online.

I appreciate that CDDS data should not be used to make inferences about the prevalence of autism in the general population, but it is clearly used by others in this manner, and I mostly analyze it in this manner in response to that.

In any case, given the data that Dr. Geier does have, would it be possible in your opinion to conclude what he concludes in the title of the paper?

Early Downward Trends in Neurodevelopmental Disorders Following Removal of Thimerosal-Containing Vaccines

This suggests to me and I suppose to any reader that the number of diagnoses of autism are currently dropping in California.

Even if the data does confirm that there is some drop in the number of real new cases (which is highly unlikely IMHO) much of the paper is still wrong in its use of terminology as are its conclussions from the data it does present, and some kind of errata will need to be published. I'm sure Mr. Huntoon agrees.

Thanks again,

Joseph



I should emphasize that there is no question there is a terminology mistake in the Geiers paper which invalidates the entire paper. This is true regardless of whether the new data shows there is a drop in the number of true new cases in the last few quarters. This is why I claim that some kind of retraction is a done deal, provided JPANDS has some scientific integrity as I'm trusting it does.

Now, based on trends in the 3-5 age range, I do not expect the numbers to show there is a drop in true new cases. I suspect it will show numbers are either stable or going up. If at least 3 of 4 quarters in 2005 as well as the entire year of 2005 do not show a drop in true new cases, I will definitely have a word with Dr. Huntoon.

8 comments:

  1. Well done Joseph, your persistence for the truth is most appreciated.

    ReplyDelete
  2. FYI, Jonathan has written a critique of the paper which includes a number of additional significant errors, and is also recommending that JPANDS retract the paper.

    ReplyDelete
  3. Dang, well done Joseph.

    I was in contact with the CDDS and JPANDS yesterday. I don't think the Geiers work is going to hold up via the CDDS data as they can't change the fact that the Lanterman Act changed the criteria which came into effect in August 2003 and is about as serious a confound as could be seen.

    I am not convinced that we will get a retraction, but I think we can at least convincingly show to a lot of people that this study is flawed. Really that is the more important thing.

    ReplyDelete
  4. I was in contact with the CDDS and JPANDS yesterday. I don't think the Geiers work is going to hold up via the CDDS data as they can't change the fact that the Lanterman Act changed the criteria which came into effect in August 2003 and is about as serious a confound as could be seen.


    Yes, I'm looking forward to the data on true new cases. I expect to see a drop in 2003, probably only one quarter, which is attributable to this change. (This can probably be validated by looking at client characteristics). After that, I'd expect the numbers to either be stable or going up slowly. But who knows, perhaps the policy change takes some time to have full effect.


    I am not convinced that we will get a retraction, but I think we can at least convincingly show to a lot of people that this study is flawed. Really that is the more important thing.


    I think we'll get it. Let's not relent. The only way we won't get any sort of retraction is if JPANDS acts as if nothing has happened. They know the terminology error invalidates the whole thing.

    ReplyDelete
  5. I'm not convinced the JAPandS editorial board would even consider a retraction necessary. That might set a (needed) precedent to go back and critically look at several articles they've published (which they were apparently incapable or unwilling to do in the first place, at the time of submission of many such articles). Look for the possibility of a re-paint by the Geiers.

    Perhaps a respectable physician would publish this in a real peer-reviewed journal?

    ReplyDelete
  6. Also, the Geiers will be shelling out $80.00 an hour to have the CDDS make that database for them.

    The CDDS folks are pretty quick, but this could get expensive for the Geiers in a hurry.

    I hope that they find it worth it.

    ReplyDelete
  7. The Geiers might change their theory based on the new data. For example, they might claim that the 2003 drop is due to thimerosal removal.

    But it seems to me it's crystal clear to Dr. Huntoon and Dr. Geier that the paper is mistaken. There's no other reason for Dr. Geier to have asked for data on true new cases unless he admits that his "new cases" terminology is false and misleading.

    In other words, for JPANDS to not retract this paper in some way they would have to have zero scientific integrity and intellectual honesty.

    ReplyDelete
  8. Well done to everyone who have pursued this question. It seems to me that ridiculing the Geiers, while being very emotionally satisfying, is not so effective as treating them as serious researchers and holding their work up to scientific scrutiny.

    ReplyDelete