Saturday, April 22, 2006

Second Open Letter to JPANDS

For readers who haven't been following the Geier & Geier (2006) retraction controversy, here's a recap:

- On March 12, 2006, I sent a retraction request to the editor of JPANDS, indicating that there is an unsalvageable flaw in the author's use of the "New Cases" terminology.

- I received a response from Dr. Huntoon asking if this was a letter to the editor. I replied indicating that it was not a letter to the editor, but a request for a response from the peer reviewers. I further explained the magnitude of the error with an analogy. Dr. Huntoon subsequently agreed to forward my comments to the authors as it was fair to allow them to respond.

- On March 15, CDDS informed me that David Geier had requested data on "new consumers with autism", thus admitting to the error, and probably hoping to salvage the conclusions of the paper by means of new supporting data.

- On March 24, I asked CDDS what the status on the requested data was. Paul Choate informed me that the dataset was basically ready, pending payment from the authors. Paul Choate also outlined a number of other known facts about the CDDS which invalidate Geier & Geier (2006).

It had been almost a month since my last contact with CDDS, so yesterday I asked Paul Choate again about the requested dataset. He tells me that the final steps of the job are on hold because payment has not been completed yet. It appears to be unclear at this point if this transaction will go through.

Paul Choate also informs me that he believes members of the CHARGE study at the MIND Institute are preparing a formal rebuttal to Geier & Geier (2006) to be published in the AMA Journal.

A formal rebuttal will be great, and it's not surprising that many rebuttals might be published challenging a paper as flawed as this one. But I think it is still important to hold JPANDS to account and test their integrity. They need to recognize that a full retraction is a necessity in this case. So I contacted Dr. Huntoon once again.


Dear Dr. Huntoon:

On March 12 and 13, 2006 we discussed a serious error in the use of the “New Cases” terminology found in Geier & Geier (2006). Subsequently, you agreed to forward my concerns to the authors in order to allow them to respond.

Mr. Geier (the senior author) tacitly admitted to this error when he requested data on “new consumers with autism” from the California Department of Developmental Services (CDDS). I understand CDDS has completed the requested work, but is awaiting payment from the authors. At this point it is unclear when and if this transaction will be completed.

While it would be informative to review data on newly reported autistic clients in order to have a more accurate picture of the administrative incidence of autism in the state of California, I believe it is necessary to point out that this is immaterial to the validity of the paper, as published in your journal.

Terms such as “New Cases” and “Newly Diagnosed Cases” referring to CDDS quarterly differences and caseload differences in general, invalidate Geier & Geier (2006) in its entirety, regardless of what the new data shows. Even if the new data were to be consistent with downward trends in the incidence of autism, which is unlikely as I argue below, much of the paper would need to be rewritten. Publishing errata is insufficient due to the magnitude of the error. Graphs would have to be redone and figures changed in many parts of the paper. Without the new data, it is impossible to do this.

The new data is unlikely to be consistent with the current conclusions of the paper, for well-known reasons. Any valid rewrite would practically need to argue the opposing hypothesis. This is because CDDS caseloads in the 3-5 and 6-9 cohorts continue to grow at a pace that is considerably faster than general population growth in the state. This alone is inconsistent with an ongoing incidence drop.

The flaws in the paper are substantial, and they go beyond what I’ve outlined. Under the circumstances, I'd like to urge JPANDS to publish a full retraction of the paper. If the authors are able to write a different paper, based on new data, the new paper would have to be considered a new submission and should be freshly reviewed. I would appreciate being informed of the actions your editorial board is planning to take regarding this matter.

Sincerely,

[Joseph]

P.S. I am blogging this communication, and copying CDDS.


Special thanks to Jennifer for her feedback on the letter.

7 comments:

  1. Joseph, why don't you write up your analysis and submit it to some journals? I don't know what kind of credentials you have, but if Lyn Redwood and Sally/ie Bernard can get published, I'm sure you can too.

    Thanks to you and Jennifer for all your work on this.

    ReplyDelete
  2. Joseph,

    I just want to say I really admire you for sticking with this. I also want to thank you for examining all of the studies you have presented in such a fair and consistent manner.

    ReplyDelete
  3. Thanks Lisa and Anonymous.

    I haven't considered publishing formally in the epidemiology and/or autism field. That's actually not where my training is, but I have published and peer-reviewed when I was in academia.

    ReplyDelete
  4. Good job Joseph. Even if a retraction never happens, this will be important. In the future, I can see a credentialled author in the field, publish a paper about "the state of the science" highlighting the eventually numerous calls for errata and retraction by autims bloggers such as yourself. The fact that such bloggers exist and can pretty easily disassemble crappy science can be a good descriptor of the state of the science.

    ReplyDelete
  5. Yeah, I think it's important for us to scrutinize the science, and treat it as serious science until it's shown that it's not. We can do this much more quickly than it would normally be done through formal channels.

    ReplyDelete
  6. Hi there, You wanted to leave a comment on my blog on an article written by the OC Register that was showing a vaccine link to autism. I didn't see a place to email you directly so I wanted to leave a comment for you here. I don't plan on posting your comment which simply leads the viewer to your blog. I accept and love my son for who he is. However, I do believe he has been harmed by exposure to vaccinations (viral and immune issues, not necessarily thimerisol/mercury) among other things before the tender age of one years old. My son is aged 2.10 and I plan to look into things that will help him. He is pre verbal and I am not certain if your message is to sit back and wait for that to change..I am sure that works for some kids but I don't sit back and wait for anything to happen naturally because for these kids it seems something hinders "nature" from taking her "course." I will leave it at that. Good luck to you and your family! Take care.

    ReplyDelete
  7. One last nudge - the Archives of Pediatric and Adolescent Medicine is doing a special issue on autism. Here are their author instructions: http://archpedi.ama-assn.org/misc/ifora.dtl

    You've already done all the hard work of analysis! It deserves a wider audience.

    ReplyDelete