Saturday, March 11, 2006

Open Letter to Journal of Physicians and Surgeons

Dear Editor:

I recently had the opportunity to review the paper titled Early Downward Trends in Neurodevelopmental Disorders Following Removal of Thimerosal-Containing Vaccines published in your Journal in the Spring of 2006.

The purpose of this communication is not to point out methodological flaws in the paper's findings and conclusions. The flaws are already well known, and this will likely be addressed in the scientific literature in due time.

I am writing to point out that there exist significant factual errors presented in the paper, due mostly to demonstrably misleading use of certain terminology. The magnitude of these errors is such that I have no choice but to urge you to retract this paper at your earliest convenience.

In particular, the following statements referring to CDDS trends are at issue:


Significant decreasing trends in newly diagnosed NDs...

Trends in New Cases of Autism entered into the CDDS...through Oct. 4, 2005 is significantly decreasing...


It is not clear how the authors define or calculate "newly diagnosed NDs" and "New Cases" given that no terminology section is provided in the paper. Nevertheless, Figure 3 in combination with a quick look at the publicly available CDDS data shows how the authors calculate "New Cases". Evidently, it is defined by the authors as the difference between the number of autistic clients in one quarter and the number of autistic clients in the previous quarter.

The terms "newly diagnosed NDs" and "New Cases" clearly can only be interpreted by your readers as meaning "individuals who have just been diagnosed". This terminology is erroneous at best, and purposefully misleading at worst. The number of clients with a new diagnosis cannot be determined by substracting the numbers from one quarter minus the numbers from the previous quarter, as explained by CDDS itself in its document titled Data Interpretation Considerations and Limitations:

Differences in the numbers from quarter to quarter reflect the net changes between individuals who are newly reported (i.e., included in the later report but not included in the earlier report) and individuals who dropped out (i.e., included in the earlier report but no longer included in the later report).


The error of assuming that quarterly differences represent "new cases" is so common that the CDDS has obviously gone out of its way to clarify it.

This is not a minor error that can be brushed aside. CDDS clients can go from a status of active to a status of inactive or closed when they move out of the state, die, are no longer found to be eligible, or choose to discontinue pursuing eligibility. Given the increasing population of autistic clients in the CDDS, the importance of these considerations cannot be overstated.

Additionally, the fact that the population of autistic clients in the CDDS continues to grow at an annual rate of 10.7% as of Dec. 2005 (a rate much higher than the general population growth in the state of California) shows that an actual decrease in the number of newly diagnosed cases is not likely occurring. The population is simply starting to level off.

Once again, given the magnitude of this error and the fact that CDDS itself has tried to prevent this exact error from occurring, I believe the only appropriate course of action is to retract this paper immediately. I trust that given your reputation as a peer-reviewed Journal, you will agree.

If you have any doubts about the facts I have presented, I suggest you contact CDDS directly and discuss the issue with them.

Sincerely,

Joseph

P.S. I will copy CDDS, and will also be blogging this.

7 comments:

  1. Nice letter Joseph,
    If they start publishing retractions for significant factual errors the errata section will be longer than the papers.

    ReplyDelete
  2. Nice letter Joseph,
    If they start publishing retractions for significant factual errors the errata section will be longer than the papers.


    It will be interesting to see if they respond at all or simply ignore it.

    ReplyDelete
  3. Great letter Joseph.
    I'll be surprised if they respond with more than a thanks for your concern. Infomercials rarely make retractions.

    ReplyDelete
  4. I noticed a funny thing in Figure 3 of the paper. Check out the downward trend line. Doesn't it look like they just drew that whereever the heck they wanted? They try to make an obvious gradual shift in population growth look like a sharp turn.

    ReplyDelete
  5. every general statement you make regarding this article:
    1. methodological flaws
    2. factual errors
    3. misleading terminology

    can be said of the poorly written article on climate change:
    http://www.jpands.org/vol12no3/robinson.pdf

    ReplyDelete
  6. Yeah, not surprising that JPANDS would publish global warming denialism too.

    JPANDS is not actually a peer-reviewed journal. See Orac's entry and from there click over to Kathleen's entry.

    ReplyDelete