So what is Kirby so worked up about? It appears that he has learned from Dan Olmstead of a NIH report titled "Thimerosal Exposure in Pediatric Vaccines" about potentially serious methodological problems in ecological studies that use the VSD database. I don't know if Kirby read the report, but I did. This is actually a very well thought out report which ideally should have been done 6 years ago. But Kirby seems to think this report is evidence of an association between autism and thimerosal, or at the very least evidence that undermines lack of evidence of an association. Something like that.
Verstraeten et al. (2003), the VSD study in question, found statistically significant associations between thimerosal and a couple neurological outcomes in some HMOs, but the associations were not consistently found in other HMOs. Because of these conflicting findings, the study authors called the results "neutral", meaning that the study can't tell us if thimerosal is associated with neurodevelopmental disorders or not. Now the NIH has come up with a list of methodological problems that put in further doubt any VSD findings by Verstraeten et al. in relation to thimerosal. In other words, the neutrality (or non-informativeness) of the findings has been strengthened.
Surprisingly, Kirby's reading of the report conflicts with his other views, because it completely undermines the foundations of the Simpsonwood conspiracy theory. You see, Verstraeten et al. were supposed to have found significant associations between thimerosal and neurodevelopmental outcomes beyond those that were reported in 2003. But now Kirby is endorsing a NIH report which says that ecological studies on the VSD database, specifically those done by Verstraeten et al., are likely flawed. I'm not sure if Kirby realizes this. If he does, he's being duplicitous. If he does not, he just shot himself in the foot.
Furthermore, there is a quote by Thomas Verstraeten which is often used to argue in favor of the thimerosal association:
Personally, I have three hypotheses. My first hypothesis is it is parental bias. The children that are more likely to be vaccinated are more likely to be picked up and diagnosed. Second hypothesis, I don't know. There is a bias that I have not yet recognized, and nobody has yet told me about it. Third hypothesis. It's true, it's thimerosal. Those are my hypotheses.
Verstraeten was likely being honest in his assessment. What the NIH report has done, effectively, is provide many possible "Second hypotheses".
Kirby also believes the Verstraeten et al. study is the cornerstone of the case against the thimerosal hypothesis. It's not clear why he believes that is true. I think one of the best studies on the matter is Fombonne's out of Canada. Even if you don't care to analyze those findings, it's enough to consider that thimerosal was fully removed from pediatric vaccines in Canada in 1994, and the prevalence of ASD among Kindergarten children there is currently around 1%. You don't have to be an epidemiologist to realize what the implications of these facts are. Kirby thinks Fombonne's findings are inconclusive, but again, it is not clear what he bases that on. I imagine it's the very weak rebuttal by SafeMinds, an organization that is apparently no longer even putting minimal effort into arguing its case.
Kirby makes light of the Andrews et al. (2004) study out of the UK. That study found statistically significant negative associations ("protective" effects) between thimerosal and autism. Kirby's interpretation of those findings is either naive or desingenuous. If thimerosal is phased out at the same time the incidence of autism increases, it is obvious that the study will find that children not receiving thimerosal tend to become autistic more often than those vaccinated with it. So the finding of a statistically significant negative association is not surprising. It doesn't mean thimerosal protects against autism, but simply that there were coincidentally divergent trends of thimerosal exposure and autism incidence. Incidentally, if a VSD study were done on the birth year cohort expanding, say, 1998 to 2002, it would also find a "protective effect" of thimerosal, no doubt.
Back to the NIH report. I actually welcome this report, and I'm not just saying this now that it has come out. Back on November 20, I analyzed claims about early drafts by Verstraeten et al. In that post I mentioned several ridiculous negative associations found between various outcomes and thimerosal. I argued that the findings of Verstraeten et al. cannot be taken at face value, and that while VSD has its merits, "this doesn't mean VSD is impervious to confounds beyond random error." I further stated that "VSD still records outcomes based on existing diagnoses, not the results of whole population screenings. In a cohort of children expanding many years, recent diagnoses are not necessarily equivalent to older diagnoses. Surely, things like coincidental trends and left censorship could have an impact on any findings." What does the NIH report say?
The panel identified several serious problems that were judged to reduce the usefulness of an ecologic study design using the VSD to address the potential association between thimerosal and the risk of AD/ASD. These included uncertainties in case ascertainment, heterogeneity of business practices within and across managed care organizations (MCOs) and their systematic changes over time, misclassification of exposure status using comparisons of before vs. after removal of thimerosal from most childhood vaccines, and the inability to control for temporal changes in awareness, diagnostic practices and potential confounding factors.
I'm glad there are people who think about these things in more professional depth, believe in the self-correcting principle of science, and are willing to criticize existing scientific work, no matter who did it.
The NIH panel makes some interesting proposals for future VSD studies and they are apparently taking suggestions from the autism community. I think the biggest potential problem of a VSD study is that of coincidental trends (specifically, thimerosal exposure rising at the same time as administrative incidence of ASD in the early 1990s). That is why I think a VSD study should not look at a cohort expanding many years, but maybe only one year. This is difficult, nonetheless, because the number of children with a given outcome in an HMO is small. An alternative is to look at a cohort where the average thimerosal exposure was roughly constant over time (e.g. 1995 to 1998).
Other confounds are not as easy to control for I'd imagine. Either way, more studies would be good to have. Matters that appear unresolved don't help things move along.