ASD - Both SexesStrikingly, the survey found that while 3.01% of all vaccinated children had an ASD diagnosis, about 3.73% of all unvaccinated children did. That's right. The survey found autism to be more common among the unvaccinated. While this difference is not statistically significant, I think Kev was correct when he characterized it as disastrous for Generation Rescue's political goals. It also contradicts years of claims by GR itself and Dan Olmstead in regards to the Amish and so on.
Later on I will explain how they managed to spin this result.
[Note: 'Statistical significance' means that we can assert the survey found a difference, with 95% confidence.]
As noted shortly after Kev posted his analysis, both ASD rates are very high relative to the consensus prevalence of autism. Let's compare them to a recent CDC phone survey which found the diagnosed prevalence of autism in the US to be 0.57%.
What might explain the discrepancy? For that, we need to consider the survey's methodology. This was an automated phone survey. That is, households are contacted at random by a computer with a short recorded introductory message, in this case one that read as follows.
This is SurveyUSA calling Sonoma County parents with a private, confidential survey about vaccinations and children's health. If you have a child age 4 to 17, press 1. Otherwise, press 2.
Most persons answering the phone will hang up at this point. But that would be fine if the introduction weren't so obviously biased. Evidently, parents of autistic children who are familiar with the anti-vaccination debate will be more likely to continue with the survey. In fact, given the results of the survey, it is clear that households with autistic children are about 5 times more likely to continue with the survey than households in the general population. (We can also conclude something about SurveyUSA's best-case response rates, but that's not important to this analysis).
This bias in the introductory message ('introduction bias' from now on) easily explains the high prevalence of ASD found in the vaccinated population of children. Consider that the vast majority of children are vaccinated.
I do not, however, believe it fully explains the high rate found in the unvaccinated group. I say this because households with unvaccinated children are no doubt likely to continue with the survey, regardless of the presence of autistic children. This is a key point. In fact, Generation Rescue was somewhat surprised to find 6% of children were completely unvaccinated. Heightened interest in the part of parents of autistic children is not going to be as significant a factor in the unvaccinated group.
Therefore, a second bias, in the opposite direction, seems necessary to explain the high prevalence of autism (6.5 times the CDC prevalence) in the unvaccinated population of children. I will call this the 'genetic bias', one that is well known. That is, parents of autistic children will often not vaccinate subsequent siblings, and such siblings have a considerable higher rate of autism compared to the general population. Even in the relatively small group of parents who blog, I know of two parents who have unvaccinated autistic children: Not Mercury and Kim Stagliano.
ASD - Boys vs. GirlsIn boys, vaccination was found to be a non-significant risk factor relative to ASD (OR = 1.17). In girls, however, vaccination was found to be a statistically significant "protective" factor (OR = 0.37). That is, girls are almost 3 times more likely to be diagnosed with ASD if they are not vaccinated, according to the survey results.
The Generation Rescue leadership must not believe the results of their own survey. Otherwise, I'd expect them to post a warning such as the following on their website.
Please Note: We believe vaccines pose a non-significant risk of autism in boys, but they are a protective factor when it comes to girls. We urge parents to vaccinate girls.
OK. They might believe only some of their results, or they might have decided that throwing girls under the bus is the politically beneficial course of action.
It seems far fetched that something would be a risk factor in boys, but a protective factor in girls. From a quick Google search I conclude this would be unprecedented. Nevertheless, the boy vs. girl discrepancy in the survey is of interest. How might it be explained?
I first checked if the introduction bias is the same across boys vs. girls. It turned out to be close. I calculated the CDC prevalence of autism for boys vs. girls given the ratio reported in the press as "nearly 4:1". The resulting prevalence was 0.91% for boys and 0.23% for girls. The resulting bias factors (survey prevalence divided by CDC prevalence) were 5.07 and 5.57 respectively. This tells me that parental interest in the anti-vaccination issue is not significantly skewed by the child's sex.
It follows that the genetic bias must be skewed, and this does appear to be the case. The bias factors in the unvaccinated group were 4.33 for boys and 15.17 for girls. That's quite surprising. To emphasize, unvaccinated girls participating in the survey were found to be diagnosed with ASD 15 times as often as in the general population of girls.
Either not vaccinating girls is a significant cause of ASD, or familial autism is more common in the families of autistic girls. (Any other explanations?) I'm inclined to believe the second explanation, though I haven't found much support for this in the literature. Tsai et al. (2005) comes close. Either way, this seems to be an interesting direction of future research.
"Like" Autism and ADHDSo far we have noted that the survey did not find vaccines to be a significant risk factor for ASD, either in boys or children in general. It did find vaccines to be a "protective" factor in girls. Despite this, Generation Rescue made the following claim.
We surveyed over 9,000 boys in California and Oregon and found that vaccinated boys had a 155% greater chance of having a neurological disorder like ADHD or autism than unvaccinated boys.
(source) (Emphasis mine)
What they did is aggregate the data for ADHD and autism and made a general claim about "neurological disorders". The claim is technically true. I'm sure they could've aggregated asthma and diabetes and made a general claim about health outcomes, but that would've been too transparent. The statement is clearly not a bastion of honesty. (I'm sure readers can tell I'm holding back on my characterization of the statement).
Another thing they did in their survey information page is cherry pick the highest risk ratio they found for boys with ASD, the one for plain autism, which is 1.61. They did not mention the ASD risk ratio, 1.17, which not as impressive, or the risk ratios for PDD-NOS or Asperger's in boys.
ADHDThe survey did find significant risk ratios in the ADHD population, and these deserve to be analyzed in their own right. About 11.02% of the vaccinated children were reported to have ADHD vs. 5.15% of the unvaccinated children.
[Note: What I'm calling ADHD here corresponds to both ADD and ADHD in the survey results. Concensus in Psychiatry is that there's just one condition called ADHD and not two.]
The rate of ADHD in the vaccinated group is a bit high, although not by much. A 2003 CDC survey found that 7.8% of all children in the US had an ADHD diagnosis. (It did not vary a lot from state to state). This tells me that the introduction bias exists in ADHD too, but is not as significant as in ASD households, and this makes sense. The vaccine hypothesis is not as politically hot in the ADHD world. It follows that the genetic bias will also not be significant, because it's unlikely that parents will stop vaccinating subsequent siblings based on an ADHD diagnosis.
We're still left with a relatively low prevalence of ADHD in the unvaccinated population. But I think this can be easily explained by a third bias, which I will call the 'healthcare bias'. That is, households where children are not vaccinated are also less likely to seek diagnoses of ADHD. This is quite plausible when it comes to ADHD, given that it's such a subjective and controversial diagnosis. This bias probably exists in ASD as well, but I'd suggest it can't be nearly as common.
Partial VaccinationThe survey included a category called "Partially Vaccinated." No doubt Generation Rescue's intention was to find some sort of dose-response relationship between ASD and vaccination. But they did not. What they found is that ASD is over-represented among the partially vaccinated, more so than among both the vaccinated or unvaccinated. This is an obvious finding, in retrospect, considering that many parents will stop vaccinating a child once he or she is diagnosed with ASD. The survey designers must not have considered this problem.
Comment and PredictionsGiven that fairly obvious biases explain the survey results, and given that the risk factors involved are underwhelming, I do not believe that public resources should be spent on a methodologically valid follow-up. Generation Rescue could decide to invest on a follow-up, but I see a couple problems with this: (1) Eliminating the introduction bias will result in a much more expensive survey, as more calls (probably 5 times as many) will need to be made to achieve the same level of statistical significance; (2) Generation Rescue probably realizes that by eliminating the introduction bias, vaccines will be found to be a clear "protective" rather than a risk factor for ASD. (Yes, you may consider that a "dare").
Generation Rescue has brushed off the introduction bias. I don't know if they realize this bias is only unimportant if it affects both the vaccinated and unvaccinated populations equally. I have explained why it obviously does not.
Either way, a phone survey can only tell us about the approximate rates of diagnosed ASD in a population. It doesn't tell us much about the true prevalence of ASD. For that, we would need a whole-population screening. Even then, other biases, such as the genetic bias, need to be accounted for.
The most interesting finding of the survey, in my view, is that unvaccinated girls are 15 times as likely to have ASD than expected based on prior surveys. This is probably not explained by the introduction bias. I believe it would have to be explained by an extraordinary genetic bias or some other bias not considered thus far.