Wednesday, August 29, 2007

My Anecdotal Evidence is Better Than Everyone Else's!

Whenever someone claims that they know some biomed treatment is useful, even in the absence of clinical trials, I like to remind them of Secretin. See Lessons from Secretin (Volkmar, 1999). Secretin was once believed to be a "cure" of autism. Bernie Rimland was a fan of Secretin and claimed it had an improvement rate of 70% to 75%. The mom who first noticed her son started to speak after getting Secretin had been referred to as a "hero" for discovering the "cure". There were even multiple case reports and at least one non-controlled study noting substantial improvement with Secretin.

Then the double-blind placebo-contolled studies came. Nearly all of them showed no difference in outcome between Secretin and placebo. Some of them seemed to show that Secretin was, in fact, a little bit worse than placebo. (That's right, something that was believed to be a "cure" might have been causing damage instead). The studies generally demonstrated what are now believed to be robust placebo effects in autism. Naturally, Bernie Rimland was reluctant to accept the results of the studies, I'm not sure for how long.

There's obviously some level of self-deception in discussions about biomed treatments that appear to "work". When I bring up Secretin, some parents will say that it's a good point, but then they'll turn around and use what I will call the my-anecdote-is-better-than-everyone-else's gambit. They will say things like "I've been very careful in my observations of the eye contact improvements" and so on. They will claim self-deception could never apply to them, and seem to be offended by the suggestion. They also often appear to mistake "placebo effects" with "you are lying".

You see, even if a non-verbal child turns into a great conversationalist at around the same time that you give him [horseshit of choice here], this doesn't prove the treatment had any effect. It's not just that correlation does not imply causation, but also that anecdotes of this nature occur all the time in autism. Consider that if 10-15% of all autistic children lose their diagnosis eventually, there must be tens of thousands of such children around the world. One more anecdote of that nature doesn't tell us much.

You will find anecdotes of recovery for nearly anything you can think of: homeopathic remedies (chemical formula: H2O), exorcism, holding therapy, removal of TV, juice from an Amazonian fruit, etc. Heck, even Bettleheim had glowing testimonials.

Really, why should I take any anecdote on the latest biomed fad seriously?

35 comments:

  1. Thank you, Joseph, for this important posting.

    I am packing our bags for Arizona! :)

    Joe

    ReplyDelete
  2. Well what about acupuncture? Pretty convincing studies on that :-)

    http://www.bbc.co.uk/worldservice/sci_tech/highlights/010625_tongue.shtml

    ReplyDelete
  3. Tongue accupuncture? Ouch.

    ReplyDelete
  4. From what I've heard of acupuncture, it doesn't cause any more than minor pain, actually. It's one of those treatments that doesn't do any harm, unless the child is held down and develops a permanent fear of needles...

    ReplyDelete
  5. Both Scientific and anecdotal evidence have type I and type II errors. Neither is perfect or 100% proven.

    ReplyDelete
  6. Both Scientific and anecdotal evidence have type I and type II errors. Neither is perfect or 100% proven.

    While both false positive (Type 1 error) and false negative (Type 2 error) occur in all kind of studies, it is only in properly conducted scientific studies that these types of error can be controlled for, measured, and reported.

    Nothing is 100% except for death and taxes. Everything else is a probability. There is greater than a 99% chance that I won't die before I finish typ

    ReplyDelete
  7. Correct. Double-blinding does not eliminate the errors, but it controls for them. As long as the no breaching of the blinding, the rest is up to statistical significance.

    ReplyDelete
  8. A secretin study was carried out here on PEI in 1999. The study & results can be found here: http://web.archive.org/web/20050212145607/http://www.isn.net/~jypsy/secrepei.htm
    Alex was not involved.

    ReplyDelete
  9. Chaoticidealism:
    "...that doesn't do any harm, unless the child is held down and develops a permanent fear of needles..."

    Don't forget to think about the possibility of infection by the use of unclean needles Please!

    ReplyDelete
  10. I'm waiting with bated breath, anticipating whatever bollocks Fore Skin tries to post here...

    I need a good laugh!

    ReplyDelete
  11. if john best jr comes back. i'd like to ask him why he repeatedly denied being 'abe lincoln' on the autism speaks board, and then later admitted to it on his blog.
    i'd ask him too if it's true that jb handley is on the a.s. board as ihatethefda and if lisa handley is lisa782. i could ask handley on the handley blog but i think he won't post it. funny that handley has abused people on their blogs but refuses to allow people to say any unsettling things him or his bloggers on his blog.

    ReplyDelete
  12. Secretin was quite controversial.... but they are all not that controversial. To be so black and white is such a shame. I suppose each person needs to justify to themselves to get through the day. I believe that everyone has the right to reach their full potential. There are a lot of biomedical and natural treatments that do help. What you put in = what you get out.

    Here is some other food for thought. The first vaccine was created in 1930. Kanner reported his first case of "autism" in 1931.

    ReplyDelete
  13. According to Dr Treffert, quoted on the Neurodiversity blog
    "Dr. Down reported regressive autism 125 years ago (based on his 30 years of observations) and, without doubt, autism, like mental retardation, has been around as long as man has been around."

    ReplyDelete
  14. Secretin was quite controversial.... but they are all not that controversial.

    Oh, I see. Forget the double-blind studies. Here's a better test. If it's controversial, it doesn't work. If it's not controversial, it works.

    Thanks. I guess chelation therapy must suck then.

    There are a lot of biomedical and natural treatments that do help.

    I feel the argument in my post must have failed completely. Here we go again. On what basis do you believe some unproven treatments work, Anon?

    ReplyDelete
  15. Great title -- and a point that's much needed, too.

    ReplyDelete
  16. Kanner has been quite quick to describe in 1931 a condition created barely a year ago by vaccines. Those scientists who found the link between thalidomide and birth defects, and between HIV and AIDS, must have been fools, because it took so much longer.
    BTW where does the word "vaccine" come from? Isn't it from the use of cowpox virus to immunize against smallpox? Dr. Jenner would be quite surprised to learn that vaccines didn't exist prior to 1930.

    ReplyDelete
  17. Some other things that happened in 1930:

    - Mickey Mouse made its first appearance.

    - The 3M company markets Scotch Tape.

    - Hostess Twinkies are invented.

    - Jake paralysis (organophosphate-induced delayed neuropathy) outbreak occurs in United States. (Caused by Jamaican Ginger Extract, which was popular during the prohibition).

    ReplyDelete
  18. BTW, vaccines were invented in 1796. Kanner first characterized autism in 1943. What anon refers to is that Kanner's patients were below the age of 11. The oldest patient must have been born in 1933.

    Regardless, I would think that this notion that autism didn't exist before the 1930s would be erradicated by now. It's completely ridiculous.

    Brad Handley, of Generation Rescue no less, did a phone survey of vaccinated and unvaccinated children. He found the rate of autism among unvaccinated children to be 6 times that of the general population. The survey is garbage, OK, but even Brad Handley couldn't possibly claim that autism doesn't exist among the unvaccinated right now.

    ReplyDelete
  19. While looking up some inventions that occurred in the 20's and 30's I read Joseph's comment that:

    ...- Mickey Mouse made its first appearance.

    - The 3M company markets Scotch Tape.

    - Hostess Twinkies are invented.

    - Jake paralysis (organophosphate-induced delayed neuropathy) outbreak occurs in United States. (Caused by Jamaican Ginger Extract, which was popular during the prohibition). ...


    Great stuff!

    I would add-

    -Bubble gum was invented in 1928

    -Car radios were invented in 1929

    and the big one-

    -Baby food was invented in 1931!

    Explain that, huh????

    Joe

    ReplyDelete
  20. Great to see you blogging again Joseph :o)

    ReplyDelete
  21. -Baby food was invented in 1931!

    Coincidence? I think not! :)

    Kev (and everyone else who said the same) - thanks.

    ReplyDelete
  22. I'm repeatedly disappointed to find the side-taking in this hugely political "world" of autism.

    It seems that every personal blog regarding autism is reflective of one side or the other. Or if the blog itself in neutral, the side-taking is reflected in the comments. Either autism is to be accepted as a gift or something to be defeated, apparently with no middle-ground.

    What about being open-mindedness instead of being at war with each other.

    What causes autism hasn't been proven yet, whether it's evolution, or mercury, or whatever, except in the cases of gene mutation such as Fragile X.

    Doing "something" gives parents a sense that they're doing something to help their children live a happier life. If that "something" isn't harmful, how is that worse than doing nothing and ASSUMING that your child isn't suffering from being aware of his/her differentness, ridicule from society, being embarrassed by lack of social or adaptive skills.

    I don't know of any parents, personally, who try treatments (those which are non-harmful)just to try to "cure" their child or eliminate some "curse". Most parents try interventions very specific for problematic behaviors that interfere with the child's quality of life. For instance, ABA can be used to help decrease aggressive behaviors that effect school placement and later whether a child can continue to even live with others peacefully. At the same time, ABA can be used to help a child communicate effectively, what he/she is upset about so that aggressive behavior isn't necessary to get the message across. And yet there are people who HATE ABA and everyone who uses it.

    I'm just using ABA as an example of how close-minded certain individuals have become.

    Who's to say what caused autism in my child or what I should or should not do about it? I'm talking about things within reason.

    Is there no middle-ground left on the issue of autism, or has it just become two sides?

    Why can't we celebrate the uniqueness of autism but agree that we all, even those of us on the spectrum, have room for improvement especially in the areas of social skills and tolerance of others and their opinions?

    I know, I'm an idealist. I can appreciate everyone who has strong opinions and often those people push science in new directions but why does there always have to be name-calling involved and such divisiveness?

    Aren't we all just interested in what's best for our kids with autism and accepting their uniqueness and helping them to live within society in a way that helps them to be happy, with their self-esteem intact?

    ReplyDelete
  23. Anon: Middle-gound is always relative. What to you might seem moderate, to me might seem extreme and misguided. If you're uncomfortable with strong opinions, I'd recommend other blogs.

    Doing "something" gives parents a sense that they're doing something to help their children live a happier life. If that "something" isn't harmful, how is that worse than doing nothing and ASSUMING that your child isn't suffering from being aware of his/her differentness, ridicule from society, being embarrassed by lack of social or adaptive skills.

    Doing biomed is not the opposite of "doing nothing". False dichotomy, and straw-man.

    Autistic children suffer from the way the environment approaches them, no doubt. But do you think they really suffer solely from being aware of their differences? I'd say that can only happen if they have learned that different=bad.

    ABA is not really part of the quackery debate, BTW. It's mainstream. The evidence on it is poor in my opinion, and I don't believe it's harmless, but at least is has been studied a lot.

    If there were proven, long-term safe, ethical treatments for autism, I assure you, neurodiversity advocates would have trouble arguing against them. It would become a debate about choice at that point.

    And here's the thing. How do you determine something is not harmful, as you claim? It's the same issue. You either have evidence for that claim or you don't. Some of us don't believe in experimenting with our kids with stuff whose safety is unknown.

    ReplyDelete
  24. "if john best jr comes back. i'd like to ask him why he repeatedly denied being 'abe lincoln' on the autism speaks board, and then later admitted to it on his blog."

    I can tell you. It's because he's a piss-arse liar, basically.

    "i'd ask him too if it's true that jb handley is on the a.s. board as ihatethefda and if lisa handley is lisa782."

    Probably are. They're not that clever, really.

    "i could ask handley on the handley blog but i think he won't post it."

    Dmmn right he won't. You know why...

    "funny that handley has abused people on their blogs but refuses to allow people to say any unsettling things him or his bloggers on his blog."

    ... and that's why: he's a bullying git with insufficent balls to cope with it when he gets challenged on anything he says.

    Just your basic toss-pot wuss-bully, who can't handle getting back what he gives out.

    ReplyDelete
  25. Actually, horse shit will make ASD symptoms better. All manure that has been aerobically composted is a rich source of nitrifying bacteria. These will raise the basal NO level and improve ASD symptoms.

    The reason that ASD symptoms are so sensitive to the placebo effect is because the placebo effect occurs by raising NO. ASD symptoms are made worse by lowering NO, so all placebos will make ASDs better.

    ReplyDelete
  26. Usually when I see reference to the Horse Shit story, the link given is not from the Autism-Mercury list but from the "ADHL" (St. Johns autism list/bit.listserve.autism) from back in 1996, posted by yours truly. http://groups.google.com/group/bit.listserv.autism/browse_thread/thread/627665d97ac6edee/c60933379e3af071?#c60933379e3af071

    11 years ago.... geez, I've been at this a while....

    ReplyDelete
  27. Anon,
    I'll be happy to explain why I used an alias if you'll start by using your name so I know who I'm talking to. Perhaps everyone else who uses an alias will all come out of the closet and identify themselves. We could have a coming out party and identify all of the cowards.

    Joe,
    The proven, long-term, safe, ethical treatment for mercury poisoning, as practiced by mainstream medicine and confirmed by your Dr Orac is chelation. Now all you have to do is accept what CDC employee Verstraeten told us in 1999 which was confirmed by 51 scientists at Simpsonwood in 2000, that thimerosal had caused the autism epidemic.

    It may be too late to save our kids since the mercury turned their brains to mush before we could get it all out but it would be nice if you stopped advising us that leaving the mercury in the brain is a good idea.

    ReplyDelete
  28. John & David: Try to behave yourselves please. I'm just saying that preemptively. Keep it civilized as much as possible.

    daedalus2u: Sorry, but I couldn't help laughing by your managing to play your NO hypothesis promotion into the horse shit spoof.

    jypsy: Thanks for the link. I had seen it in Usenet before, but I didn't realize it was 11 years old.

    John: Chelation is believed to be an effective treatment of severe heavy metal poisoning if you catch it in time. Additionally, the treatment usually lasts a few months only. If the poisoning is moderate (e.g. children with moderate levels of lead), double-blind studies show that chelation with DMSA is useless. Further, DMSA, and all chelators for that matter, are serious business. They chelate minerals in addition to heavy metals. In studies in rats, DMSA appeared to cause brain damage.

    To summarize, given that autism is almost certainly not related to heavy metal poisoning, and given that you have not tested your son for heavy metals in the blood, and given the unprecedented length of treatment compared to what you find in the literature, it's clear you don't know what the hell you're doing, John.

    ReplyDelete
  29. Fiona Smith-Fontaine8/30/2007 10:50 PM

    John Best Jr.
    " We could have a coming out party and identify all of the cowards."

    So you are saying that JB Handley and Lisa Ackerman or Lisa Handley and Smith are cowards? I'd agree with that.

    ReplyDelete
  30. "In studies in rats, DMSA appeared to cause brain damage."

    Water appears to cause death in humans too. It's all to do with appropriate dosing and timing. Those darn details!

    ReplyDelete
  31. Are you saying that they overdosed the rats with DMSA on purpose? The study was an animal model of chelation treatment. Whatever its methodological flaws (including the fact that it may not apply to humans) the prudent thing to me would be to err on the side of caution.

    ReplyDelete
  32. Sorry, I didn't realize it was a spoof when I first read it. Some of us are too literal I guess. I have actually done research on the NO release from chicken manure, it is a massive and gigantic NO source.

    Since it actually wasn't tested, the question of whether horse manure or any other kind of manure would have any effect is still unknown. We already know that chelation doesn't work because chelation has been tested. So I don't seem them as the same at all.

    ReplyDelete
  33. Before they dismiss treatments, they should study autism and divide up the cases into subgroups. Depending on the subgroup, they could test different treatments. As anyone who is involved in the autism community knows, there are kids with ASD who have seizures, kids with allergies, kids with GI problems and chronic diarrhea, kids who never get sick. Perhaps these "anecdotal" treatments work miraculously for a subgroup. Until they study the subgroups the treatments will be claimed to be ineffective. Come to my house if you want to see treatments in action. Seeing is believing. But my son had severe allergies, the genes for celiac and an overnight regression. I am sure others with different symptoms won't respond to the treatment my son has gotten better on. How are you so sure nothing gets anyone better because one study with secretin didn't reveal any benefits??? A study not finding a benefit doesn't mean the treatment doesn't work--it means the treatment didn't work in the study for the people that were using it. I love my son as he is, but I sure am grateful his pain and agony and diarrhea are over and he is speaking and connected. And I thank god for those parents who moved to university towns twenty years ago and did research...research that saved my son from being trapped inside himself. Also, a mainstream doctor gave my son secretin when he had an endoscopy...I don't think it was the thing that got him better, but she wasn't a DAN doctor--she was actually a doctor who did a study disproving secretin and had huge objections to people trying it on kids in unsafe setting, but she used it on my son thinking it might help him. She didn't believe in biomed at all, but she must have seen something when she did that study which suggested my son might benefit from secretin. I'm not saying secretin works...but it might work for some people and not work in a study because of the way the study is set up. If it works for 1 in 1000 kids and the study only has 85 kids...perhaps they didn't include the 1 in 1000 kid that it would work for. it doesn't mean it doesn't work...it means the study didn't find that it works.

    ReplyDelete
  34. @gfcfmom: What you're saying basically is that the studies can't prove a negative. That's true enough. But they also haven't proven a positive. What they do demonstrate is that there's no basis for making general recommendations about certain treatments which are, let's face it, completely fabricated, without a scientific foundation. It's very unlikely that the treatments will help a random child, and it's possible they don't help anyone at all.

    BTW, they do sometimes test subgroups. For example, some of the Secretin studies did look at children with diarrhea.

    ReplyDelete