Evidence Based Practice is Bul$*%!@ , Let's Fix It!
The McNamara fallacy, named for Robert McNamara, the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations and ignoring all others. The reason given is often that these other observations cannot be proven. The fallacy refers to McNamara's belief as to what led the United States to defeat in the Vietnam War—specifically, his quantification of success in the war (e.g., in terms of enemy body count), ignoring other variables. -From Wikipedia
I remember going into my first day of research class during my masters program. We sat and learned the evidence based practice system that the psychology profession is based on. Put simply, evidence based practice is the system by which clinicians make sure that the techniques that they are using are backed by science. Evidence based practice means that psychotherapists only use interventions that research has proved are effective. Evidence is determined by research studies that test for measurable changes in a population given a certain intervention.
What a brilliant system, I had thought. I then became enamored with research journals. I memorized every methodology by which research was conducted. I would peruse academic libraries at night for every clinical topic that I encountered clinically. I would select studies that used only the best methodologies before I would believe that their findings had merit. I loved research and the evidence based practice system. I was so proud to be a part of a profession that took science so seriously and used it to improve the quality of care I gave patients.
There was just one problem. The more that I learned about psychotherapy the less helpful I found research. Every expert that I encountered in the profession didn’t use methods that I kept reading about in research. In fact there were actually psychological journals from the nineteen seventies that I found more helpful than modern evidence based practice obsessed publications. They would come up in digital libraries when I searched for more information about the interventions my patients liked. Moreover I found that all of the most popular and effective private practice clinicians were not using the techniques that I was reading about in the scientific literature either. What gives?
Psychological trauma and the symptoms and conditions psychological trauma causes (PTSD, dissociative disorders, panic disorders, etc) are some of the most difficult symptoms to treat in psychotherapy. It therefore follows that patients with disorders caused by psychological trauma would be one of the most studied populations in research. So what are the two most commonly researched interventions for trauma? Prescribing medication and CBT or cognitive behavioral therapy.
One thing that most of the best trauma therapists in the world all agree on is that CBT and medication don’t actually process trauma at all, but instead assist patients in managing the symptoms that trauma causes. As a trauma therapist it is my goal to help patients actually process and eliminate psychological trauma. Teaching patients to drug or manage symptoms might be necessary periodically, but surely it shouldn’t be the GOAL of treatment.
I’m mixing metaphors but this image might help clarify these treatment modalities for those unfamiliar. Imagine that psychological trauma is like an allergy to a cat. Once you have an allergic reaction to the cat, a psychiatrist could give you an allergy medication like benadryl. A CBT therapist would teach you how to change your behavior based on your allergy. They might tell you to avoid cats or wash after touching one. A therapist practicing brain based or somatic focused trauma treatment would give you an allergy shot to help you develop an immunity to cats. The CBT patient never gets to know a cat's love.
I don’t have time to explore here why therapy that gives patients scripted ego management strategies like CBT took over the profession after the nineteen eighties . If you have any interest in why check out my article Is the Corporatization of Healthcare and Academia Ruining Psychotherapy?. Suffice it to say that insurance and american healthcare companies pay for much of the research that is conducted and they like to make money. CBT and prescribing drugs are two of the easiest ways for those institutions to accomplish those goals.
Many of the MOST effective ways to treat trauma use the body and deep emotional brain system to assist patients in processing and permanently releasing psychological trauma.
Unlike CBT the modalities that accomplish this are not manualizable. They can not be reduced to a “if they say this then you say that'' script. Instead somatic therapies often use a therapist's intuition and make room for the patient to participate in the therapeutic process. CBT on the other hand is a formula that a therapist is performing “correctly” or “incorrectly” based on their adherence to a manual. Right now hospitals are rushing to program computers to do CBT so they can reduce overhead. Yikes! Think of a therapy experience like the self checkout at Walmart.
If myself and most of the leading voices of the profession agree that newer brain based and body based therapy modalities are the future of trauma treatment then why hasn’t research caught up yet? To stop this article from becoming a book I will break down the failure of modern research to back the techniques that actually work in psychotherapy.
It’s Expensive - cash moves everything around me
Research studies cost tons of money and take tons of time. Researchers have to plan studies and get the studies cleared with funders, ethics boards, university staff, etc.. They then have to screen participants and train and pay staff. The average study costs about $45,000.
I would love to do a study myself on some of the therapy modalities that we use at Taproot Therapy Collective, but unfortunately I have to pay my mortgage. Studies get more expensive when you are studying things that have more moving parts and variables. Things like, Uh… therapy modalities that actually work to treat trauma. These modalities are unscripted and allow a clinician you use their intuition, conventional wisdom, and make room for a patient to discover their own insights and interventions.
Someone has to pay for those studies and those someones usually aren't giving you that money without an agenda. Giant institutions are the ones most likely to benefit from researching things like prescribing drugs and CBT. They are also the ones that are the most likely to be in control of who gets to research what.
The sedative drugs prescribed to treat trauma work essentially like alcohol, they dull and numb a person's ability to feel. Antidepressants reduce hopelessness and obsession. While this might help manage symptoms, it doesn't help patients process trauma or have insight into their psychology. Antidepressants and sedatives also block the healthy and normal anxieties that poor choices should cause us to feel. Despite this drugs are often prescribed to patients that have never been referred to therapy.
For all the “rigorous ethical standards” modern research mandates, it doesn't specify who pays the bills for the studies. Drug companies conduct the vast majority of research studies in the United States, and those drug companies also like to make money. Funnily enough most of the research drug companies perform tends to validate the effectiveness of their product.
Does anyone remember all the 90s cigarette company research that failed to prove that cigarettes were dangerous? All those studies still passed an ethics board review though. Maybe we should distribute research money to the professionals wha are actually working clinically with patients instead of career academics who do research for a living. At the very least keep it out of the hands of people who have a conflict of interest with the results.
This leads me to my next point.
3. We Only Use Research to Prove Things that we Want to Know - Duh!
The thing that got left out of my research 101 class was that the research usually has an agenda. Even if the science is solid there are some things that the commissioners of the studies don't want to know. For example, did you know that the D.A.R.E. program caused kids to use drugs? Uh..yeah, that wasn't what patrons of that research study meant to prove, so you never heard about it. It also didn't stop the DARE program for sticking around for another 10 years and 10 more studies that said the same thing.
Giant institutions don’t like to be told that their programs need to change. They wield an enormous amount of power over what gets researched and they tend to research things that would validate the decisions that they make, even the bad decisions.
If you want research to be an effective guide for clinicians to use evidence interventions then you have to research all modalities of psychotherapy in equal measure. When the vast majority of research is funneled into the same areas, then those areas of medicine become better known clinically regardless of their validity. When very few models of therapy are researched, then those few models appear, falsely, to be superior.
Easier and cheaper research studies are going to be designed and completed much more often than research studies that are more complicated. Even when institutional or monetary control of research is not an issue, the very nature of research design means that it is trickier to research things like “patient insight” than it is to research “hours of sleep”. This leads me to my next point.
4. Objective is not Better - People are not Robots
CBT was designed by Aaron Beck to be a faster and data-driven alternative to the subjective and lengthy process of Freudian Psychoanalysis. Beck did this by saying that patient’s had to agree on a goal that was measurable with a number, like “hours of sleep” or “times I drank” and then complete assessments to see if the goal was being accomplished. Because of this CBT is inherently objective and research based. CBT is therefore extremely easy to research.
This approach works when it works, but a person's humanity is not always reducible to a number. I once heard a story from a colleague who was seeing a patient who had just completed CBT with another clinician to “reduce” marijuana use. The patient, who appeared to be very high, explained that his CBT clinician had discharged him after he cut back from 6 to only one joint per day. The patient explained proudly that he had simply begun to roll joints that were 6 times the size of the originals.
That story is humorous, but it shows you the irony of a number based system invading a very human type of medicine. Squeezing people and behavior into tiny boxes means that you miss the whole person.