All approaches have valid explanations and solutions for the problems that clients bring to us. It makes sense to expand our theoretical horizons and learn multiple ways to serve client goals. Similarly, it also makes good clinical sense to be “evidence based” in our work. In truth, no one says, “Evidence, smevidence! It means nothing to my work—I fly by the seat of my pants, meander Willy Nilly through sessions, and rely totally on the wisdom of the stars to show the way.” Saying you don’t believe in the almighty evidence in tantamount to not believing in Mom or apple pie, or whatever your sacrosanct cultural icons happen to be. So what is the controversy about?

On the heels of the American Psychiatric Association’s development of practice guidelines in 1993, to ensure their continued viability in the market, psychologists rushed to offer magic bullets to counter psychiatry’s magic pills—to establish empirically supported treatments (EST). With all good intentions, the task force of Division 12 (Task Force on Promotion and Dissemination of Psychological Procedures, 1995) reviewed available research and catalogued treatments of choice for specific diagnoses based on their demonstrated efficacy in two RCTs. On one hand, the Division 12 Task Force effectively increased recognition of the efficacy of psychological intervention among the public, policymakers, and training programs; on the other hand, it simultaneously promulgated gross misinterpretations—that ESTs have proven superiority over other approaches, and therefore, should be mandated and exclusively reimbursed. Unfortunately, many now believe, to paraphrase Orwell, that some therapies are more equal than others.

The notion, however, that any approach is better than another is indefensible in light of the evidence covered extensively throughout The Heart and Soul of Change that support the outcome equivalence of the different models (the “dodo verdict”) as well as the relative influence of other factors than model and technique. I encourage you to dig a little deeper and bolster your ability to respectfully counter statements that suggest mandates for practice. Littell’s (2010) scathing commentary of ESTs in The Heart and Soul of Change is a good place to start. Littell provides a useful template for understanding the varied ways that findings can be distorted and evidence constructed from underwhelming results.

Like understanding anything else, there is a language involved here and it takes a bit of wading through tedious material. But it is worth it if you desire to counter mandates for specific approaches and promote the freedom for therapists to practice as they see fit according to client preferences and benefit. Our necessary pluralism, the theoretical breadth so important to resonating with clients and accentuating our development, is at stake, as well as our identity—ESTs suggest a therapist identity based on technical acumen in administering manualized, cookie cutter interventions (Duncan & Miller, 2006).

Efficacy over placebo, sham, or no treatment is not efficacy over other approaches, or what is called differential efficacy. In the minority of studies that claim superiority over treatment as usual (TAU) or another approach, you need only to ask one question of the investigation (see Duncan et al., 2004 and Sparks & Duncan, 2010 for a full discussion and examples): Is it a fair contest? Is the study a comparison of two valid approaches intended to be therapeutic administered in equal amounts by therapists who equally believe in what they are doing and who are equally supported to do it—are the therapists from the same pool with equal caseloads or is the experimental group specially selected, trained, and supervised by the researcher/founder of the approach, and have reduced caseloads?

I have never seen an advantage of any approach over another (or TAU) that wasn’t a lopsided contest that had its winner predetermined. Consider Trauma Focused (TF)-CBT, an approach to child sexual abuse that is getting a lot of press as the preferred approach that should be implemented across the board. Let’s look at their “definitive study:” Cohen, J. A., Deblinger, E., Mannarino, A. P., & Steer, R. A. (2004). A multisite, randomized controlled trial for children with sexual abuse-related PTSD symptoms. Journal of the American Academy of Child and Adolescent Psychiatry, 43(4), 393-402.

SSDD all the way! It is always the same when you scratch below the surface of superiority claims—they just don’t hold up to critical scrutiny. First let me say that there is nothing wrong with TF-CBT. It has good ideas and good possibilities, and is surely helpful for some kids and parents. I just wish they would present it that way; i.e., if you work with kids and families where abuse and trauma are involved, you might consider adding these ideas and interventions to your repertoire—they probably will make some sense to some of your clients. But, of course, that is not what they say and instead they claim superiority and folks get the crazy idea that it should be mandated or practiced exclusively.

As always, you gotta consider whether or not it is a fair contest or one in which the winner is pre-determined by the design (imagine the porpoise and the cow in a swimming contest), the pet approach of the researcher pitted against a less than equal opponent. Child Centered Treatment (CCT), the comparison treatment in this study, is not a fair comparison—it is a sham treatment. Therapists did not see the kids and parents together at all, whereas the TF-CBT therapists saw kids and parents together 3 times out of the 12 possible sessions. It just is not reasonable care of a kid who has been sexually abused without meeting with both the child and parent (or caring adult) together to make sense of what has happened. That’s one thing, and then there is the real kicker: Therapists in the CCT condition did not provide advice or suggestions to kids or parents. This is not a real treatment. In the face of such serious concerns, even the most died in the wool “client centered” therapist would address client requests for suggestions and guidance.

Given this mock therapy, one might also suspect that the therapists likely believed that the TF-CBT offered some advantages over CCT given there was at least some structure and ideas offered to these struggling families. Enter allegiance factors. Therapists served as their own controls (performed both TF-CBT and CCT) and were monitored for fidelity, or other words to ensure they didn’t offer guidance (beyond processing feelings and finding client solutions) in the CCT condition. It doesn’t say who provided the “intensive supervision” but that probably means it was the researchers.

So given that it was an unfair comparison of an active treatment model to one unlikely to ever happen in the real world, and given the therapists in the study could hardly help but like to offer some guidance to clients when asked and therefore likely were more committed to TF-CBT, the results are particularly underwhelming. First off, there was a main effect for both conditions. Both treatments worked, which is a real testament to client factors given the CCT didn’t provide any structure or practical intervention. There were 16 measures for the kids and 4 for the caregivers. 3 of the 16 were clinician rated measures (diagnostic interview by folks trained by the researchers). Of the 16, 8 found a significant advantage for TF-CBT. But 3 of those were the from the clinician’s point of view. Only 5 of 13 client rated measures found an advantage for TF-CBT. All 4 of the adult measures found an advantage for TF-CBT. An inspection of the results table reveals that many of the “significant” findings arise from pretty small differences in the means at post-treatment, challenging at least some of the clinical significance of the findings. Finally, it seems that the measures chosen were reactive, or selected to reflect the very things that TF-CBT directly address while the comparison treatment does not address these aspects at all.

In summary, as always you have to ask yourself when superiority is claimed, “as compared to what?” This is study does not provide compelling evidence that TF-CBT is superior to anything else but rather that TF-CBT has demonstrated that it is a viable way to approach children and families who have suffered the trauma of sexual abuse. Regarding superiority claims, the TF in TF-CBT means totally false!

A summary of the problems often found in such claims can be found at http://heartandsoulofchange.com/resources/handouts/

Thankfully, there is a sanctioned argument to help efforts to rescind mandates for particular approaches. In the face of growing criticism, 2005 APA President Ronald Levant appointed the Presidential Task Force on Evidence-Based Practice (hereafter Task Force). The Task Force defined evidenced based practice (EBP) as “the integration of the best available research with clinical expertise in the context of patient (sic) characteristics, culture, and preferences (Task Force 2006, p. 273). This definition transcends the “demonstrated efficacy in two RCTs” mentality of ESTs and finally makes common clinical sense.

The Task Force also said:
The application of research evidence to a given patient always involves probabilistic inferences. Therefore, ongoing monitoring of patient progress and adjustment of treatment as needed are essential (Task Force, 2006, p. 280).

Proponents from both sides of the common v. specific factors aisle recognized that outcome is not guaranteed regardless of evidentiary support of a given technique or the expertise of the therapist (Anker et al., 2009). Practice based evidence must become routine. The new definition supports an identity of plurality, essential attention to client preferences, a focus on therapist expertise, and the importance of feedback.

Bottom Line: There is nothing wrong with ESTs or evidence based practice. Challenge statements, however, that use evidence based practice to justify mandates, exclusive reimbursement, or dictates about “the” way to address client problems. Know about the dodo verdict and unfair contests in research. Educate others about APA’s definition and the importance of measuring the client’s response to any delivered treatment—advocate for practice based evidence as an evidence based practice.

Next Blog: The Recovery Revolution