Later, in the 1970’s, with the passing of freedom of choice legislation guaranteeing parity with psychiatrists, psychologists (and later others) learned to collect from third-party payers using only a psychiatric diagnosis for reimbursement. Thereafter, drowning any possibilities for other psychosocial systems of understanding human challenges, the National Institute of Mental Health (NIMH), the leading source of research funding for psychotherapy, decided to apply the same methodology used in drug research to evaluate psychotherapy (Goldfried & Wolfe, 1996)—the randomized clinical trial (RCT) requiring both diagnosis and manualized treatments. Diagnosis reached its pinnacle. Now both reimbursement and research funding depended on it. Funding for studies not related to specific treatments for specific disorders precipitously dropped as both research and psychotherapy itself became more and more medicalized, and dependent on diagnosis, manualization, and RCTs for credibility.

Diagnosis is the beginning point, the foundation of the both the medical model’s simple equation as well as the RCT. Unlike with medical treatments, diagnosis is an ill-advised starting point for psychotherapy. Diagnosis simply lacks reliability. In an interview, Robert Spitzer, the architect of the DSM III, admitted:
“To say that we’ve solved the reliability problem is just not true…It’s been improved. But if you’re in a situation with a general clinician it’s certainly not very good. There’s still a real problem, and it’s not clear how to solve the problem” (Spiegel, 2005, p. 63).

In addition to underwhelming reliability, psychiatric diagnosis lacks validity. Allen Frances, lead editor of the fourth edition of the DSM, recently confessed, “there is no definition of a mental disorder. It’s bullshit. I mean, you just can’t define it” (Greenberg, 2010, p. 1). Psychiatric diagnoses fail the most basic definition of validity—they lack empirical standards to distinguish the hypothesized pathological states from normal human variation or other disorders. Consequently, diagnosis always begs numerous, unanswered questions concerning cultural expectations and the role that power, privilege, gender, and race play in the identifying, cataloguing, and addressing client distress. The result is a set of murky over-inclusive criteria, often disadvantaging those who are racially or ethnically different, for an ever growing list of disorders (Duncan et al., 2004).

Finally and particularly germane to practitioners, diagnosis tells little about a person that is relevant to therapeutic change. Diagnosis in mental health is not correlated with outcome or length of stay (Brown et al., 1999; Wampold & Brown, 2005), and given the dodo verdict (see below) cannot provide reliable guidance to clinicians or clients regarding the best approach to resolving a problem. Diagnosis does not address what is most relevant to the helping process, namely the impact of the “disorder” in the client’s life and what can be done about it. Diagnosis also does not cover the range of reasons for which people seek therapy—relational, situational, and quality of life related, not symptom oriented. Nevertheless, the DSM, in spite of a long history of detailed critique (Carson, 1997; Duncan et al., 2004; Kirk & Kutchins, 1992), poor reliability and validity, and limited power to predict treatment outcome, lives on. It remains a fixed part of graduate training programs, a prominent feature of ESTs, and a prerequisite for funding in most mental health and substance abuse delivery systems—all engendering an illusion of scientific aura and clinical utility that far overreaches the DSM’s deeply flawed infrastructure.

Turning to the second part of the equation, that psychotherapists might possess the psychological equivalent of a “pill” for emotional distress resonates strongly with many, and is nothing if not seductive as it teases the desire to be helpful as possible to clients. A treatment for a specific “disorder,” from this perspective, is like a silver bullet, potent and transferable from research setting to clinical practice. Any therapist need only to load the silver bullet into any psychotherapy revolver and shoot the psychic werewolf stalking the client. Perhaps in its most unfortunate interpretation, clients are reduced to a diagnosis and therapists defined by a treatment technology—both interchangeable and insignificant to the procedure at hand

Consider the RCT. It was designed to compare the effects of a drug (an active compound) to a placebo (a therapeutically inert or inactive substance) for a specific illness. The basic assumption of the RCT is that the specific (unique) ingredients of different drugs (or psychotherapies) will produce different effects, superior over placebo, with different disorders. In effect, this assumption likens psychotherapy to a pill, with discernable unique ingredients that can be shown to have more potency than other active ingredients of other drugs.

There are three empirical arguments that cast doubt upon this assumption. First is the dodo bird verdict, which colorfully summarizes the robust finding that specific therapy approaches do not show specific effects or relative efficacy. In 1936, Saul Rosenzweig first invoked the dodo’s words from Alice’s Adventures in Wonderland, “Everybody has won and all must have prizes,” to illustrate his observation of the equivalent success of diverse psychotherapies. Almost 40 years later, Luborsky, Singer, and Luborsky (1975) empirically validated Rozenzweig’s conclusion in their now classic review of comparative clinical trials. The dodo bird verdict has since become the most replicated finding in the psychological literature, encompassing a broad array of research designs, problems, and clinical settings.

Three classic comparative clinical trials illustrate the dodo verdict. Ushering in the RCT in psychotherapy research was the Treatment of Depression Collaborative Research Program (TDCRP) (Elkin et al., 1989). The TDCRP randomly assigned 250 depressed participants to four different conditions: CBT, interpersonal therapy (IPT), antidepressants plus clinical management (IMI), and a pill placebo plus clinical management. The four conditions—including placebo—achieved about the same results, although both IPT and IMI surpassed placebo (but not the other treatments) on the recovery criterion. Project MATCH is the “largest and most statistically powerful clinical trial” in the history of alcohol and drug treatment (Project MATCH Research Group, 1997). Three widely divergent approaches were included: motivational enhancement therapy (MET), 12-Step facilitation (TSF), and CBT. The results revealed considerable improvement, but no differences in outcome emerged among the three approaches. Follow up ten years later (Tonigan et al, 2003) found no support for differential outcomes among the three therapies on percent days abstinent, drinks per drinking day, and total standard drink measures. In the Cannabis Youth Treatment (CYT) Study (Dennis et al., 2004), considered by many to be the largest and most methodologically sound investigation of adolescents to date, 600 adolescents were assigned either to treatment with MET plus CBT ( 5 or 12 sessions), family education and therapy, Adolescent Community Reinforcement Approach, or Multidimensional Family Therapy (MDFT). Comparisons between conditions found roughly equivalent significant pre-post treatment effects that were stable in terms of days of abstinence and percent in recovery by the end of the study.

Meta-analyses have yield similar results. A meta-analysis, designed specifically to test the dodo bird verdict (Wampold et al., 1997), included some 277 studies conducted from 1970 to 1995. This analysis verified that no approach has reliably demonstrated superiority over any other. At most, the effect size (ES) of treatment differences was a weak .2. “Why,” Wampold et al. ask, “[do] researchers persist in attempts to find treatment differences, when they know that these effects are small?” (p. 211).

The preponderance of the data, therefore, indicate a lack of specific effects and refute any claim of superiority when two or more bona fide treatments fully intended to be therapeutic are compared. If there are no specific technical operations that can be reliably shown to produce a specific effect, then prescriptive treatments in psychotherapy (i.e., mandating specific models and techniques for particular disorders) seems to make little sense.

The second argument shining a light on the specific ingredients assumption comes from component studies. Component studies, which dismantle approaches to tease out unique ingredients, have similarly found little evidence to support any specific effects of therapy. For example, a meta-analytic investigation of component studies (Ahn & Wampold, 2001) located 27 comparisons in the literature between 1970 and 1998 that tested an approach against that same approach without a specific component. The results revealed no differences. These studies have shown that it doesn’t matter what component you leave out—the approach still works as well as the treatment containing all of its parts.

A final empirical argument challenging the assumption comes from estimates regarding the impact of specific technique on outcome. After an extensive, but non-statistical analysis of decades of outcome research, Lambert (1986, 1992) suggests that model/technique factors account for about 15% of outcome variance. An even smaller role for specific technical operations of various psychotherapy approaches is proposed by Wampold (2001). His meta-analysis assigns only a 13% (derived from a .8 ES) contribution to the impact of therapy, both general and specific factors combined. Of that 13%, a mere 8% is portioned to the contribution of model effects. Of the total variance of change, only 1% can be assigned to specific technique. A consideration of Lambert’s and Wampold’s estimates of variance reveals that specific treatments do not account for 85% and 99%, respectively, of the variance of outcome. Other variables–the client, the therapist, and their relationship–account for far more of outcome variance. When taken in total–the equivalent results of comparative clinical trials and meta-analytic investigations, component studies, and analyses of the amount of variance attributed to specific effects –the evidence points in the same direction. There are no significant unique ingredients to therapy approaches and therefore little justification for basing psychotherapy on prescriptive or empirically supported treatments. Psychotherapy, therefore, has been shoehorned into the medical model.

But The Medical Model is not the Borg, nor am I Captain Picard fighting for the survival of therapists. Psychotherapy, however, is not a medical endeavor, it is a relational one. There is nothing wrong with the medical model. But it is not empirically supported nor an apt description of our work.

On another note, the last free webinar about my book, On Becoming a Better Therapist is coming up on January 21. Of course you can catch all the free webinars anytime here, but attending live allows you to ask that question you always wanted to ask or make a comment that occurred to you while you were reading the book. In any event, I hope you join me. Here is the info:

Dr. Barry Duncan – On Becoming a Better Therapist: Chapter Seven Discussion
On Becoming presents a five-step method of integrating outcome management with therapists’ long-term professional development. In this seventh of seven webinars corresponding to the seven chapters of the book, I present the fifth step to keep your development on the front burner, the Treasure Chest. I’ll also discuss the controversial issues of the day as they pertain to your identity as a therapist: managed care, evidence based practice, psychiatric drugs, and the medical model. We’ll begin with a 25 minute overview followed by your questions, comments, and reflections. My hope is that the book and these discussions will inspire you to rediscover purpose in your work and become a better therapist.
Friday, January 21, 2011, 6:00 to 7:30 PM
Reserve your Webinar seat now at: https://www2.gotomeeting.com/register/595664219

On the heels of the American Psychiatric Association’s development of practice guidelines in 1993, to ensure their continued viability in the market, psychologists rushed to offer magic bullets to counter psychiatry’s magic pills—to establish empirically supported treatments (EST). With all good intentions, the task force of Division 12 (Task Force on Promotion and Dissemination of Psychological Procedures, 1995) reviewed available research and catalogued treatments of choice for specific diagnoses based on their demonstrated efficacy in two RCTs. On one hand, the Division 12 Task Force effectively increased recognition of the efficacy of psychological intervention among the public, policymakers, and training programs; on the other hand, it simultaneously promulgated gross misinterpretations—that ESTs have proven superiority over other approaches, and therefore, should be mandated and exclusively reimbursed. Unfortunately, many now believe, to paraphrase Orwell, that some therapies are more equal than others.

The notion, however, that any approach is better than another is indefensible in light of the evidence covered extensively throughout The Heart and Soul of Change that support the outcome equivalence of the different models (the “dodo verdict”) as well as the relative influence of other factors than model and technique. I encourage you to dig a little deeper and bolster your ability to respectfully counter statements that suggest mandates for practice. Littell’s (2010) scathing commentary of ESTs in The Heart and Soul of Change is a good place to start. Littell provides a useful template for understanding the varied ways that findings can be distorted and evidence constructed from underwhelming results.

Like understanding anything else, there is a language involved here and it takes a bit of wading through tedious material. But it is worth it if you desire to counter mandates for specific approaches and promote the freedom for therapists to practice as they see fit according to client preferences and benefit. Our necessary pluralism, the theoretical breadth so important to resonating with clients and accentuating our development, is at stake, as well as our identity—ESTs suggest a therapist identity based on technical acumen in administering manualized, cookie cutter interventions (Duncan & Miller, 2006).

Efficacy over placebo, sham, or no treatment is not efficacy over other approaches, or what is called differential efficacy. In the minority of studies that claim superiority over treatment as usual (TAU) or another approach, you need only to ask one question of the investigation (see Duncan et al., 2004 and Sparks & Duncan, 2010 for a full discussion and examples): Is it a fair contest? Is the study a comparison of two valid approaches intended to be therapeutic administered in equal amounts by therapists who equally believe in what they are doing and who are equally supported to do it—are the therapists from the same pool with equal caseloads or is the experimental group specially selected, trained, and supervised by the researcher/founder of the approach, and have reduced caseloads?

I have never seen an advantage of any approach over another (or TAU) that wasn’t a lopsided contest that had its winner predetermined. Consider Trauma Focused (TF)-CBT, an approach to child sexual abuse that is getting a lot of press as the preferred approach that should be implemented across the board. Let’s look at their “definitive study:” Cohen, J. A., Deblinger, E., Mannarino, A. P., & Steer, R. A. (2004). A multisite, randomized controlled trial for children with sexual abuse-related PTSD symptoms. Journal of the American Academy of Child and Adolescent Psychiatry, 43(4), 393-402.

SSDD all the way! It is always the same when you scratch below the surface of superiority claims—they just don’t hold up to critical scrutiny. First let me say that there is nothing wrong with TF-CBT. It has good ideas and good possibilities, and is surely helpful for some kids and parents. I just wish they would present it that way; i.e., if you work with kids and families where abuse and trauma are involved, you might consider adding these ideas and interventions to your repertoire—they probably will make some sense to some of your clients. But, of course, that is not what they say and instead they claim superiority and folks get the crazy idea that it should be mandated or practiced exclusively.

As always, you gotta consider whether or not it is a fair contest or one in which the winner is pre-determined by the design (imagine the porpoise and the cow in a swimming contest), the pet approach of the researcher pitted against a less than equal opponent. Child Centered Treatment (CCT), the comparison treatment in this study, is not a fair comparison—it is a sham treatment. Therapists did not see the kids and parents together at all, whereas the TF-CBT therapists saw kids and parents together 3 times out of the 12 possible sessions. It just is not reasonable care of a kid who has been sexually abused without meeting with both the child and parent (or caring adult) together to make sense of what has happened. That’s one thing, and then there is the real kicker: Therapists in the CCT condition did not provide advice or suggestions to kids or parents. This is not a real treatment. In the face of such serious concerns, even the most died in the wool “client centered” therapist would address client requests for suggestions and guidance.

Given this mock therapy, one might also suspect that the therapists likely believed that the TF-CBT offered some advantages over CCT given there was at least some structure and ideas offered to these struggling families. Enter allegiance factors. Therapists served as their own controls (performed both TF-CBT and CCT) and were monitored for fidelity, or other words to ensure they didn’t offer guidance (beyond processing feelings and finding client solutions) in the CCT condition. It doesn’t say who provided the “intensive supervision” but that probably means it was the researchers.

So given that it was an unfair comparison of an active treatment model to one unlikely to ever happen in the real world, and given the therapists in the study could hardly help but like to offer some guidance to clients when asked and therefore likely were more committed to TF-CBT, the results are particularly underwhelming. First off, there was a main effect for both conditions. Both treatments worked, which is a real testament to client factors given the CCT didn’t provide any structure or practical intervention. There were 16 measures for the kids and 4 for the caregivers. 3 of the 16 were clinician rated measures (diagnostic interview by folks trained by the researchers). Of the 16, 8 found a significant advantage for TF-CBT. But 3 of those were the from the clinician’s point of view. Only 5 of 13 client rated measures found an advantage for TF-CBT. All 4 of the adult measures found an advantage for TF-CBT. An inspection of the results table reveals that many of the “significant” findings arise from pretty small differences in the means at post-treatment, challenging at least some of the clinical significance of the findings. Finally, it seems that the measures chosen were reactive, or selected to reflect the very things that TF-CBT directly address while the comparison treatment does not address these aspects at all.

In summary, as always you have to ask yourself when superiority is claimed, “as compared to what?” This is study does not provide compelling evidence that TF-CBT is superior to anything else but rather that TF-CBT has demonstrated that it is a viable way to approach children and families who have suffered the trauma of sexual abuse. Regarding superiority claims, the TF in TF-CBT means totally false!

A summary of the problems often found in such claims can be found at http://heartandsoulofchange.com/resources/handouts/

Thankfully, there is a sanctioned argument to help efforts to rescind mandates for particular approaches. In the face of growing criticism, 2005 APA President Ronald Levant appointed the Presidential Task Force on Evidence-Based Practice (hereafter Task Force). The Task Force defined evidenced based practice (EBP) as “the integration of the best available research with clinical expertise in the context of patient (sic) characteristics, culture, and preferences (Task Force 2006, p. 273). This definition transcends the “demonstrated efficacy in two RCTs” mentality of ESTs and finally makes common clinical sense.

The Task Force also said:
The application of research evidence to a given patient always involves probabilistic inferences. Therefore, ongoing monitoring of patient progress and adjustment of treatment as needed are essential (Task Force, 2006, p. 280).

Proponents from both sides of the common v. specific factors aisle recognized that outcome is not guaranteed regardless of evidentiary support of a given technique or the expertise of the therapist (Anker et al., 2009). Practice based evidence must become routine. The new definition supports an identity of plurality, essential attention to client preferences, a focus on therapist expertise, and the importance of feedback.

Bottom Line: There is nothing wrong with ESTs or evidence based practice. Challenge statements, however, that use evidence based practice to justify mandates, exclusive reimbursement, or dictates about “the” way to address client problems. Know about the dodo verdict and unfair contests in research. Educate others about APA’s definition and the importance of measuring the client’s response to any delivered treatment—advocate for practice based evidence as an evidence based practice.

Next Blog: The Recovery Revolution