2. Why, given all the time, money, and other resources dedicated to assessing outcomes in behavioral health, is there not a widespread adoption and implementation of outcome measurement?
There are a few reasons. First it hasn’t been a part of the vernacular of our field at the clinician level. For anyone in the field for a few years, it is a totally foreign concept and most folks assume the best regarding effectiveness. It is critical that graduate education step up to plate here and groom a new generation of mental health/substance abuse professionals that are savvy about outcome informed practice. Second, the field has not provided clinicians with any feasible, clinician friendly way to manage outcomes until recently. Some have been turned off by cumbersome and lengthy measures designed by researchers that don’t appear to be related to the day-to-day work of the front line therapist. Finally, many are afraid because of all the talk about P4P and other ideas that suggest that some will be punished who do not measure up to some arbitrary standard. The whole process of outcome measurement and management need to makes sense to front line therapists and appeal to their nearly universal desire to do good work and get better over the course of their careers. That is why the implementation process of the Project emphasizes both a top down and bottom process. It includes attention to things that makes sense to therapists: 1) common factors; 2) a nuanced clinical process; 3) and therapist development.

The common factors, those elements of psychotherapy running across all models that account for change (Duncan, 2010; Duncan et al., 2010), provide an overarching framework for the PCOMS intervention. Integrating the use of PCOMS within the larger literature about what works in therapy promotes therapist understanding of the feedback process and adherence to the feedback protocol. PCOMS is presented as the tie that binds these healing components together, allowing the factors to be expressed one client at a time. Soliciting systematic feedback is a living, ongoing process that engages clients in the collaborative monitoring of outcome, heightens hope for improvement, fits client preferences, maximizes chances for a strong alliance, and is itself a core feature of therapeutic change (Duncan, 2010).

Although the over 300,000 administrations of the measures has yielded invaluable information regarding the psychometrics of the measures, trajectories, algorithms, etc., PCOMS remains a clinical intervention embedded in the complex interpersonal process called psychotherapy. For successful implementation and ongoing adherence, PCOMS must appeal to therapists at a clinical level in ways that the numbers or data or even the research never can. Consequently, PCOMS is described as the clinical process that it is—one that requires skill and nuance to achieve the maximum feedback effect. PCOMS speaks to therapists “where they live” by providing a methodology to address those clients who do not benefit from their services.

Similarly, a focus on therapist development provides a positive motivation for therapists to invest time and energy in PCOMS. There will always be organizational motivations for PCOMS in terms of improved outcomes and reduced costs–the language of “return on investment” and “proof of value.” But there is also the personal motivation of the therapist, the very reason most got into this business in the first place: to make a difference in the lives of those served. The groundbreaking research by Orlinsky and Rønnestad (2005) about therapist development (now over 11,000 therapists included) demonstrates that nearly all therapists want to continue to improve throughout their careers and harnessing this motivation is part and parcel to successful implementation. PCOMS appeals to the best of therapist intentions and encourages therapists to collect ORS data so that they can track their development and implement strategies to improve their effectiveness (Duncan, 2010).

Including these larger themes allows therapists to see that the intentions of PCOMS go well beyond management or funder’s cost or efficiency objectives—client based outcome feedback is about client privilege and benefit, and helping therapists get better at what they do. In addition, it is also critical that therapists know that management only intends to use data to improve the quality of care that clients receive, that there will be no punitive use of the data in any way, shape, or form. Given that most therapists improve their outcomes with feedback ( 9 of 10 therapists improved in the Anker et al. trial), a positive, non-competitive approach goes a long way to assuage therapists’ fears.

3. We go to the doctor and expect that our blood pressure will be taken, we will be weighed, and our heart rate monitored. But when we go see a behavioral health professional there is no such standard measures. Is this patient preference? Clinician Preference? Both?
It is definitely not client preference. Consumers want to be involved in their own care. However, they don’t want to do meaningless paperwork that takes away from their time with the therapist. Consumer involvement in all decisions that affect care is the foundation of the PCOMS intervention, including persons not of the dominant culture as well as the traditionally disenfranchised. We have found that when people understand the purpose of the measures (keeping their voice central and making sure they are getting what they want), refusal rates are about one in a hundred. This is far more of an issue for therapists as discussed above.

4. Where do you see the field going in the near term?
Given that there are now nine RCTs supporting it, the time for client-based outcome feedback seems to have arrived (Lambert, 2010). I think that within 5 years, it will be standard practice. My optimism comes from several recent events. For example, the American Psychological Association (APA) Presidential Task Force (hereafter Task Force) on Evidence-Based Practice in Psychology (EBPP) defined EBPP as “the integration of the best available research with clinical expertise in the context of patient (sic) characteristics, culture, and preferences” (Task Force 2006, p. 273). Two parts of this definition draw attention to client feedback and to tailoring services to the individual client. First, regarding clinical expertise, the Task Force submitted:
Clinical expertise also entails the monitoring of patient progress… If progress is not proceeding adequately, the psychologist alters or addresses problematic aspects of the treatment (e.g., problems in the therapeutic relationship or in the implementation of the goals of the treatment) as appropriate. (APA, 2006, p. 276-277)

And second, “in the context of patient characteristics, culture, and preferences,” emphasizes what the client brings to the therapeutic stage as well as the acceptability of any intervention to the client’s expectations. The Task Force said:
The application of research evidence to a given patient always involves probabilistic inferences. Therefore, ongoing monitoring of patient progress and adjustment of treatment as needed are essential (Task Force, 2006, p. 280).
Outcome, in other words, is not guaranteed regardless of evidentiary support of a given technique or the expertise of the therapist. Client-based outcome feedback must become routine.

Further support comes from APA’s Division 29 Task Force on Empirically Supported Relationships who advised practitioners “…to routinely monitor patients’ responses to the therapy relationship and ongoing treatment. Such monitoring leads to increased opportunities to repair alliance ruptures, to improve the relationship, and to avoid premature termination” (Ackerman et al., 2001, p. 496). Finally, two other recent endorsements of outcome management by APA have emerged. First the APA Commission on Accreditation (2011) states that students and interns: “Be provided with supervised experience in collecting quantitative outcome data on the psychological services they provide…”(2011, C-24). And second, APA recently created a new outcome measurement database to encourage practitioners to select outcome measures for practice ((http://practiceoutcomes.apa.org).

So change is on the horizon.

Join the CDOI community
Join the Heroic Agencies List
Join the Mailing List
Become a CDOI Member
List Yourself as a CDOI Provider

We have three RCTs in progress: one with returning veterans, one with kids in the schools, and one study seeking to ferret out what really causes the feedback effect, or what is called a component study. Stay tuned.

Next in print is the “Footprints” article to appear next month the in Journal of Family Psychotherapy. This article looked at 6 month follow up data from the Norway Feedback Trial. As just a teaser of a study that reaffirms the importance of the alliance plus throws in a few curves, we found that clients in the non-feedback group were significantly more likely to complain about the therapy service delivery than feedback clients. More on this next month.

And a soon to be published (in the 2nd edition of the John Norcross book, Psychotherapy Relationships that Work) meta-analysis of PCOMS studies conducted by feedback pioneer Michael Lambert and K. Shimokawa found that those in feedback group had 3.5 higher odds of experiencing reliable change and less than half the odds of experiencing deterioration.

Finally, check out the next webinar by Dr. Mary Haynes: Creative Applications: CDOI in Case Managment
This workshop explores the ground-breaking expansion of the use of feedback to case management services. Based on her eight years of experience in extending the use of outcome management to settings other than traditional therapy, Mary will address the unique benefits and challenges of incorporating client feedback in community-based work with adults.

Join the member site now!

Where did this great idea of feedback come from? Howard, Moras, Brill, Matinovich, and Lutz (1996) were the first to advocate for the systematic evaluation of client response to treatment during the course of therapy. When this occurs—when client feedback is systematically collected and used to tailor treatment—good things happen.

For example, using the Outcome Questionnaire 45.2, Michael Lambert really brought this great idea to fruition. He has conducted five RCTs and all five demonstrated significant gains for feedback groups over treatment as usual (TAU) for clients at-risk for a negative outcome. Twenty two percent of TAU at-risk cases reached reliable improvement and clinically significant change compared with 33% for feedback to therapist groups, 39% for feedback to therapists and clients, and 45% when feedback was supplemented with support tools such as measures of the alliance. The addition of client feedback alone, without new techniques or models of treatment and leaving therapists to practice as they saw fit, enabled over two times the amount of at- risk clients to benefit from psychotherapy. Think of the advantage this brings to clinical practice. Systematic feedback allows good outcomes with many of those clients who would otherwise not benefit. 

I am very happy to announce that Michael Lambert, the person most responsible for bringing the power of client feedback to the forefront, will be conducting the next webinar to set the stage for his Heart and Soul of Change conference presentations:

“Yes, It Is Time for Clinicians to Track Outcomes”

Wednesday, January 27, noon to 1:00 Central

Join the person most responsible for the greatest innovation in clinical effectiveness since the beginning of psychotherapy. Register now by joining the CDOI membersite, now over a $400 value for a one year $120 subscription at http://www.cdoimembers.com/

Michael Lambert also inspired our client feedback process, The Partners for Change Outcome Management System’s (PCOMS). PCOMS appeal rests on the brevity of the measures and therefore its feasibility for everyday use in the demanding schedules of front-line clinicians. PCOMS was based on Lambert’s continuous assessment model using the Outcome Questionnaire 45.2, but there are differences beyond the measures. First, PCOMS is integrated into the ongoing psychotherapy process and routinely includes a transparent discussion of the feedback with the client (The Heroic Client). Session by session interaction is focused by client feedback about the benefits or lack thereof of psychotherapy. Second, PCOMS assesses the therapeutic alliance every session and includes a discussion of any potential problems. Lambert’s system includes alliance assessment only when there is a lack of progress. 

Three studies have demonstrated the benefits of client feedback with the ORS and SRS. Miller, Duncan, Brown, Sorrell, and Chalk (2006) explored the impact of feedback in a large culturally diverse sample utilizing a telephonic employee assistance program (EAP). Although the study’s quasi-experimental design qualifies the results, the use of outcome feedback doubled overall effectiveness and significantly increased retention. Two recent RCTs used PCOMS to investigate the effects of feedback versus TAU. First, in an independent investigation, Reese, Norsworthy, & Rowlands (2009) found that clients who attended therapy at a university counseling center or a graduate training clinic demonstrated significant treatment gains for feedback when compared to TAU. Finally, our recent study in Norway (Anker, Duncan, & Sparks, 2009), the largest RCT of couple therapy ever done, found that feedback clients reached clinically significant change nearly four times more than non-feedback couples. The feedback condition maintained its advantage at 6 month follow-up and achieved nearly a 50% lower separation/divorce rate.

A fourth study, a replication of the Norway Feedback Study by Jeff Reese has been submitted and a fifth study addressing feedback in an acute inpatient unit is about to get underway.

Read more on the resources page at http://heartandsoulofchange.com/resources/