What's Next for Evidence-Based Practice at SAMHSA?

NREPP SAMHSA statementOn January 11, Assistant Secretary Elinore McCance-Katz addressed the controversy over the terminated National Registry of Evidence-Based Programs and Practices. After several days of silence from SAMHSA, the McCance-Katz sought to explain why SAMHSA ended the registry which lawmakers and mental health providers alike had both lauded and criticized.
Her defense of the move was forceful and appealed to the nation’s opioid addiction crisis. In a conference call with reporters she said, “We in the Trump administration are not going to sit back and let people die.” Without giving any instances linked to NREPP, she added that people will die “if we leave things up on the website that don’t help people.”
In her statement, McCance-Katz lamented that the registry was inadequate. She especially regretted the lack of evidence-based treatments for severely mentally ill persons. She added, “If someone with limited knowledge about various mental and substance use disorders were to go to the NREPP website, they could come away thinking that there are virtually no EBPs for opioid use disorder and other major mental disorders – which is completely untrue.” In fact, I was able to find two such treatments but they were short term interventions.
McCance-Katz also faulted developers of evidence-based practices. She said:

We at SAMHSA should not be encouraging providers to use NREPP to obtain EBPs, given the flawed nature of this system. From my limited review – I have not looked at every listed program or practice – I see EBPs that are entirely irrelevant to some disorders, “evidence” based on review of as few as a single publication that might be quite old and, too often, evidence review from someone’s dissertation.
This is a poor approach to the determination of EBPs. As I mentioned, NREPP has mainly reviewed submissions from “developers” in the field. By definition, these are not EBPs because they are limited to the work of a single person or group. This is a biased, self-selected series of interventions further hampered by a poor search-term system. Americans living with these serious illnesses deserve better, and SAMHSA can now provide that necessary guidance to communities.

There is some evidence for her criticism. I found at least one program which had next to no peer reviewed evidence (“moral reconation therapy”). The journal listed as providing the bulk of the data was a newsletter published by the originators of the therapy.

Plenty of Blame to Go Around

However, one proponent of evidence-based practice said SAMHSA shouldn’t assign blame exclusively on developers and providers. Steven Hayes, Foundation Professor of Psychology at the University of Nevada and one of the developers of Acceptance and Commitment Therapy told me that SAMHSA set the standards for developers and should raise them if the agency wants better results.
Hayes told me, “If SAMHSA was unhappy with the standards, why didn’t they raise them?” Hayes said there are gaps in research knowledge, but wonders how SAMHSA is going to bridge that gap by terminating NREPP. Hayes asked, “If large bodies of such research exist, why didn’t researchers submit the data before? If it existed, why didn’t SAMHSA just take steps to ensure that it was submitted to NREPP?”
Hayes continued, “The reasons there are gaps in research knowledge is that gaps can only be filled by creativity, linked to adequate funding to test creative ideas. Of course SAMHSA can declare that providers should do this or do that based on what their experts say, but this will be a far more political process and it is certain to be both conventional and more dominated by experts who are beholden to those with strong financial interests in the current system.”
Hayes is concerned that drug companies may play an outsized role. He said, “We know who those voices are in the main. We are already at a place in which more than 60% of those with mental health needs receive medications only, and 10% or less receiving psychosocial methods only. Given the issues of side effects, long term effects, opponent processes, and cost effectiveness, the data simply do not support this huge imbalance.”
Since NREPP was a way for developers to get their programs to the public, Hayes said, “Cutting off one of the few avenues program developers had available to them to provide scientifically valid information to the public, will likely leave the public with access to fewer voices and less information rather than more.”

Timing of SAMHSA’s Move to the Policy Lab

One of my concerns about the termination of the NREPP has been the lack of anything to replace it. Chris Garrett, spokesman for SAMHSA told me that SAMHSA is sensitive to this concern and provided this statement:

SAMHSA is working as expeditiously as possible to stand up the Policy Lab in order to carry forward this important work. Expert staff from SAMHSA are being reassigned to the Policy Lab as it stands up. We expect basic operations of the Lab to follow within the next month. In the meantime, SAMHSA has a number of evidence-based approaches already available on our website in the form of Treatment Improvement Protocols and Technical Assistance Publications. We highly recommend their use. The TIPs series can be found here. The TAPs series can be found here. The SAMHSA store also has a number of toolkits that describe evidence-based practices, as well as provide instructions on how to implement them.

Garrett said users could search for “toolkits” in the SAMHSA store to access these resources.
 
 

Many Questions Unanswered About the Future of Evidence-Based Policy at SAMHSA (UPDATED)

UPDATE (1/12/18) – Last night, SAMHSA Asst. Secretary Elinore McCance-Katz released a statement about the termination of NREPP.  The full statement is at the link and the end of this post. She also held a brief conference call with reporters. Although I wasn’t on it, I communicated with two people on the call. On the call and in her written statement, Dr. McCance-Katz criticized the NREPP. On the call, she was quoted by a reporter on the call as saying, “We in the Trump administration are not going to sit back and let people die,” she said, which will happen “if we leave things up on the website that don’t help people.”
There still is no time table for the implementation of the Policy Lab or any new approach. The statement is light on specifics. When I ask about when and how this is going to happen, I have gotten no responses. I hope to have another post later today with reactions to the Asst. Secretary’s remarks and am holding out hope that SAMHSA might address the issues of timing and implementation.
………………………………..
(Original post begins here)
One thing is clear. The National Registry of Evidence-Based Programs & Practices contract has been terminated. SAMHSA’s statement about the registry first posted here on this blog is now on the NREPP website (see the statement in red below):
NREPP SAMHSA statement

What Now?

In one form or another, the question on the minds of many mental health researchers and advocates is “what now?” After I received the statement above on 1/8, I asked a SAMHSA spokesman when researchers would be able to submit programs or update new programs. There has been no answer. It appears that SAMHSA discontinued NREPP even though the agency is not prepared to “reconfigure its approach” to evidence-based practice. Thus far, something is being replaced by nothing.
Media accounts of NREPP’s demise reflect the reaction of the mental health community. The Week‘s headline this morning reads, “Trump officials froze a federal database of addiction and mental health treatments. Nobody’s sure why.
On Tuesday, Think Progress led with “Without warning, the government just ended a registry of mental illness and drug abuse programs.” The subtitle? “And didn’t bother to warn program participants ahead of time.”
Yesterday, the Boston Globe‘s health news service Stat began:

The Trump administration has abruptly halted work on a highly regarded program to help physicians, families, state and local government agencies, and others separate effective “evidence-based” treatments for substance abuse and behavioral health problems from worthless interventions.

Also out this morning, the Washington Post covers much of the same ground but provides no details from SAMHSA about when the new process will begin.
I say again, SAMHSA replaced something with nothing and did so in the middle of an addiction epidemic.

Rep. Grace Meng Wants to Know What’s Going On with NREPP

Rep. Grace Meng represents the Sixth District of New York and is a member of the House Appropriations Committee. On January 5, unaware of the impending controversy over NREPP, she wrote SAMHSA to praise NREPP and ask how she could help support the registry. She closed her letter by saying,

Again, thank you for the NREPP.  I feel that the registry and its website are crucial public health tools.  Respectfully, I wish to know how you will grow the NREPP and its website, how you intend to review more opioid abuse-specific programs, and how I may be of help to you in these endeavors.

When Rep. Meng learned about the NREPP’s termination, she followed up with another letter on January 8 requesting the answers to several questions. The first question was “Why, with specificity, was this contract terminated?” That was followed by nine additional questions:
Meng question NREPP
These are good questions. I would add, why did SAMHSA terminate NREPP when it appears that SAMHSA doesn’t have another evidence-based process ready to go?
 
For background, see my first post on the termination of NREPP.
…………..
Statement of Elinore F. McCance-Katz, MD, PhD, Assistant Secretary for Mental Health and Substance Use regarding the National Registry of Evidence Programs and Practices and SAMHSA’s new approach to implementation of evidence-based practices (EBPs)
Thursday, January 11, 2018
SAMHSA and HHS are committed to advancing the use of science, in the form of data and evidence-based policies, programs and practices, to improve the lives of Americans living with substance use disorders and mental illness and of their families.
People throughout the United States are dying every day from substance use disorders and from serious mental illnesses. The situation regarding opioid addiction and serious mental illness is urgent, and we must attend to the needs of the American people. SAMHSA remains committed to promoting effective treatment options for the people we serve, because we know people can recover when they receive appropriate services.
SAMHSA has used the National Registry of Evidence Programs and Practices (NREPP) since 1997. For the majority of its existence, NREPP vetted practices and programs submitted by outside developers – resulting in a skewed presentation of evidence-based interventions, which did not address the spectrum of needs of those living with serious mental illness and substance use disorders. These needs include screening, evaluation, diagnosis, treatment, psychotherapies, psychosocial supports and recovery services in the community.
The program as currently configured often produces few to no results, when such common search terms as “medication-assisted treatment” or illnesses such as ”schizophrenia” are entered. There is a complete lack of a linkage between all of the EBPs that are necessary to provide effective care and treatment to those living with mental and substance use disorders, as well. If someone with limited knowledge about various mental and substance use disorders were to go to the NREPP website, they could come away thinking that there are virtually no EBPs for opioid use disorder and other major mental disorders – which is completely untrue.
They would have to try to discern which of the listed practices might be useful, but could not rely on the grading for the listed interventions; neither would there be any way for them to know which interventions were more effective than others.
We at SAMHSA should not be encouraging providers to use NREPP to obtain EBPs, given the flawed nature of this system. From my limited review – I have not looked at every listed program or practice – I see EBPs that are entirely irrelevant to some disorders, “evidence” based on review of as few as a single publication that might be quite old and, too often, evidence review from someone’s dissertation.
This is a poor approach to the determination of EBPs. As I mentioned, NREPP has mainly reviewed submissions from “developers” in the field. By definition, these are not EBPs because they are limited to the work of a single person or group. This is a biased, self-selected series of interventions further hampered by a poor search-term system. Americans living with these serious illnesses deserve better, and SAMHSA can now provide that necessary guidance to communities.
We are now moving to EBP implementation efforts through targeted technical assistance and training that makes use of local and national experts and will that assist programs with actually implementing services that will be essential to getting Americans living with these disorders the care and treatment and recovery services that they need.
These services are designed to provide EBPs appropriate to the communities seeking assistance, and the services will cover the spectrum of individual and community needs including prevention interventions, treatment and community recovery services.
We must do this now. We must not waste time continuing a program that has had since 1997 to show its effectiveness.
But yet we know that the majority of behavioral health programs still do not use EBPs: one indicator being the lack of medication-assisted treatment, the accepted, life-saving standard of care for opioid use disorder, in specialty substance use disorder programs nationwide.
SAMHSA will use its technical assistance and training resources, its expert resources, the resources of our sister agencies at the Department of Health and Human Services, and national stakeholders who are consulted for EBPs to inform American communities and to get Americans living with these disorders the resources that they deserve.

Trump Administration Halts Contracted Work on NREPP and Shifts Evidence-Based Focus to SAMHSA's Policy Lab (UPDATED)

UPDATE (1/8/2018) – This morning, I received this statement from a SAMHSA spokesman:

Although the current NREPP contract has been discontinued, SAMHSA is very focused on the development and implementation of evidence-based programs in communities across the nation.  SAMHSA’s Policy Lab will lead the effort to reconfigure its approach to identifying and disseminating evidence-based practice and programs.

The Policy Lab is referred to on SAMHSA’s website and is led by Christopher Jones. The Policy Lab was created by the 21st Century Cures Act and is an evolution of SAMHSA’s Office of Policy, Planning and Innovation. Clearly, the issue with the change from NREPP isn’t with the term “evidence-based” since the above statement uses the term and the Cures Act requires evidence-based interventions. For the exact language of the Cures statute, scroll to the end of this post.
——————-
(original post starts here)
Yesterday (Jan 4, 2018), a contractor for the Substance Abuse and Mental Health Services Administration alerted program participants that funding for work onnrepp the National Registry of Evidence-Based Programs & Practices had been terminated “for the convenience of the government.” According to a source with the contractor, the work was not terminated due to any problems with their work but because the administration did not want to continue it. According to my source, this action follows a freeze in the work which had been in effect since September, 2017.
The NREPP is an effort to alert the public and professional community about evidence-based practices in mental health treatment and prevention. According to SAMHSA’s 2018 budget justification, NREPP helps meet the requirements of the 21st Century Cures Act which requires the government to provide accurate information about what works in the treatment of mental illness and drug/alcohol addiction. SAMHSA is responsible to post this information on an agency website. SAMHSA requested $2.8-million in FY 2018 for NREPP.
According to an email I obtained which was later posted on Twitter by someone else, Development Services Group alerted their constituents that their contract to manage the NREPP’s contents and website had been terminated on December 28, 2017.
The email stated:

It is with great regret that we write to inform you that on December 28, 2017, we received notification from SAMHSA that the NREPP contract is being terminated for the convenience of the government.
This cancellation means that we can no longer make any updates to your program profile. We thank you for the help and cooperation you gave so that we could complete your review.
We are deeply saddened by the government’s sudden decision to end the NREPP contract, under which we have been able to provide and strengthen science-based information about mental health and substance use treatment and prevention programs, both nationally and internationally.
All comments and concerns should be directed to [email protected]

According to DSG, it isn’t clear what will become of NREPP. The reason the process of evaluating programs was given to a contractor was because SAMHSA did not have a sufficient number of staff to do the job. The website may remain but at present no additional guidance has come from SAMHSA. According to DSG, all materials are being returned to the government and not sent to another contractor.
My calls and emails to SAMHSA have not been returned.
It isn’t clear how SAMHSA will meet the mandates of the Cures Act without a functioning evidence-based program. Another open question is why the program was halted in the middle of the fiscal year without cause (“for the convenience of the government”).
(Updates will be added to this posts through the day)
——————
The entire text of the 21st Century Cures Act is here. For the section relevant to the Policy Laboratory and the work on evidence based practices and programs, see below.

SEC. 7001. ENCOURAGING INNOVATION AND EVIDENCE-BASED PROGRAMS.
    Title V of the Public Health Service Act (42 U.S.C. 290aa et seq.)
is amended by inserting after section 501 (42 U.S.C. 290aa) the
following:
``SEC. 501A. <<NOTE: 42 USC 290aa-0.>>  NATIONAL MENTAL HEALTH AND
                          SUBSTANCE USE POLICY LABORATORY.
    ``(a) In General.--There shall be established within the
Administration a National Mental Health and Substance Use Policy
Laboratory (referred to in this section as the `Laboratory').
    ``(b) Responsibilities.--The Laboratory shall--
            ``(1) continue to carry out the authorities and activities
        that were in effect for the Office of Policy, Planning, and
        Innovation as such Office existed prior to the date of enactment
        of the Helping Families in Mental Health Crisis Reform Act of
        2016;
            ``(2) identify, coordinate, and facilitate the
        implementation of policy changes likely to have a significant
        effect on mental health, mental illness, recovery supports, and
        the prevention and treatment of substance use disorder services;
            ``(3) work with the Center for Behavioral Health Statistics
        and Quality to collect, as appropriate, information from
        grantees under programs operated by the Administration in order
        to evaluate and disseminate information on evidence-based
        practices, including culturally and linguistically appropriate
        services, as appropriate, and service delivery models;
            ``(4) provide leadership in identifying and coordinating
        policies and programs, including evidence-based programs,
        related to mental and substance use disorders;
            ``(5) periodically review programs and activities operated
        by the Administration relating to the diagnosis or prevention
        of, treatment for, and recovery from, mental and substance use
        disorders to--
                    ``(A) identify any such programs or activities that
                are duplicative;
                    ``(B) identify any such programs or activities that
                are not evidence-based, effective, or efficient; and
                    ``(C) formulate recommendations for coordinating,
                eliminating, or improving programs or activities
                identified
[[Page 130 STAT. 1221]]
                under subparagraph (A) or (B) and merging such programs
                or activities into other successful programs or
                activities; and
            ``(6) carry out other activities as deemed necessary to
        continue to encourage innovation and disseminate evidence-based
        programs and practices.
    ``(c) Evidence-Based Practices and Service Delivery Models.--
            ``(1) In general.--In carrying out subsection (b)(3), the
        Laboratory--
                    ``(A) may give preference to models that improve--
                          ``(i) the coordination between mental health
                      and physical health providers;
                          ``(ii) the coordination among such providers
                      and the justice and corrections system; and
                          ``(iii) the cost effectiveness, quality,
                      effectiveness, and efficiency of health care
                      services furnished to adults with a serious mental
                      illness, children with a serious emotional
                      disturbance, or individuals in a mental health
                      crisis; and
                    ``(B) may include clinical protocols and practices
                that address the needs of individuals with early serious
                mental illness.
            ``(2) Consultation.--In carrying out this section, the
        Laboratory shall consult with--
                    ``(A) the Chief Medical Officer appointed under
                section 501(g);
                    ``(B) representatives of the National Institute of
                Mental Health, the National Institute on Drug Abuse, and
                the National Institute on Alcohol Abuse and Alcoholism,
                on an ongoing basis;
                    ``(C) other appropriate Federal agencies;
                    ``(D) clinical and analytical experts with expertise
                in psychiatric medical care and clinical psychological
                care, health care management, education, corrections
                health care, and mental health court systems, as
                appropriate; and
                    ``(E) other individuals and agencies as determined
                appropriate by the Assistant Secretary.
    ``(d) Deadline for Beginning Implementation.--The Laboratory shall
begin implementation of this section not later than January 1, 2018.
    ``(e) Promoting Innovation.--
            ``(1) In general.--The Assistant Secretary, in coordination
        with the Laboratory, may award grants to States, local
        governments, Indian tribes or tribal organizations (as such
        terms are defined in section 4 of the Indian Self-Determination
        and Education Assistance Act), educational institutions, and
        nonprofit organizations to develop evidence-based interventions,
        including culturally and linguistically appropriate services, as
        appropriate, for--
                    ``(A) evaluating a model that has been
                scientifically demonstrated to show promise, but would
                benefit from further applied development, for--
[[Page 130 STAT. 1222]]
                          ``(i) enhancing the prevention, diagnosis,
                      intervention, and treatment of, and recovery from,
                      mental illness, serious emotional disturbances,
                      substance use disorders, and co-occurring illness
                      or disorders; or
                          ``(ii) integrating or coordinating physical
                      health services and mental and substance use
                      disorders services; and
                    ``(B) expanding, replicating, or scaling evidence-
                based programs across a wider area to enhance effective
                screening, early diagnosis, intervention, and treatment
                with respect to mental illness, serious mental illness,
                serious emotional disturbances, and substance use
                disorders, primarily by--
                          ``(i) applying such evidence-based programs to
                      the delivery of care, including by training staff
                      in effective evidence-based treatments; or
                          ``(ii) integrating such evidence-based
                      programs into models of care across specialties
                      and jurisdictions.