Cultural Competency Methodological and Data Strategies to Assess the Quality of Services in Mental Health Systems of Care
Updated June 2002

Principal Co-Investigators: Carole Siegel, Ph.D., Gary Haugland, M.A., Ethel Davis Chambers, R.N., M.S. Study Team Co-Investigators: Carmen Aponte, Ph.D., Ralph Blackshear, Jr., Tony Ng, M.D., Charles Swift, Jr., MSW

•  SAMSHA/CMHS, Division of Special Programs Development and Special Populations contracted with the NYS OMH and the Center to develop and benchmark performance measures of the cultural competency of behavioral healthcare organizations. 
  

• In Phase I a set of performance measures was selected, with input from a broad multi-cultural and multi-disciplinary base of researchers, administrators, advocates, practitioners, consumers and other mental health stakeholders.  
  

• In Phase II, a smaller set of performance measures was selected, benchmarked, and related to a stage in an implementation process of cultural competency.
  
  RESEARCH ACTIVITIES AND RESULTS

Methods: An implementation logic model was developed related to the stages of implementation of cultural competency procedures within a mental health care organization.  Measures were identified related to each of these stages for three organizational levels: administrative, provider and individual.  An expert panel of multicultural stakeholders reduced the set to a feasible list and made an assessment of the earliest "stage" in the implementation process in which a measure could be applied . Twenty  “good practice” sites were identified by the Expert Panel, and a telephone survey was used to query key informants in these sites on the valuation of these measures in their organizations. The Panel used the survey responses to set benchmarks, but set them  at the highest levels that they deemed could be achieved by an organization, rather than on the norms of the survey data, so that the benchmarks would serve to signal the directions in which organizations must move in order to achieve higher levels of CC.  Benchmarks were set for the administrative and service entities levels.  

            Results: The rating process resulted in 41 administrative level performance measures: 18 are early stage measures (stage 1), 13 are middle stage measures (stage 2) and 10 are later stage measures (stage 3).  Forty-four service entity level performance measures were selected: 19 are stage 1, 22 are stage 2 and 3 are stage 3.   No decisions were made about the stages of the 18 individual level performance measures, but it was recognized that CC had to be in place at the organizational levels before individual caregivers and support staff could be evaluated for their cultural competence. Problematic issues related to financial implications were identified around definitions of “cultural groups”, threshold levels, and service coverage areas. The selected measures were re-reviewed to insure that they could be used to guide an implementation process and to support the integration of CC throughout the organizational levels of an organization.  The selected list speaks to the structures that should exist, the processes that should take place and the outcomes that should be achieved to indicate that CC is in place. 

POLICY IMPLICATIONS   

            Use of these measures is expected to move and promote organizations and their personnel toward culturally competent behaviors in the delivery of mental health services.  The CC performance measures developed in this project are fully compatible with prior work of other groups and in particular with the CLAS standards.    

            
PLANS

            The report will be finalized based on review by SAMHSA and national reactors.  A paper will be written for dissemination and study team members will participate in dissemination activities such as presentations and panel participation.  With regard to the ability of an organization to utilize the performance measures, the Study Team recommended that the following tasks be carried out: 

• 1) Pilot tests of the understandability of the CC measures;  
• 2) Implementation analysis in which both the process of implementation, the modes of data collection and the costs to conduct performance measurement are examined; and 
• 3) Establishment of federal and state guidelines and/or mandates for conducting CC evaluations. 
  
  We are currently working with Robert Drake, M.D. of Dartmouth on the possibility of interfacing these measures with material in the Evidence-Based Practice (EBP) Toolkits to ensure that cultural competency is taken into account when disseminating and adapting EBPs. 

FINAL REPORT

Siegel C, Haugland G, Chambers ED (2002). Cultural Competency Methodological and Data Strategies to Assess the Quality of Services in Mental Health Systems of Care
A Project to Select and Benchmark Performance Measures of Cultural Competency
Orangeburg NY: CSIPMH; NKI; NYSOMH (89 pp.) 

Click Here to Download Report (in PDF format)

Updated: Aug. 12, 2002