Supporting Statement for National Standards for Culturally and Linguistically Appropriate Services (CLAS) in Health and Health Care: Evaluation of Awareness, Adoption, and Implementation
Date: June 8, 2015
OMB Cont.: 201412-0990-002 0990
Title: National Standards for Culturally and Linguistically Appropriate Services (CLAS) in Health and Health Care: Evaluation of Awareness, Adoption, and Implementation
Program: HHS/OS/OMH
Contact: Rashida Dorsey, Director—OMH Division of Policy and Data, 240-453-2882
A. Justification
Circumstances Making the Collection of Information Necessary
The Office of Minority Health (OMH), Office of the Secretary (OS), Department of Health and Human Services (HHS) is requesting approval from OMB to collect information for an evaluation of the National Standards for Culturally and Linguistically Appropriate Services in Health and Health Care (the National CLAS Standards) within non-federal health and health care organizations in the United States. While much progress has been achieved in the provision of culturally and linguistically appropriate services, little is known about the process(es) through which health and health care organizations utilize the National CLAS Standards to implement these services. Despite their increased recognition as a fundamental tool for health and health care organizations to use in their efforts to become more culturally and linguistically competent, neither the original (released in 2001) nor the enhanced National CLAS Standards (released in 2013) have been systematically evaluated in terms of public awareness, organizational adoption and implementation, or impact upon health services outcomes. Given this, it is difficult for those who wish to support and use the National CLAS Standards in their efforts to become more culturally and linguistically appropriate, because of limited understanding of how the Standards are supposed to be implemented to achieve desired outcomes. It is also difficult to determine whether the enhanced National CLAS Standards actually achieve their expected outcomes (i.e., advancing health equity, improving quality, and reducing health care disparities). The goal of this evaluation is to generate detailed descriptions of the implementation of the National CLAS Standards, and to develop a foundational knowledge base required to formulate and explore additional questions.
This assessment will systematically examine and document the awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards among non-federal health and health care organizations. The information collected through this evaluation is necessary to assess how the Standards are currently being used by health and health care organizations, and is especially paramount given the lack of any such comprehensive study since the Standards’ original launch in 2001. Specifically, this evaluation is an attempt to describe the complex processes of awareness, knowledge acquisition, adoption, and implementation in health and health care organizations, using a small number of information-rich health and health care organizations as cases to identify: how the National CLAS Standards are implemented, the barriers and facilitators of organizational implementation of the National CLAS Standards, and intended outcomes of implementation.
In addition to the valuable knowledge gained from this evaluation, the results will inform the development of descriptive technical assistance materials, and preliminary support for the identification of promising practices, related to the implementation of the National CLAS Standards. The technical assistance materials, in particular, will serve as practical tools to guide health and health care organizations in their efforts to provide quality care to the culturally and linguistically diverse populations they serve through implementing the National CLAS Standards.
This evaluation supports the Office of Minority Health within the Office of the Secretary of the Department of Health and Human Services (HHS/OS/OMH), and is consistent with activities described in the HHS Action Plan to Reduce Racial and Ethnic Health Disparities, and the National Stakeholder Strategy for Achieving Health Equity.
Purpose and Use of Information Collection
The purpose of this evaluation is to systematically examine and document the process of awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards. Information will be collected directly from assessment participants through either a telephone or in-person interview, an online survey, or a web-based form. The information is needed from participants, who are knowledgeable of and/or currently utilize, the Standards, to provide a rich description of the complex processes of awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards in non-federal health and health care organizations.
Thirteen overarching evaluation questions are guiding this evaluation, each related to an aspect of awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards (See Appendix A). The information to be collected in this assessment will be collected from five categories of respondents. The respondents include: 1) National CLAS Standards stakeholders (individuals directly involved in the development of the original and/or enhanced National CLAS Standards), 2) CLAS Stakeholders (recognized experts on culturally and linguistically appropriate services (CLAS) and/or the National CLAS Standards), 3) leadership and senior executive members (e.g., CEOs, COOs, Director of Human Resources, designees, etc.) of up to 20 selected health and health care organizations that self-identify as a “CLAS adopter” and meet other inclusion criteria as outlined in Section B , 4) key staff in health and health care organizations that self-identify as a “CLAS adopter” and 5) Think Cultural Health registrants (users of the Think Cultural Health (TCH) website with knowledge and/or experience in using the National CLAS Standards in a health or health care setting). Individual-level information collected includes demographic information such as sex, race, ethnicity, current position/work title, and years in their position, and questions examining the respondent’s actual and perceived awareness, knowledge, adoption, and implementation of the National CLAS Standards among health and health care organizations. Organizational-level information collected will include descriptive information such as the type of health or health care organization, location of the organization, size of the organization, and information about the organization’s adoption and implementation of the National CLAS Standards. Please see the appendices for the evaluation questions, the full data collection instruments for each category, and a crosswalk of the evaluation questions, data collection instruments, and data analyses.
The information generated from this evaluation will have multiple applications. It will be used by OMH to develop and disseminate technical assistance on implementation of the National CLAS Standards for health and health care organizations. The information collected from this project will also be used to identify strengths and areas of improvement for policy surrounding the Standards themselves, and will inform the future promotion and dissemination of the Standards to health and health care organizations. Information from this evaluation will also be used to contribute to the body of literature in the field of culturally and linguistically appropriate services and about the National CLAS Standards, of which there is a paucity.
Use of Improved Information Technology and Burden Reduction
To obtain the information necessary for this assessment to meet its goals and objectives, while simultaneously minimizing the burden on participants, multiple data collection approaches will be used, consistent with a concurrent (convergent) mixed-methods design. Given the exploratory and process-oriented nature of the assessment and evaluation, key informant interviews lasting no more than 60 minutes will be conducted via telephone or in-person with National CLAS Standards and CLAS Stakeholders, and key informant interviews lasting no more than 90 minutes will be conducted via telephone or in person with Health and Health Care Leadership in up to 20 selected information-rich health and health care organizations. Collecting information in this time span will enable us to gather the rich, descriptive, and accurate type of information needed to describe appropriately the complex process of adoption and implementation of the National CLAS Standards. In addition, a survey, designed to take no more than 30 minutes to complete, will be administered to key staff within up to 20 selected health and health care organizations that identify themselves as “CLAS adopters” via a web-based survey platform. The online survey method was selected to minimize burden upon respondents. Finally, a web-based form, designed to take no more than 10 minutes to complete, will be available on the Think Cultural Health website soliciting voluntary feedback on National CLAS Standards’ implementation from registered users of the site.
Efforts to Identify Duplication and Use of Similar Information
A review of the literature currently shows little research that comprehensively examines the public awareness, organizational adoption, and implementation of the National CLAS Standards among health and health care organizations in the United States. Though there are some examples of studies describing one facet of these processes (e.g., implementation), they are limited in their scope of information provided and their generalizability. In 2004, for example, ORC Macro conducted a study on the implementation of the National CLAS Standards at the Alameda Alliance for Health, a managed care organization in Alameda, California. However, because this assessment and evaluation used a single-case study approach, it limited the generalizability and applicability of results to the diversity of types of health and health care organizations across the United States. The proposed assessment project attempts to expand both the number of types of health and health care organizations examined, and the processes related to implementation of the National CLAS Standards that are explored. Therefore, the information collected through this assessment and evaluation is not duplicative of other information collection.
Currently, there is no existing repository of this information that can be leveraged to substantially inform or conduct the assessment and evaluation.
Impact on Small Businesses or Other Small Entities
No small businesses will be involved in this study.
Consequences of Collecting the Information Less Frequent Collection
Information will be collected only once from all respondents; no data will be collected from a respondent periodically. Depending upon respondent category, each individual who agrees to participate will complete one telephone or in-person interview, a web-based survey, or a web-based form. If this data is not collected, OMH will have difficulty understanding and demonstrating how and to what extent the National CLAS Standards have been implemented by health and health care organizations. Furthermore, it will not be possible to answer pressing questions about whether the enhanced National CLAS Standards achieve their expected outcomes (i.e., advancing health equity, improving quality, and reducing health care disparities) without first understanding current organizational awareness, knowledge acquisition, adoption, and implementation of the Standards. There are no increases of burden for respondents. There are no legal obstacles to reduce the burden.
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
Given the variety of health and health care organization typologies, and the diversity of organizations within each typology (e.g., geographic location, patient demographics, payor mix, etc.), the results of this evaluation are not intended to be generalized across the entire universe of health and health care organizations within the United States. The aim of this evaluation is to examine and document the awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards among a select number (up to 20) of information-rich non-federal health and health care organizations to describe and inform the development of technical assistance materials related to the implementation of the National CLAS Standards, both of which can be used by health and health care organizations. Beyond this aspect, there are no special circumstances that would result in data collection outside of this process.
Comments in Response to the Federal Register Notice/Outside Consultation
The following sentence will suffice: A 60-day Federal Register Notice was published in the Federal Register on September 26, 2014, vol. 79, No. 187; pp. 57930-31 (see attachment K). There were no public comments.
The information collection instruments were developed in 2014 in consultation with OMH and a 5-person Advisory Group consisting of content and evaluation experts. The Advisory Group members included:
Jessica Briefer French
Senior Research Scientist, National Committee for Quality Assurance
202-955-1776
Iris
Garcia-Caban, Ph.D.
Executive Office Health Human Services
Medicaid
(MassHealth) Office of Providers & Plans
617-847-6528
Robert Like, M.D.
Professor and Director,
Center for Healthy Families and Cultural Diversity
Department of Family Medicine and
Community Health
Rutgers Robert Wood Johnson Medical School
732-235-7662
Francis Lu, M.D.
Luke & Grace Kim Professor in Cultural Psychiatry, Emeritus
University of California, Davis
916-708-5542
Matthew Wynia, M.D.
Director, Center for Bioethics and Humanities
University of Colorado Health Sciences Center
312-464-7980
In summary, Advisory Group feedback included clarifications of interview prompts, explicitly differentiating between the two versions of the National CLAS Standards (2001 and 2013), adding and revising probes and wording, clarifying which leadership will be targeted for health and health care organization interviews, and considering global questions versus nuanced, standard-specific questions in light of interview length and burden.
The instruments were further refined based on the results of a pilot study.
Explanation of any Payment/Gift to Respondents
No payments or gifts are provided to respondents.
Health and health care organizations that participate in the assessment and evaluation will receive an aggregate summary of the anonymous responses from key staff who complete the survey, which may be useful to the organization as it plans and monitors activities related to the adoption and implementation of the National CLAS Standards.
Assurance of Confidentiality Provided to Respondents
There is no assurance of confidentiality provided to respondents; however, data will be kept private to the extent allowed by the law. This project has been reviewed by the George Mason University Institutional Review Board, and has been considered exempt per category #2 (“Some research involving educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures, or observation of public behavior.”)
Justification for Sensitive Questions
Per HHS data collection requirements, the sex, ethnicity, and race of respondents are requested.
Estimates of Annualized Hour and Cost Burden
Data will be collected only once from each participant. Based on estimates from conducting a pilot test with a sample of nine individuals total (three in each category) of the interview protocols and measures to be used in this evaluation, it is expected that it will take no more than 1 hour for respondents to complete the interviews, no more than 15 minutes to complete the online staff survey, and no more than 10 minutes to complete the National CLAS Standards Experience Form. On average, it took approximately 45 minutes (0.75 hours) to complete the National CLAS Standards Stakeholder Interview; 1 hour to complete the CLAS Stakeholder Interview; and 1 hour to complete the Health and Health Care Organization Leadership Interview. On average, the Health and Health Care Organization Staff Survey took 15 minutes (0.25 hours) to complete. On average, the National CLAS Standards Experience Form took approximately 10 minutes (0.17 hours) to complete. The Health and Health Care Organization Screener survey is estimated to take approximately 5 minutes (0.08 hours) to complete. The maximum respondents for the National CLAS Standards Stakeholder and CLAS Stakeholder interviews will be 30% of the total number of individuals within each Stakeholder group. The maximum respondents for the Health and Health Care Organization Leadership Interview will be seven (7) leadership from each of the 20 participant organizations (five organizations from each of the four typologies), generating a total of 140 respondents. The respondents for the Health and Health Care Organization Staff Survey is an estimated total staff size of 125 from each of the 20 participant organizations (five organizations from each of the four typologies), generating a total of 2,500 total respondents. The respondents for the screener survey are an estimated total of 50,000. The maximum respondents for the National CLAS Standards Experience Form are approximately 240,000 (total number of newsletter registrants). Data will be collected only once from each participant.
12A. Estimated Annualized Burden Hours
Form Name |
Type of Respondent |
No. of Respondent |
No. Responses per Respondent |
Average Burden per Response (hours) |
Total Burden (hours) |
National CLAS Standards Stakeholder Interview |
Training and Development Specialists and Managers; Other Management Occupations |
21
|
1 |
45/60 |
16 |
CLAS Stakeholder Interview |
Training and Development Specialists and Managers; Other Management Occupations |
21 |
1 |
1 |
21 |
Health and Healthcare Organization Leadership Interview |
Health and Health Care Organization Executives and Managers |
140
|
1
|
1
|
140 |
Health and Health Care Organization Staff Survey |
Health and Health Care Providers, Managers, and Support Staff |
2,500 |
1 |
15/60 |
625
|
Health and Healthcare Organization Screener Survey |
Health and Health Care Organization Executives
|
50,000 |
1 |
5/60 |
4,167
|
National CLAS Standards Experience Form |
Health Care Practitioners and Technical Occupations |
240,000 |
1 |
10/60 |
40,000 |
TOTAL |
|
|
|
|
44,969 |
12B.
This is not an annual collection. This is a one-time collection used to inform an assessment and evaluation. Though there are no annual burden costs to participants, a one-time cost can be calculated.
The assessment does not request salary information of participants as part of the data collection; therefore, figures were based on statistics provided by the Department of Labor, Bureau of Labor Statistics.
Using this resource, it was determined that the national average hourly wage for the types of participants in the National CLAS Standards Key Stakeholder interview is $41.78. The average participant is expected to complete the interview in 45 minutes (0.75 hours), thus making the cost of burden $41.78 x 0.75 hours = $31.34; this is a one-time total respondent cost burden per interview.
It was determined that the national average hourly wage for the types of participants in the CLAS Stakeholder interview is $41.78. The average respondent is expected to complete the interview in 60 minutes (1 hour), thus making the cost of burden $41.78 x 1 hour = $41.78; this is a one-time total respondent cost burden per interview.
It was determined that the national average hourly wage for the types of participants in the Health and Health Care Organization Leadership Interview is $61.53. The average staff respondent is expected to complete the interview in 60 minutes (1 hour), thus making the cost of burden $61.53 x 1 hour = $61.53; this is a one-time total respondent cost burden per interview.
It was determined that the national average hourly wage for the types of participants in the Health and Health Care Organization Staff Survey is $32.34. The average staff respondent is expected to complete the survey in 15 minutes (or 0.25 hours), thus making the cost of burden $32.34 x 0.25 hours = $8.09; this is a one-time total respondent cost burden per survey.
It was determined that the national average hourly wage for the respondents to the screener survey is $61.98. The average staff respondent is expected to complete the survey in 5 minutes (or 0.08 hours), thus making the cost of burden $61.98 x 0.08 hours = $4.96; this is a one-time total respondent cost burden per survey.
It was determined that the national average hourly wage for the types of participants in the National CLAS Standards Experience Form is $48.72. The average staff respondent is expected to complete the survey in 10 minutes (or 0.17 hours), thus making the cost of burden $48.72 x 0.17 hours = $8.28; this is a one-time total respondent cost burden per survey.
Type of Respondent |
Total Burden (hours) |
Hourly Wage Rate (Mean) |
Total Respondent Costs |
Training and Development Specialists and Managers; Other Management Occupations |
16 |
$41.78 |
$668.48 |
Training and Development Specialists and Managers; Other Management Occupations |
21 |
$41.78 |
$877.38 |
Health and Health Care Organization Executives and Managers |
140 |
$59.59 |
$8342.60 |
Health and Health Care Providers, Managers, and Support Staff |
625 |
$32.34 |
$20,212.50 |
Health and Health Care Organizations Executives |
4,167 |
$71.98 |
$299,940.66 |
Health Care Practitioners and Technical Occupations |
40,000 |
$48.72 |
$1,948,800 |
TOTAL |
|
|
$2,278,841.62 |
Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/Capital Costs
There is no burden of cost to registrants/users as the collection participation is voluntary and free to registrants/users. HHS OMH assumes the burden of cost pertaining to the maintenance and acquisition of necessary equipment (i.e., computer hardware and software) associated with the collection.
Annualized Cost to Federal Government
The total cost to the Federal Government for the implementation of this assessment project by the contractor, SRA International, will be approximately $139,622.00 per year for two years for a cumulative total of $279,244.00. This amount was arrived at by calculating the number of hours respective staff will spend working on this project. It is estimated that staff whose hourly rates (loaded) to the government range from $55 to $75 will use a total of 1294 hours per year. Also included in the total amount are consultant and travel costs.
Explanation for Program Changes or Adjustments
This is a new data collection.
Plans for Tabulation and Publication and Project Time Schedule
Interviews with National CLAS Standards stakeholders, CLAS stakeholders, and health and health care organization leadership will be recorded with a digital tape recorder and transcribed verbatim to enable analysis and interpretation. The data will be coded independently by members of the project staff using Nvivo software to identify structural and thematic codes and to facilitate review of any memos or other documentation collected from the interviews and site visits (if applicable). Data analysis will include immersion in the audiotapes and notes from the interviews; documenting awareness of nuance and meaning through coding for common themes and unique ideas between and across target audiences; and explaining and synthesizing opinions and experiences expressed by interview participants. The focus of analyses will be to develop robust descriptions of the phenomena of adoption and implementation in health and health care organizations, and to help assess and establish the transferability, credibility, dependability, and validity of the implementation process and the project’s findings. The qualitative analyses will also be used to contextualize the data collected through the health and health care organization staff surveys. These data sources will be aggregated to provide an increased, in-depth understanding of National CLAS Standards adoption and implementation among a small number of different types of health and health care organziations. The findings will be triangulated with other data available (e.g., site visit notes/memos, document review) for convergence validation.
Quantitative data will be analyzed using STATA software. Summary statistics (i.e., descriptive statistics such as percentages, means, medians, modes, and standard deviations) and missing data diagnostics will be computed for all variables. Outliers will be evaluated for elimination and missing data will be examined prior to any modeling. Frequencies will also be computed and, to the extent allowable, will be compared across various response categories or variables (i.e., comparative frequencies of responses to items by different types of health care organization leaders, health care organization demographics, etc.). Descriptive analysis of the organizational demographic characteristics will be examined.
After qualitative and quantitative data is analyzed separately, the data will then be merged using data comparison and data transformation techniques. Specifically, the qualitative and quantitative data will be compared and contrasted based on evaluation questions and overall evaluation objectives. Similarities and differences will be organized and displayed. If necessary, one type of data will be transformed into the other type of data, to facilitate additional comparative analysis and interpretation.
All analyses will be directed toward answering the stated evaluation questions, and will be reported as part of the evaluation findings.
The SRA-HDDP Evaluation Team will report on the progress of this evaluation effort in monthly HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee meetings, semiannual reports, and annual reports submitted to the HHS OMH. Upon HHS OMH review and approval, progress and results from this evaluation project will also be shared with key CLAS and National CLAS Standards stakeholders (e.g., the National CLAS Standards National Project Advisory Committee members), and posted on the HHS OMH’s Think Cultural Health website (thinkculturalhealth.hhs.gov). By posting the assessment and evaluation progress and results on the Think Cultural Health website, the progress and findings of this assessment and evaluation will also be publically available.
In coordination and conjunction with the HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee, the SRA-HDDP Evaluation Team will also convene informational and briefing meetings to share and discuss the findings of the evaluation with HHS OMH leadership and staff, and will present progress and findings to the HHS OMH National CLAS Standards Implementation Team.
In consultation with and approval by the HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee, the SRA-HDDP Evaluation Team will discuss and pursue appropriate vehicles to disseminate project findings, including publications, presentations at local and national conferences, webinars, and presentations to health and health care organizations and stakeholders. Contingent upon this, the SRA-HDDP Evaluation Team will work with the HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee to develop and implement a dissemination plan for project findings, including a plan for submission of peer-reviewed publications. For the manuscript development process, SRA-HDDP Evaluation Team will compile the findings of the evaluation and submit at least two manuscripts for peer-review and publication to appropriate academic and/or professional journals in collaboration with the HHS OMH National CLAS Standards Implementation Team Evaluation Subcommittee. Upon HHS OMH review and approval, findings will also be shared in a pre-determined medium and format with other federal, national, regional, state, tribal, and community stakeholders, including policy briefs and technical assistance documents.
See Appendix J for the project time schedule.
Reason(s) Display of OMB Expiration Date is Inappropriate
Not applicable.
Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification.
Collection of Information Employing Statistical Methods
The information collection is a voluntary process for registrants/users registering on the TCH website to receive a newsletter and to complete training.
For this collection, item 17 on OMB 83-1 form is “No”. Therefore, section B of the supporting statement is not applicable.
Attachments to the Supporting Statement
Overarching Evaluation Questions
National CLAS Standards Evaluation Question/Instrumentation Crosswalk
National CLAS Standards Stakeholder Interview
CLAS Stakeholder Interview
Health & Health Care Organization Leadership Interview
Health & Health Care Organization Staff Survey
National CLAS Standards Experience Form
Health & Health Care Organization Screener
Assessment Informed Consent Form
Assessment and Evaluation Plan Timeline
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | How to Write and Submit |
| Author | CMS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |