_WHLI OMB Supporting Statement_FINAL

The U.S. Department of Health and Human Services (HHS) Office on Women’s Health (OWH) is requesting Office of Management and Budget (OMB) approval for new data collection to conduct an evaluation of the utility and outcomes of the Women’s Health Leadership Institute (WHLI) program. The WHLI trained community health workers (CHWs) to gain leadership skills and to use a public health systems approach to address chronic disease and health disparities in their communities. NORC at the University of Chicago (NORC) is conducting this evaluation on behalf of OWH. The goal of the evaluation is to assess the long-term outcomes of the WHLI by examining the extent to which the WHLI was effective in training CHWs to create system-level changes that reduced health disparities.

As populations continue to shift and change, understanding why disparities persist, how they impact women specifically, and determining strategies to address health disparities remains an important priority for policymakers and researchers. Increasingly, evidence suggests the importance of implementing local strategies and solutions to target health disparities.¹ As of 2013, one-third of women nationwide self-identified as a member of a racial or ethnic minority group. Racial and ethnic disparities persist throughout the country and have been found to vary by geographic location.² In general, women have unique health care needs and more frequently utilize healthcare services.³^,⁴In addition, women have higher rates of documented chronic illness, unique reproductive health care needs, and due to social and cultural factors, often experience a heavier burden of health care challenges than men.⁵ For example, because women earn a lower income than men, on average, they are in greater need of state and community resources to assist in paying for health care.⁶ Women are also key to providing social support and maintaining healthy families as most women serve as the primary caregiver for their families, a role that greatly influences household health overall. Women’s health promotion programs provide resources to enable women to take control of their health.

The Patient Protection and Affordable Care Act (ACA) has provided new opportunities to build on and expand existing strategies for improving care for women. In particular, the ACA provides funding for providers to explore a wide variety of service delivery and payment system reforms, all with the goal of making health care more affordable, accessible, and coordinated. The ACA calls on CHWs to strengthen the connection between communities and health systems to improve population health. In addition, the ACA provides an important foundation for research that explores how CHW interventions can be best utilized to improve health systems.⁷

For more than 20 years, OWH has worked to improve the health of millions of women and girls in the United States, focusing specifically on the underserved. OWH was established in 1991 with the mission of improving the health of American women and girls by advancing and coordinating a comprehensive women's health agenda throughout HHS. OWH provides national leadership and coordination on key issues for women and girls by informing and advancing policies, educating the public and professionals, and supporting model programs.

One of OWH’s initiatives is the WHLI. WHLI was developed as an adaptation of the Border Women’s Health Leadership Institute (BWHLI). From 2004 to 2009, BWHLI helped promotoras—lay community health workers (CHWs) who worked on both sides of the U.S.-Mexico border— gain leadership skills to use a public health systems approach to address chronic diseases and health disparities in their communities. CHWs are employed by a variety of organizations and work in nonprofits, community health centers, federally qualified health centers, public health departments, schools, health care facilities, and other private organizations. In 2012, OWH contracted with Mariposa Community Health Center to design, test, and produce the BWHLI curriculum in partnership with the University of Arizona Mel and Enid Zuckerman College of Public Health. (AzPRC).

In 2012, Mariposa Community Health Center began the development and implementation of the WHLI; the program concluded in 2014. The WHLI used the English translation of the BWHLI curriculum and trained CHWs with different skills and experiences—not solely promotoras. WHLI employed a Train the Trainers Model, where Master Trainers (MTs) learned to deliver the WHLI training curriculum to CHWs. In addition to receiving training on outreach, community education, counseling, and social support, CHWs were trained to create public health systems changes in their communities. Following completion of the WHLI, CHWs were expected to develop and complete a community action project (CAP) and to use their expanded skills in community assessment, community organizing, and advocacy to address disparities. WHLI successfully trained 18 MTs to conduct workshops using the WHLI curriculum, and these 18 MTs trained 422 CHWs across the U.S. to address risk factors associated with health disparities.

OWH has contracted with NORC to provide program evaluation and planning support for the Office. As part of this project, NORC will conduct a retrospective evaluation of the WHLI that will expand upon a previous evaluation conducted by AzPRC that focused on process measures during the implementation of the training and short-term and selected intermediate outcomes. The goal of the current evaluation is to assess intermediate and long-term outcomes of the WHLI by examining the extent to which the WHLI has been effective several years later.

OWH is seeking approval from OMB to conduct:

A short online survey of CHWs trained by the program to assess experience with and perceptions about the WHLI training and associated activities, and the impact of the training within the communities they serve.
In-depth semi-structured telephone interviews with CHWs, MTs, CHW worksite supervisors, and community stakeholders to assess policy and other system-level changes that have been established to promote health and wellness as a result of the WHLI in organizations and/or communities.

This evaluation supports the mission and strategic plan for both HHS and OWH. The study supports the HHS strategic goal of “advancing scientific knowledge and innovation.” Specifically, this evaluation addresses objective D by, “increasing our understanding of what works in public health and human services practice.”⁸ In addition, the evaluation supports the HHS strategic goal of “ensuring efficiency, accountability, and effectiveness of HHS programs.”⁹ The evaluation results are critical to measuring the efficacy of the use of government funds. The evaluation results will also help OWH with planning and developing future training initiatives and supports the OWH strategic goals of informing and advancing policies as well as identifying, testing and promoting effective programs for women and girls.¹⁰ In addition, the evaluation results will help OWH to better understand the intermediate and long-term outcomes of the WHLI program years after completion, and to assess the extent to which CHWs are able to make an impact on health disparities in their communities, as a result of the WHLI training.

The following subsections of this document provide detailed justification of the collection of this data, in accordance with OMB requirements. This collection of data is authorized by Section 301 of the U.S. Public Health Service Act (42 U.S.C.241). A copy of this legislation can be found in Attachment 1.

2. Purpose and Use of Information

The purpose of this evaluation is to examine the extent to which the WHLI has been effective in training CHWs to implement activities in communities that result in system-level changes to reduce health disparities. This study has several key research questions:

Did CHW trainees retain the leadership knowledge and competencies they learned through the WHLI, since the WHLI training?
Did CHWs apply the competencies in their leadership activities? If yes, how?
Were positive systemic and/or community level changes made around women’s health issues as a result of CHWs’ increased leadership capacity?
Does leadership knowledge and competencies differ by individual CHW characteristics (e.g. years of work experience, education level)?
Do CHWs’ CAPs and leadership activities vary by individual CHW characteristics and the characteristics of their home organization (e.g. type and size of organization)?
What are the differences between CHWs who developed and implemented CAPs and those who did not? (e.g. education and knowledge, age, leadership activities, external factors)
How could the WHLI training be improved?

Primary data will be first collected via an online survey which will be followed up with in-depth telephone interviews. The online survey will be administered to the WHLI-trained CHWs, and will ask questions about CHWs’ competencies as a result of the training and their application of the skills to their work since completion of the training. The online survey will also include selected questions about CAPs, such as the facilitators to and barriers of developing and implementing CAPs. In-depth telephone interviews with CHWs, Master Trainers, CHW worksite supervisors, and community stakeholders will also be conducted. These in-depth interviews will provide more detailed information about the impact of the WHLI from a range of perspectives that will be important for determining outcomes of the program. Upon completing the evaluation, a qualitative and quantitative analysis will be conducted and a final report will be developed and disseminated.

Data from the WHLI survey and interviews will provide OWH with information to better understand what parts of the WHLI program were most successful for implementing changes in communities, and to identify aspects of the training that need improvement. This information will help OWH understand for whom the WHLI training may have worked best, through examining how intended outcomes differed by individual and organizational characteristics. In addition, the collection of information on facilitators to and barriers of CHWs’ leadership activities and the development of the CAPs will also help OWH to take these contextual factors into consideration when developing future trainings, in order to work more effectively with its stakeholders to achieve its goals and mission of improving the health of women and girls in communities around the country. Further, collecting this information will increase the utility of prior evaluations of the WHLI, and will help OWH accurately assess and evaluate the long-term impact of the program. In addition, findings from this evaluation may also help identify and validate evidence-based practices in CHW training models for the field of women’s health that can be disseminated nationally.

3. Use of Improved Information Technology and Burden Reduction

The WHLI evaluation will involve a secure online survey using a CAWI (computer-assisted web interview) system – data collection technology specifically designed to reduce respondent burden. CAWI is a user-friendly web interface designed to assist respondents in completing the survey by providing features that reduce respondent burden and ensure complete and accurate data. For example, the survey design will incorporate proper use of visual stimuli (e.g., “continue” and “back” arrows) to ensure that respondents can easily navigate the questionnaire, as well as range checks and automatic prompts to ensure inter-item consistency and that questions are not inadvertently skipped. Respondents will be provided with a “resume” capability that allows them to break off the session mid-survey and then return to the survey at a later time to complete it without losing previously entered data. In addition, respondents will be provided with contact information so they can easily submit requests for technical support via email or telephone.

A sequential multi-mode contact approach (email, mail, telephone) will be used to contact CHWs to participate in the evaluation. Recruitment will first entail emailing an online survey invitation to CHWs that provides an introduction to the evaluation and contains a link to the online survey. Two reminder email invitations will then be sent to non-responders over a week’s time. CHWs who do not respond to the email invitations or who do not have a valid email address will be sent a letter through the mail followed by a reminder postcard. The mail materials will also provide an introduction to the evaluation and a link to the online survey. Additional targeted telephone prompting of non-respondents will be used to increase participation. Telephone interviewers will use standard scripts for answering machine messages and when initial contact is made with a CHW or associated household member. Respondents contacted by telephone will be given the choice to complete the survey by telephone. It is anticipated that respondents will choose the option of least personal burden, thereby reducing the overall burden of the study. If the respondent opts to complete the survey by telephone, the interviewer will access the respondent’s case online and enter responses directly into the online survey.

Participants will complete the survey at their own pace; completion of the survey should take approximately 25 minutes. The majority of survey questions will be in a multiple choice format in order to reduce the amount of time and burden on participants. The online survey is compatible with mobile devices, tablets, as well as desktop and laptop computers, providing respondents multiple device options of completing the survey. In addition, the survey is designed to allow respondents to skip any questions that they do not wish to answer, and to leave the survey and resume it at any time.

4. Efforts to Identify Duplication and Use of Similar Information

The purpose of this evaluation is to assess the intermediate and long-term outcomes of the WHLI by studying the experiences of the CHWs and MTs several years after participation in the program. The previous evaluation conducted by AzPRC evaluated short term (with all trainees immediately following the WHLI training) and selected intermediate outcomes (with 28 selected CHWs 9 months after the WHLI training) of the training by comparing CHWs’ knowledge and competencies before and after the training. AzPRC found that the CHWs acquired knowledge related to the WHLI competencies and assumed a greater leadership role in their organization and in their community after the training. In addition, the National Community Health Worker Advocacy Survey, conducted by the University of Arizona, found that WHLI-trained CHWs gained knowledge about how to influence community and organization decision-making processes and develop strategies for change.

This evaluation will build upon this previous work by assessing the intermediate and long-term outcomes of the WHLI, which has never been done. Specifically, the evaluation will examine the impact of WHLI on organizational and community changes. Although short-term outcomes explored through this evaluation were previously examined by AzPRC, these outcomes will be used to compare results from the previous evaluation to see if training from the WHLI was sustainable and led to similar outcomes years later. The previous work helped to inform the development of survey and interview questions for the current data collection effort, ensuring that questions are not duplicative of any previous data collection. A review of documents from the previous evaluation, including the final report, informed the development of survey instruments for the current data collection effort.

No effort to collect similar data is currently being conducted within the agency. Additionally, no data collection efforts outside the agency have been made to collect this specific data.

5. Impact on Small Businesses or Other Small Entities

This data collection may involve the collection of information from employees of small businesses or other small entities, i.e. CHWs may work for small businesses. In consideration of respondents’ time, NORC has developed the data collection instruments to include the minimal amount of information required to effectively evaluate the WHLI. Completion of the survey should take approximately 25 minutes, and interviews will not exceed 30 minutes each. In addition, utilizing an online survey method allows for flexibility in when the participants choose to participate, which alleviates the burden further. We believe the burden of participation is low and therefore the impact on small entities is minimal. This study will not unduly affect small businesses or entities.

6. Consequences of Collecting the Information Less Frequently

The design of this study requires that data be collected from the majority of respondents at only one time point. All CHWs who complete a survey and indicate during the survey that they have completed a CAP, and a selected group of CHWs that has achieved substantial organizational or community changes as a result of their leadership activities, will be invited to participate in an additional telephone interview. We expect the total number of CHWs invited to participate in a follow up telephone discussion to be no more than 40. The federal government will benefit from having information available about the intermediate and long-term outcomes of the WHLI. By collecting this data, OWH will gain essential information to understand the impact of the WHLI and the types of changes that have occurred in CHWs’ organizations and communities as a result of this program. This data collection will enhance OWH’s ability to create effective programs that train CHWs to reduce health disparities, and provide important information that may be able to inform future training programs for CHWs.

There are no legal obstacles to reduce respondent burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request complies with all guidelines of 5 CFR 1320.5 (d) (2). No special circumstances apply.

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

The notice required in 5 CFR 1320.8(d) was published in the Federal Register, volume 81 number 83 page 25681, on April 29, 2016.(Attachment 2). The required notice of OMB review will be published in the Federal Register concurrently with the submission of this document. There were no comments received from the public regarding this data collection.

8. b. Outside Consultations

NORC at the University of Chicago staff and advisors consulted include (full contact details for these individuals can be found in Section B.5 of this document):

Caitlin Oppenheimer, M.P.H.

Felicia Cerbone, M.A.

Alycia Bayne, M.P.A.

Weiwei Liu, Ph.D.

Britta Anderson, Ph.D.

Michael Stern, Ph.D.

Jacquelyn George, B.A.

Amy Rosenfeld, M.P.H.

Rachel Weingart, B.A.

Katherine Groesbeck, B.A.

The DHHS/OWH Project Officer for this data collection is Dr. Adrienne Smith. During the development phase, several OWH staff reviewed both surveys and provided feedback to NORC. In addition, a WHLI-trained Master Trainer, Julie St. John, was consulted during the development of the survey questionnaire and semi-structured interview protocols. Julie helped review content for inclusion in the study and provided suggestions for increasing clarity and readability. The NORC team also selected CHWs and several NORC staff to participate in a pilot study in which the questionnaires and data collection protocol were tested. Respondents from the pilot provided detailed feedback on the survey. Please see Attachment 3 which contains the Pilot Report.

9. Explanation of Any Payment or Gift to Respondents

When piloting the survey, NORC staff experienced challenges recruiting CHWs to complete the pilot. This may be due to the fact that at least three years have elapsed since participation in the WHLI program. NORC staff had difficulty locating CHWs due to outdated contact information (i.e., emails bounced back and phone numbers no longer in service). In addition, several CHWs did not reply to emails or phone calls, either because contact information was outdated or because they were not interested in participating in the study. One CHW who was successfully contacted declined participation. Therefore, NORC plans to offer a post-incentive to improve response rates since only a very small number of respondents, if any, would still have a working relationship with WHLI since the program was completed in May 2014.

It is unlikely that many of the over 420 CHWs, 20 community stakeholders, and 20 CHW worksite supervisors still have a relationship with WHLI. There is a possibility that some of the 18 Master Trainers could still have a relationship with WHLI, but again since it has been several years since completion of this project it is unlikely. Since we have a finite number of CHWS who are eligible to participate and who can be recruited to complete the survey it is important that we are able to maximize survey response in order to have the needed information to complete the evaluation. Offering an incentive in the amount of $10 to respondents will aid us in achieving a sufficient sample size to conduct analyses.

CHWs who do not respond to email invitations (Attachment 12) to complete the online survey or do not have a valid email address will be sent an invitation letter through the mail requesting their participation in the study (Attachment 15). The invitation letter will contain a promise of a post-incentive upon completion of the survey.

All respondents will be offered a $10 gift card incentive for completion of the online survey and a $10 gift card incentive for completion of the telephone interviews. The $10 gift cards will be delivered electronically through email at the conclusion of the survey. Respondents will be able to enter an email address where the gift card can be delivered after completion of the survey. For respondents completing an interview, an email address will be recorded and the gift card will be sent electronically at the end of the interview. If a respondent does not have an available email address, we will offer to send a physical gift card to an address they provide.

Research suggests that incentives increase response rates in surveys and that, as the value of the incentive increases, so too do the number of completes. For instance, Stern, Bilgen, and Wolter have shown that a $10 gift card significantly improved response over a $5 and $2 gift card.¹¹ Other research has demonstrated that $20 produced a marked improvement over $10 by improving response and increasing variation in those who completed the survey.¹²

10. Assurance of Confidentiality Provided to Respondents

OWH is contracting with NORC to collect and analyze the survey and interview data. NORC is responsible for all data collection activities, including administrative oversight, data collection, and data analysis. This information collection will not involve any personal health information.

The information collection fully complies with all aspects of the Privacy Act. Personal identification information (i.e., respondent names and contact information) will be collected from the CHWs during survey administration, which is necessary for the purpose of merging the data with data from the previous evaluation of WHLI as well as for identifying additional participants for the telephone interviews. All identifiers collected from program participants will be for the purpose of merging with the previous evaluation data and identifying participants for telephone interviews. All personal identifier information will be maintained on secure servers at NORC and this information will not be included in any datasets delivered to OWH. Contact information including participants names will not be linked to responses. The project staff will keep all PII in a separate, secure file and will assign a unique identifier to each participant which will be kept in a separate file from the data collected. PII will not be used directly to retrieve a given record corresponding to an individual. Only approved members of the project team at NORC will have access to the data collected as well as the participant’s name and contact information. Any potential PII will be stripped from datasets that will be used in analysis. Data will be compiled into a SPSS dataset; and results will be reported in the aggregate. Respondent names and contact information will not be included in any information viewed by OWH, or by any other HHS officials. Further, the study’s final report will not identify any respondents or their organizations. In addition, any data files shared with OWH will be stripped of any personally identifiable information. At the conclusion of the study, the final dataset will be delivered to OWH in a de-identified format using a secure file transfer protocol (SFTP) site.

NORC uses a secure, online site (i.e. Voxco) designed to collect survey data. The first page of the survey will inform participants of the purpose of the assessment, how the information collected will be used, the estimated time to complete, and that all responses will be kept private (Attachment 4). In addition, all telephone interviews will begin with verbal consent informing participants that their responses will be kept private and that they will not be identified (Attachment 5). All information collected will be kept private to the extent possible by law. NORC will use the data for analytic and evaluation purposes. NORC will ensure proper storage of all project data. NORC published internal standards around data transmittal that are strictly followed. These include the transfer and receipt of all personally identifiable information (PII) in an encrypted format utilizing FIPS 140-2 compliance, transmitting all company confidential documents or files with password protection or in an encrypted format, and the transfer of all data utilizing the NIST-800-53 security framework.

IRB Approval

The proposed evaluation effort and data collection instruments have been reviewed and approved by NORC’s Institutional Review Board (Attachment 6).

11. Justification of Sensitive Questions

The items and questions asked in this evaluation are not of a sensitive nature, except for the questions regarding race and ethnicity, as a requirement of all HHS data collection. Moreover, all questionnaires used in the evaluation have been reviewed by an Institutional Review Board to ensure that respondents’ rights are protected. All participants will be advised of the voluntary nature of their participation in the survey and interview or any of its components. A “do not wish to answer” option will be provided in all survey questions, and participants are informed that they may stop the survey and interview at any time without any consequences.

12. Estimates of Annualized Hour and Cost Burden

Exhibit 1 provides estimates of the collection burden on each of the five data collection forms. This evaluation is a one-time effort conducted for four months with an estimated 245 total burden hours. The frequency of response is one survey per CHW that participated in the WHLI, and one interview per person. The online survey will be targeted to 422 CHW WHLI participants (unduplicated count). The telephone interviews will be conducted with 40 CHWs, 18 Master Trainers, 20 worksite supervisors, and 20 community stakeholders. We estimate that the survey will take approximately 25 minutes for each participating CHW to complete. We also estimate each interview will take approximately 30 minutes to conduct.

Exhibit 2 presents the annualized hourly costs for respondents. Estimates for hourly wage rates for CHWs and other respondents were taken from the U.S. Department of Labor, Bureau of Labor Statistics.ⁱ For all of these data collection efforts including both the survey and interviews, the total cost burden is estimated at $4,540.84 (see Exhibit 2).

Exhibit 1. Estimated Annualized Burden Hours

Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Online Survey— All CHWs	422	1	25/60	176*
Telephone Interviews—CHWs with completed CAPs or other leadership activities	40	1	30/60	20
Telephone Interviews—Master Trainers	18	1	30/60	9
Telephone Interviews—CHW Worksite Supervisors	20	1	30/60	10
Telephone Interviews— Community Stakeholders	20	1	30/60	10
Total	520	—	—	225

*Numbers have been rounded

Exhibit 2. Estimated Cost Burden

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
WHLI CHWs¹
Online Survey	176	$18.35	$3,229.60
Telephone Interviews	20	$18.35	$367
WHLI Master Trainers²
Telephone Interviews	9	$32.56	$293.04
CHW Worksite Supervisors²
Telephone Interviews	10	$32.56	$325.60
Community Stakeholders²
Telephone Interviews	10	$32.56	$325.60
Total	225	—	$4,540.84

¹ Based on mean hourly wage in 2014 for Community Health Workers from the Department of Labor, Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes211094.htm)

²Based on mean hourly wage in 2014 for Social and Community Service Managers from the Department of Labor, Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes119151.htm)

13. Estimates of other Total Annual Cost Burden to Respondents or Record-keepers/Capital Costs

There are no additional respondent costs associated with start-up or capital investments. Additionally, there is no operational, maintenance, or equipment respondent costs associated with continued participation in the evaluation.

14. Annualized Cost to the Federal Government

All costs for conducting the Evaluation of the Women’s Health Leadership Institute Program are included in the contract between the Department of Health and Human Services and NORC under contract HHSP233201500048I, task order TO HHSP23337005. The total estimated cost is $399,000 over a 24 month period to conduct the surveys and interviews, analyze and present findings, and write a final report and briefing. This is an annualized cost of $199,500.

15. Explanation for Program Changes or Adjustments

This is a new collection of data.

16. Plans for Tabulation and Publication and Project Time Schedule

The data collected will be analyzed and interpreted to produce a final study report to OWH as well as final briefings. NORC will deliver the final report to OWH in hardcopy and in a print-ready electronic format. Publication of findings on the internet is at OWH’s discretion. The remainder of this section discusses data sources and the analytic techniques that will be employed. Information will be collected over a four-month period following OMB approval. Exhibit 3 provides a schedule of data collection, analysis, and reporting following OMB approval.

Exhibit 3. Schedule for Data Collection, Analysis, and Publication

Activity	Expected Date of Completion
Online survey sent to respondents	1- 3 months following OMB approval
Telephone interviews with respondents	3-4 months following OMB approval
Quantitative and qualitative data analysis	4-6 months following OMB approval
Preliminary briefing and preparation of draft report	7-8 months following OMB approval
Final Report	8-9 months following OMB approval
Final Briefing	9 months following OMB approval

Quantitative Data Analysis. Quantitative data to be captured in the survey (Attachment 7) will consist of measures of knowledge and competencies, leadership activities and CAPs, and perceived impact of the training on the respondent’s ability to create positive systematic change. The data will be exported to SPSS from the online website and cleaned. Prior to conducting quantitative analysis of the survey data, quality assurance and quality control checks will be completed on the data set to identify missing, invalid, inconsistent or otherwise potentially inaccurate records. Records will be tracked through the logical flow of data to ensure that conditional skip logic was reflected in the data as expected. In order to answer the primary research questions of this evaluation, discussed above in section 2, univariate analyses on key variables will be conducted to examine response distributions and to identify outliers. Further, we will implement appropriate statistical procedures (e.g. independent sample t-test for continuous outcomes and chi-square tests for categorical outcomes), to compare the distribution of these variables by selected individual and organizational characteristics. When an identified dataset is obtained from previous evaluators, NORC will also merge the two datasets to examine changes in knowledge and competencies over time.

Qualitative Data Analysis: NORC will collect two different types of qualitative, open-ended data: open-ended responses from the online survey (Attachment 7) and interview responses from the in-depth telephone interviews (Attachments 8, 9, 10, and 11). NORC will conduct a content analysis of the qualitative data, through coding of interview notes for themes and commonalities (we will also explore software packages such as QSR NVivo v10 to assist coding). Interview transcripts will be created using interview and audio recordings of the interview. One or more members of the NORC team will code the cleaned transcripts. Once qualitative data from all the interviews is complete, formal analysis of themes and categories of themes will be conducted. A codebook will be developed based on conversations with CHWs and feedback from OWH. Coders will be trained on the initial code book and a preliminary inter-rater reliability (IRR) check will be conducted. Refinement of the codebook will continue over the life of the evaluation, using an iterative process of code generation and team consultation as coding proceeds.

Publication: Evaluation findings will be summarized in a comprehensive Evaluation Report and Executive Summary developed by NORC for OWH. The findings from this evaluation will be shared during a briefing with OWH agency officials and staff.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

OMB expiration dates will be displayed on all materials.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement identified in item 19 “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.

ATTACHMENTS

U.S. Public Health Service Act (42 U.S.C.241)
60-Day Federal Register Notice
Pilot Test Report
Informed Consent for Survey
Interview Verbal Informed Consent
NORC IRB Approval
Online Survey Instrument
Community Health Worker Interview Protocol
Master Trainer Interview Protocol
Supervisor Interview Protocol
Stakeholder Interview Protocol
Initial Email Invitation for Survey
Reminder Email #1 for Survey
Reminder Email #2 for Survey
Initial Mail Invitation for Survey
Outer Envelope for Survey
Reminder Postcard for Survey
Telephone Prompting Script for Survey
Voicemail and Callback Script for Survey
Phone Invitation for Telephone Interview (CHWs)
Email Invitation for Telephone Interview (CHWs)
Phone Invitation for Telephone Interview (MTs)
Email Invitation for Telephone Interview (MTs)
Phone Invitation for Telephone Interview (Supervisors)
Email Invitation for Telephone Interview (Supervisors)
Phone Invitation for Telephone Interview (Stakeholders)
Email Invitation for Telephone Interview (Stakeholders)

1 Kaiser Family Foundation. 2013. Putting Women’s Health Care Disparities on the Map: Examining Ethnic and Racial Disparities at the State Level. Accessed 16 February 2016 at https://kaiserfamilyfoundation.files.wordpress.com/2013/01/7886es.pdf

2 Kaiser Family Foundation. 2013. Putting Women’s Health Care Disparities on the Map: Examining Ethnic and Racial Disparities at the State Level. Accessed 16 February 2016 at https://kaiserfamilyfoundation.files.wordpress.com/2013/01/7886es.pdf

3 Cameron, KA, Song J, Manheim LM, Dunlop DD. 2010. Gender Disparities in Health and Healthcare Use Among Older Adults. Journal of Women’s Health, 19(9): 1643-1650.

4 Bertakis, KD, Azari R, Helms LJ, Callahan EJ, Robbins JA. 2000. Gender Differences in the Utilization of Health Care Services. Journal of Family Practice, 49(2): 147-52.

5 The Centers for Disease Control and Prevention (CDC). 2009. The Power of Prevention: Chronic Disease… the Public Health Challenge of the 21st Century. Accessed 26 February 2016 at http://www.cdc.gov/chronicdisease/pdf/2009-Power-of-Prevention.pdf

6 Breshears, J, Foreman M, Rueschhoff A. Improving Women’s Health Challenges, Access, and Prevention. National Conference of State Legislatures. Accessed 9 March 2016 at http://www.ncsl.org/research/health/improving-womens-health-2013.aspx

7 Shah M, Heisler M, Davis M. 2014. Community Health Workers and the Patient Protection and Affordable Care Act: An Opportunity for a Research, Advocacy, and Policy Agenda. Journal of Health Care of the Poor and Underserved, 25(1): 17-24.

8 Department of Health and Human Services. HHS Strategic Plan & Priorities for FY2014-2018. Accessed 16 February 2016 at http://www.hhs.gov/about/strategic-plan/strategic-goal-4/index.html

9 Department of Health and Human Services. HHS Strategic Plan & Priorities for FY2014-2018. Accessed 16 February 2016 at http://www.hhs.gov/about/strategic-plan/strategic-goal-4/index.html

10 Office on Women’s Health. Office on Women’s Health Strategic Plan FY2014-2016. Accessed 18 February 2016 at http://www.womenshealth.gov/about-us/who-we-are/owhstrategicplan.pdf

11 Stern, MJ, Bilgen I, Wolter KM. 2014. Do Sequence and Mode of Contact Impact Response Rates for Web Only Surveys? Presented at the 69th Annual Conference of the American Association for Public Opinion Research (AAPOR) in Anaheim, CA.

12 Lewis D, Creighton K. 2005. The Use of Monetary Incentives in the Survey of Income and Program Participation. The American Association for Public Opinion Research (AAPOR) 60th Annual Conference.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Weiwei Liu
File Modified	0000-00-00
File Created	2021-01-23

_WHLI OMB Supporting Statement_FINAL