B. Collection of Information Employing Statistical Methods
1. Respondent Universe and Sampling Methods
Primary data collection for which OMB approval is being sought consists of the following activities:
Annual interview with the co-chairs of the RHECs. Interviews will be conducted with each of the co-chairs of the RHECs. There are 10 RHECs in total and two co-chairs for each RHEC. The RHEC co-chairs are expected to play a critical role in providing leadership for the RHEC; therefore, detailed interviews with the chairs are needed. Both co-chairs in a given region will participate in a single interview, which will save time.
Annual interview with the RHEC Subcommittee chairs. The people who lead the RHEC Subcommittees will be asked to take part in a group telephone interview, answering questions on the effectiveness of the RHEC Subcommittees. There are up to five Subcommittee chairs per RHEC, and there will be one interview per RHEC. The Subcommittee chairs have the greatest insight into the work of the Subcommittees, which will play an important role in implementing the RHEC work plans.
Annual survey of all RHEC members. Each RHEC has a maximum membership of 35, and all members will be asked to participate in a Web-based survey. Membership is diverse, and including the whole population will provide more detailed data for each individual RHEC. It will be important to find out what different categories of members are saying about RHEC implementation and to use that feedback in shaping future implementation strategies.
Annual survey of key NPA partner organizations. NPA partner organizations that have developed a memorandum of agreement to support NPA implementation will be asked to complete a Web-based survey. Key partners are being interviewed because their commitment to implementing the NPA is explicit and it will be important to understand the outcomes of those commitments. It is expected that there will be up to 15 partners that have expressed their commitment in an MOA. If there are more than 15 key partners, purposive sampling will be done to select partners whose commitment is likely to be most important to the implementation of the NPA.
Annual surveys of SOMH Directors or Coordinators and representatives from State Departments of Health. SOMH Directors or Coordinators and representatives from the State Departments of Health will be surveyed regarding their State’s experiences implementing the NPA. Directors or Coordinators are being surveyed as the key informants at the State level with the most advanced knowledge of efforts around the State to address health disparities. Representatives from State Departments of Health are being surveyed because in order for the NPA to succeed, efforts related to health disparity will have to infuse the work of State Health Departments and not just be focused within Offices of Minority Health. Both these State representatives will be asked about community-level initiatives related to health disparities and State-level efforts to support communities in addressing this issue. The full population of States and jurisdictions implementing the NPA are being included because it is a national plan and it will be important to assess how State characteristics are affecting implementation at the State level. Since the relevant State characteristics are not yet known, a sampling strategy would run the risk of missing important factors that will shape implementation. The results of this survey will be used to select regions, states, and communities for further analysis through the case study.
2. Procedures for the Collection of Information
The procedures listed below are part of a mixed-methods design that addresses the complex nature of NPA implementation. An evaluation of the NPA requires surveys and interviews with a variety of respondents who operate in different organizations and at different levels. Primary data collection will be supplemented by the collection of available documents. These documents will help Community Science provide additional details about topics addressed in the interviews or surveys and inform follow-up questions during interviews.
Web-based Surveys
The Web-based surveys are designed to gather data about the processes used in implementation and how the NPA is being implemented. The surveys will consist primarily of close-ended questions with categorical response options. The data collected from the surveys will be used to analyze how effectively various NPA partners are carrying out their expected role in implementing the NPA. In addition, the surveys will be used to help determine the sample and focus for other data collection activities. For example, the survey with State representatives will be used to determine which state representatives should be interviewed and for what information. All surveys and interviews will be used to determine the regions, states, and communities that will be the focus of case study activities.
To limit cost to the government and minimize burden on respondents, surveys will be conducted via a secured Web site. Recruitment for the survey will occur through an initial notification letter from OMH and follow-up contacts via email and telephone, if necessary, from Community Science. Respondents will be sent a link to the Web-based survey and detailed instructions on how to enter and submit their responses. Community Science will also offer technical assistance, if needed, to help respondents successfully complete the survey. Once a survey has been submitted, the Community Science survey manager will review the survey file to confirm that it is complete and, if necessary, follow up with respondents to fill in any missing information. Once surveys are completed, the data file will be exported from the survey software and into a format usable by statistical software packages.
Telephone Interviews
The purpose of the telephone interviews is to gather more-comprehensive information about NPA implementation across a range of respondents who will play a key role in implementing the plan. The semi-structured interview guides consist of open-ended questions and a series of probes to elicit more detailed responses.
To limit cost to the government and minimize burden on respondents, the interviews will be conducted over the telephone. Recruitment for the survey will occur through an initial notification letter from OMH; Community Science will follow up via telephone and email. The contacts will be asked to identify appropriate interviewees and several possible dates and times for the interviews; Community Science will make every effort to accommodate interviewees’ schedules. Prior to the scheduled interview date, interviewers will review any available information from the respondent’s organization related to NPA implementation, including survey responses. This information will be used to familiarize interviewers with the respondent’s organization and its activities.
During the interviews, a notetaker will take detailed typed or written notes using a standard template. Following each interview, notetakers will be required to finalize and submit their notes within 48 hours. Data from the interviews will be compiled in a qualitative software program for coding and detailed analysis.
Case Studies
Although OMB clearance is not being sought for the case study protocols, it is important to note that efforts have also been made to minimize burden during this phase of data collection as well. The surveys and other data sources will be used to select case study targets that will yield the richest information about NPA implementation. The case studies will focus on the areas where activities are most likely to be occurring in a given year. For instance, during the first year of the evaluation, case studies were conducted at the regional level, since the RHECs were the first form of structure to be established and become operational. Since that time, the RHECs and other NPA stakeholders have supported efforts at the national, state, local, and community levels. Therefore, in subsequent years of the evaluation, case studies have been conducted with entities at the national, state, local, and community levels and we will continue this procedure in future years of the evaluation. The protocol development process will continue to include a review of all relevant data collected at earlier stages of the evaluation, including surveys, interviews, and documents related to the subject of the case study. Questions will continue to be designed to collect additional detail or clarify ambiguities.
3. Methods to Maximize Response Rates and Deal with Nonresponse
Obtaining the cooperation of respondents is critical to the success of this evaluation. Every effort will be made to maximize the response rate to the surveys and interviews. There are many reasons to expect cooperation from the respondents. Many of the respondents have committed their time and effort to implementing the NPA, and it is likely that they will be fully supportive of efforts to evaluate the plan. While State Health Department officials are not as directly involved with NPA implementation, they are committed professionals who are widely supportive of efforts to reduce health disparities. They also have frequent contacts with Federal officials and are likely to be responsive to requests from Federal agencies to participate in the evaluation. The targeted response rate for both the Web-based survey and the telephone interviews is therefore 85 percent to 90 percent.
Every effort will be made to meet or exceed the targeted response rate for data collection. A notification letter from the OMH NPA Lead will be sent to sites to introduce the study, explain its importance, and encourage sites to participate. Community Science staff will make a series of follow-up contacts with each site by telephone or email to reinforce the importance of participation, answer questions, and offer support.
In addition, Community Science will use the following proven methods to maximize participation and cooperation in the study:
• Paper copies of the survey instrument will be distributed to sites that prefer not to utilize the survey Web site.
• Community Science staff will provide technical support if respondents encounter difficulties completing the Web-based survey.
• Throughout the telephone interview recruitment process, staff will work with sites to accommodate the schedules of all interviewees. Multiple staff will be available to conduct interviews, which will increase the likelihood that at least one Community Science team member will be available at times specified by respondents. Also, interviewers will make themselves available in both the early morning and evening to accommodate sites in different time zones.
• Telephone interviewees will be provided with the discussion topics in advance to allow them sufficient time to prepare.
• All sites and respondents will be given a realistic appraisal of the time, information, and human resources that will be expected.
In previous years of the evaluation, these steps produced collegial relationships between the research team and participants and contributed to high response rates.
4. Tests of Procedures or Methods to Be Undertaken
Prior to the launch of the evaluation, a pilot test was conducted to ensure that the data collection process proceeds as intended. Each survey instruments was tested with two respondents from the appropriate respondent category. Each interview guide was tested with nine respondents from the appropriate respondent category. For the Web-based surveys, a link to the beta version of the survey was sent to two respondents who agreed to test it. This process ensured that questions elicit interpretable and intended responses and revealed any problems with completing the survey on the Web site or processing survey data. In addition, Community Science conduced pilot-test telephone interviews with the same or a different set of respondents. This process ensured that interview questions are easy to understand and produce intended responses.
The survey and interview instruments were revised based on information gleaned from the pilot tests and submitted to OMB for review, along with a refined estimate of the response burden.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Analyzing Data
The following individuals contributed to the questionnaires and study design and will be involved in the interpretation and analysis of findings:
Kien Lee, Ph.D.
Principal Associate/Vice President
(301) 519-0722
Oscar Espinosa
Senior Associate
(301) 519-0722
oespinosa@communityscience.com
Brandon Coffee-Borden
Associate
(301) 519-0722
bcoffeeborden@communityscience.com
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Abigail Baum |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |