TABLE OF CONTENTS
B.1. Respondent Universe and Sampling Methods 3
B.2. Information Collection Procedures 3
B.2.1. Statistical Methodology for Stratification and Sample Selection 4
B.2.2. Estimation Procedure 6
B.2.3. Degree of Accuracy Needed for the Purpose Described in the Justification 6
B.2.4. Unusual Problems Requiring Specialized Sampling Procedures 7
B.2.5. Use of Periodic (Less Frequent Than Annual) Data Collection Cycles 7
On behalf of the Administration for Community Living/Administration on Aging (ACL/AoA), within the U.S. Department of Health and Human Services (DHHS), NORC at the University of Chicago, will collect information for the Outcome Evaluation and Special Studies Related to the Long-Term Care Ombudsman Program (LTCOP) (Contract # HHSP233201500048I). Data collection will involve gathering data from four State Long-Term Care Ombudsman programs (SLTCOPs), each representing one of four program structure/organizational placement combinations.
For each SLTCOP, the contractor is responsible for the design and administration of focus group discussion guides, interview protocols, and web-based surveys. ACL seeks clearance for these data collection activities. This package includes the following data collection instruments:
Focus Groups: In each state, focus groups will be conducted with 4 groups of residents, 2 groups of family members, and 4 groups of facility staff.
Interview Protocols: In each state, interviews will be conducted with 10 selected stakeholders/representatives of entities with whom the program coordinates.
Surveys: Surveys also will be administered to a random sample of nursing home and board and care facility administrators in each of the four selected states. Target sample sizes in each state will vary depending on the number of nursing homes and board and care homes that are operating in those states.
Once the four participating SLTCOPs have been identified, ACL/AoA will reach out to State Ombudsmen to inform them of their inclusion in the study. Notifications will take place in late 2020. The project team will then contact and work with each State Ombudsman to coordinate and optimize data collection activities. This includes identifying 2 nursing homes and 2 board and care homes where focus groups can be conducted with residents, family members, and facility staff. Facility staff will be compensated with a $20 gift card for their participation. A one-page flyer will also be developed to share with programs to assist in the recruitment effort.
This initial contact will provide the project team with an opportunity to ask State and local program staff about how to best reach out to facility administrators and encourage participation (including whether to engage in additional recruitment efforts, such as holding informational sessions), any issues that may impact the ability to collect data from respondent groups, as well as any other considerations that may impact response rates.
The project team will also seek State Ombudsmen’s assistance with identifying and providing contact information for program stakeholders and communicating the importance of their participation in the study. A member of the team will contact individuals by phone or email to schedule an interview. Verbal consent will be obtained prior to conducting interviews with stakeholders. Interviews will last approximately 60 minutes. Stakeholders will not receive any form of monetary or tangible compensation for participation.
In each state, the project team will administer web-based surveys to a random sample of nursing home and board and care home administrators. To help optimize response rates, the project team will work with State Ombudsmen and the state provider association to encourage participation. Facility administrators will be contacted by email in ____ of 2020 with a request to complete the survey. Respondents will click on a survey link in the email and enter a unique user ID and password. Survey participants will be presented with a screen that provides a brief overview of the study and informs them about confidentiality and privacy, requests their voluntary participation, and provides a frequently asked questions link, a toll-free telephone number, and email address for questions about the survey. Participants provide consent by clicking a button at the bottom of the consent screen. We estimate the survey will take approximately 20 minutes to complete.
The project team anticipates working with participating SLTCOPs over a period of approximately three months, or possibly more. This schedule will vary by program. Factors affecting the timeline include the ability to identify and enlist facility administrators’ support for the data collection and recruiting participants for the study. The project team will also be talking with program staff during this time to understand the unique contexts in which programs operate.
Selecting Sites/Programs. The sampling plan begins with stratifying programs by the four program structure/organizational placement categories that characterize programs. Based on past research and findings from the Process Evaluation and Special Studies Related to the Long-Term Care Ombudsman Program, these characteristics have been identified as critical factors in affecting program operations. This includes:
1 centralized program outside of State Unit on Aging (SUA)
1 centralized program within SUA
1 decentralized program outside of SUA
1 decentralized program within SUA
State LTCOPs will be invited to participate in the study. Given that each structure/placement combination may have between 7 to 25 states, and only one program will be selected to represent each combination, opportunities for participation will vary. If more than one state is interested in participating within a given structure/placement group, one will be randomly selected. In each state, qualitative data will be collected from residents, family members, facility staff, and stakeholders through focus groups and interviews.
Facility Administrator Survey. In each state, a random sample of facility administrators of nursing homes and board and care homes in each of the four participating states will be invited to participate in the survey, collecting quantitative data. Although definitive numbers will not be available until programs have been selected, we estimate that between approximately 18 and 1,098 subjects will be invited to participate in each state to enroll a sample size ranging between 18 and 366. We base these estimates on the range of nursing homes (18 to 1,244) and board and care homes (40 to 7,406) that operated in states in 2018, using National Ombudsman Reporting System (NORS) data.
B.2.2. Estimation Procedure
In many research settings, a primary hypothesis is defined, that hypothesis is based on quantitative data, and sample size calculations are conducted to meet the specifications of that hypothesis. In the LTCOP evaluation, there is no single primary hypothesis, data collection consists of both qualitative and quantitative data, and in some cases, the goal of the research does not involve formal hypothesis testing. The evaluation is designed to answer multiple research questions using a variety of qualitative and quantitative research methods. To achieve this goal, our protocol calls for data collection from residents, family members, facility staff, stakeholders, and facility administrators. The rationale for these sample sizes is provided below for each respondent class.
Residents, Family Members, Facility Staff: Data collection from residents, family members, and facility staff is qualitative in nature and is not designed to generate population-based estimates, and it will not be used for hypothesis testing or inferential statistics. Individuals will be selected based on their familiarity with the Ombudsman program in order to respond to questions of interest for the LTCOP evaluation.
Stakeholders: Data collection from stakeholders is qualitative in nature and is not designed to generate population-based estimates, and it will not be used for hypothesis testing or inferential statistics. Stakeholders will be identified with the assistance of the State Ombudsman of each participating program. Those interviewed will offer sufficient level of information concerning the intra- and inter-agency relationships that are of interest for the LTCOP evaluation. As with residents, family members, and facility staff, these individuals will be selected in a purposive manner based on the needs of the project.
Facility Administrators: For each participating state, the project team will design a protocol in which we would identify a random sample of each state’s universe of nursing homes and board and care homes in a manner that would yield robust estimates of population parameters. To this end, we would use simple sample size calculation software to explore various scenarios concerning the sample size needed to generate population proportions from a universe consisting of between 18 to 1,244 nursing homes and 40 to 7,406 board and care homes. These calculations require specifying several parameters: a margin of error, confidence level, population size, and the likely proportion of the parameter to be estimated. Conventionally, conservative margins of error and confidence levels are specified at 5 percent and 95 percent, respectively and the population is defined at 18 to 1,244 nursing homes and 40 to 7,406 board and care homes. Given that we will be estimating many population proportions, there is no single parameter that is appropriate for sample size calculations. However, for the purposes of these calculations, we selected 50 percent, the most conservative proportion—the one that yielded the largest sample size—because this scenario would provide a sufficient sample size to meet all other proportions that we will encounter in the evaluation. With these parameters, the sample size needed to estimate a population proportion for program staff was between 18 and 366.
The contractor will collect data from 16 focus groups of residents, 8 focus groups of family members, 16 focus groups of facility staff, and 40 stakeholders, for a total of between 280 and 376 individuals. It should be noted that the size of focus groups for each respondent class is likely to vary, with the number of individuals ranging between 8 to 10 for residents and family members and 3 to 6 for facility staff.
Sample Size Needs. An important consideration prior to developing our sampling plan for selecting facility administrators was the number of nursing homes and board and care homes in each state to generate robust estimates for the research questions outlined in the outcome evaluation. Our calculations indicate that a sample size of between approximately 18 and 366 will allow us to generate accurate estimates of key population-level parameters as well as to conduct bivariate analyses linking predictors and outcomes. The sample will then be used to generate frequencies and means that are representative of the population. We also will perform statistical tests (t-test and chi-square) to identify significant relationships between facility characteristics and program outcome measures.
As noted above we anticipate a 40% percent response rate for nursing home administrators and 30% board and care home administrators for the web-based survey. For nursing home administrators, we consider this estimated response rate to be reasonable because the survey takes a short amount of time to complete (20 minutes based on a pretest), respondents are familiar with the use of email and the Web, and participation will be encouraged by program staff as well as their provider association. For board and care facilitators, we consider this estimate response rate to be reasonable because the survey takes a short amount of time to complete (20 minutes based on a pretest), participation will be encouraged by program staff as well as their provider association, and respondents may be less familiar with the program, given that Ombudsmen typically visit these care settings less frequently, based on 2018 NORS data. However, we are prepared to handle a lower-than-expected response rate. If a lower response rate occurs in either group, we will conduct non-response bias tests to determine if any bias may have been introduced into the sample.
There are no unusual problems requiring specialized sampling procedures.
There are no periodic data collection cycles associated with this study. The process evaluation is a one-time data collection.
Since the start of the evaluation planning process, a key objective has been to ensure internal and external support for the study to maximize participation in the evaluation. To optimize response rates for the qualitative data collection, the project team will (1) work closely with each SLTCOP to identify facilities for inclusions and encourage participation among respondent groups; and (2) provide monetary incentives to facility staff. Each strategy is discussed in more detail below.
Support of State Ombudsmen: Once SLTCOPs have been identified for inclusion in the study, the project team will work with the 4 State Ombudsmen to encourage facilities to participate in the data collection. The project team will ask State Ombudsmen (1) for assistance in identifying and contacting facility administrators to conduct focus groups at their sites, and (2) to communicate the importance of the project team’s data collection efforts.
After initial communications with State Ombudsman, the project team will follow-up with local staff if necessary to solidify the contact, begin relationship-building, explain the data collection activities, and provide local staff with an approximate date when the project team will reach out again to conduct the data collection. This initial contact will provide the project team with an opportunity to ask local program staff about how best to communicate with facility administrators, any issues that may impact the ability to collect data from sampled persons in each program, as well as any other considerations that may impact response rates. The project team will also share a one-page flyer to each local program to distribute to potential respondents to inform them of the upcoming data collection effort.
Monetary Incentive for Facility Staff: Among facility staff, a monetary incentive ($20 gift card) will be used to encourage participation.
Accommodations: The project team will accommodate respondents who require additional supports to participate in the study. We anticipate that participants may include individuals with a disability or whose primary language is not English.
To optimize response rates among facility administrators, two strategies will be used, including (1) enlisting the support of State Ombudsmen; and (2) enlisting the support of provider associations communicating the importance of the study to participants and ensuring ease of survey administration and completion. Each of these strategies is discussed in more detail below.
Support of State Ombudsmen: The project team will work with State Ombudsmen to optimize response rates among facility administrators. The project team will ask State Ombudsmen to help with outreach and to communicate the importance of the project team’s data collection efforts.
Support of State/Local Provider Associations: ACL/AoA and the project team will reach out to provider associations to assist in the evaluation’s data collection. Communications will emphasize the importance of the study and value to long-term care facilities. As part of these efforts, the project team may prepare answers to frequently asked questions (FAQs) and other information on the evaluation to support facility administrators’ understanding of the study and encourage their participation.
The focus group discussion guides, interview protocols, and survey instruments have been drafted and undergone three reviews: (1) an internal review conducted by NORC’s Institutional Review Board, (2) a review by ACL/AoA staff and NORC’s evaluation team partners, and (3) a pre-test with __ to assess the reliability of the instruments. The pretest was also administered to determine the burden placed on respondents. Slight revisions were made to the order and wording of a small number of questions based on comments received from these reviews.
Modifications to the content, structure, and length of the surveys have been made based on feedback received as well as results of survey pretest interviews. Respondents provided generally positive feedback indicating that they could readily answer the questions and that the time to complete the focus groups and surveys was not burdensome.
The outcome evaluation focus group discussion guides, interview protocols and web-survey instruments have been drafted, reviewed and approved by all project staff and have undergone an internal review conducted by NORC’s Institutional Review Board. After programming the instrument for the web, the survey instrument underwent beta testing by all project staff prior to the launch of the questionnaire.
The information for this study is being collected by NORC at the University of Chicago, an independent research organization, on behalf of ACL/AoA. With ACL/AoA’s oversight, the contractor is responsible for the study design, instrument development, data collection, analysis, and report preparation.
The instrument for this study and the plans for statistical analyses were developed by ACL/AoA and its contractor. The staff team is composed of Dr. Kim Nguyen, Project Director, Dr. Michael Yang, and consultant, Dr. Helaine Resnick. Contact information for these individuals is provided below.
Name |
Phone Number |
Kim Nguyen, PhD |
301-634-9495 |
Michael Yang, PhD |
301-634-9492 |
Helaine Resnick, PhD |
202-329-8616 |
Page
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | REQUEST FOR CLEARANCE FOR |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |