Supporting Statement B
HRSA Health Center Workforce Survey: Pilot Survey
OMB Control No. 0915-0379
HRSA will not use statistical methods to select respondents for the pilot survey. Instead, the contractor will have health center liaisons participating in the previously conducted and approved learning collaborative, distribute an invitation to the pilot survey to staff in various roles. To ensure comprehensive coverage of various types of roles and health center representation, Uniform Data System (UDS) data and other publicly available information including health center characteristics and staff roles will be used to guide instructions to health center directors as to distribution of the invitations to participate in the pilot testing. All data collection activities will be administered in English. HRSA assumes that in excess of 95% of health center staff are able to communicate, written and orally, in English.
1. Respondent Universe and Sampling Methods
Recruitment
The respondent universe includes a broad range of HRSA-funded health center staff. Participants will represent a range of occupations within health centers, including medical; dental, vision, and pharmacy; behavioral health; clinical support and enabling staff; quality improvement, facility, and non-clinical support staff; and leadership. Participants will be recruited from health centers participating in the learning collaborative which represent a diverse group in location, setting, and size. The diversity of health centers and UDS major role categories will help capture a wide range of health centers and a wide range of staff during the pilot study.
Pilot Survey
HRSA and its contractor, John Snow Inc. (JSI), will build upon existing relationships with the 20 health centers participating in the learning collaborative plus additional health centers as needed to recruit individuals and reach the target number for the pilot survey. Large health centers will be asked to invite approximately 40 staff from various UDS role categories and small health centers will be asked to invite approximately 20 staff in various roles to complete the pilot survey. Approximately half of each of the large health centers and smaller health centers will be assigned to either Approach A or Approach B. A wide range of roles including staff from medical; dental, vision, and pharmacy; behavioral health; clinical support and enabling staff; quality improvement, facility, and non-clinical support staff; and leadership will be invited from each health center.
Under Approach A, JSI will provide a standard recruitment email for health center liaisons to distribute to staff which they invite for pilot survey participation. The email will include a description of the project and pilot survey, a link to the pilot survey that is specific to the health center but not to the individual employee, instructions for completing the survey, and an informed consent document. A respondent’s participation in the pilot survey serves as their agreement to the informed consent document. Under Approach A, the health center will be responsible for distribution and follow up of the survey link, and JSI will not know how many actual staff were invited. Furthermore, the survey will need to be completed in one sitting as participant identity (via IP address or cookie-based tracking) will not be tracked within Qualtrics.
Under Approach B, the health center director (or designee) will receive a link to a form allowing them to specify employees to be invited by entering email addresses for all eligible staff. These emails will then be entered into the Qualtrics system which will send out a similar invitation as in Approach A, however the invitation will come from HRSA (or JSI). The email invitation will make it clear that the contractor (JSI) (not HRSA or the health center) will track whether a survey response is received from each invitee and also that responses will not be associated with the email used for the invitation or otherwise connected to the identity of the respondent. After approximately 10 days and again at 20 days a reminder will be sent to those who have not yet completed the survey.
2. Procedures -Collection of Information
Pilot Survey
A learning collaborative liaison person will be contacted by JSI staff and they will be asked to select participants from each UDS major role category to ensure a diverse staff representation (20 to 40 depending on the size of the health center). JSI created an email invitation to be sent to selected staff asking them to aid HRSA by participating in a pilot survey to assess workforce well-being that is being developed to eventually survey all health center employees in the future. The email will explain that no one at HRSA or their health center will know which specific employees have filled out the survey.
Under Approach A the invitation email will be sent out by the health center which contains a link to the online survey. Only the health center liaison will know who were invited but even this person will not know who actually filled out the survey. However, the health center liaison will inform JSI of the number of invitees in each of the six general UDS occupation groups. For example, “x” number of dental staff, and “y” number of clinical support staff. The email invitation will contain a live link in it that will bring willing participants directly to the on-line survey using the approved Federal survey software, Qualtrics. In the settings for the Qualtrics program no information about the participants email address or name or IP address will be recorded or retained. In addition, nothing in the survey will specifically identify a participant, although there will be demographic questions and occupation related questions and the link will tell JSI which health center the response is coming from. JSI will track the number of invited staff and the number of completed surveys from each general occupation group from each health center and will assess response numbers after two weeks. If completed survey numbers are low, JSI will ask the health center liaison person to send out a reminder to all who were invited asking them to complete the survey if they have not already done so at approximately 10 days and again at 20 days after the initial invitation. Since no tracking of individual responding employees will be done, this protects their anonymity. In addition, no analyses of the pilot survey responses will be shared with HRSA in such a way that would potentially allow an individual participant to be identified.
Under Approach B, leadership at the health center will enter the emails for their nominated staff organized in each of the six general UDS occupation groups into an invitation system, which JSI will enter into the Qualtrics system to send out invitations. The link embedded in the email will be unique to the individual and the health center from which the employee was nominated. This will allow the respondent to return to a partially completed survey from any computer, and will permit JSI to conduct individual follow up to those that have not responded, as well as having a known universe of invitees. The system will track whether a completed survey has been received for each unique code and after 10 days a reminder email will be sent out to all those who have not yet completed the survey. If response is still low then potentially a second reminder will be sent out at approximately 20 days. When the data collection is completed and the data base extracted with employee answers, the individual email information will be deleted from the saved response data set, thereby at that point making individual employees anonymous in the final data.
The same pilot survey will used in approaches A and B and will be conducted on-line and take approximately 30 minutes each, as reported by participants during cognitive testing and the statistics available through Qualtrics. Qualtrics will be used as the pilot survey platform. No financial incentives will be given to individual participants.
For the pilot survey, (under both Approach A and B) all respondent answers will be kept anonymous. All raw data will only be available to the JSI project team. No individual health center responses will be returned to the health centers. It is possible that a general report on the outcome of the pilot survey process will be shared with the participating health centers. Reports made available to HRSA will consist of summary data in order to further protect individual responses and participant anonymity. In the eventual national rollout of the survey, individual health centers will receive reports (not a copy of the actual database for their employees) which will compare the average answers to the questions for their employees compared to national and regional averages. Feedback of average answers for subgroups of employees will be restricted to never provide information on groups of employees smaller than 15 persons.
3. Methods to Maximize Response Rates and Deal with Nonresponse
Response rates will be an outcome of both inviting staff that meet the mentioned criteria for participation and obtaining those employees’ cooperation and time to complete the pilot survey. The invitation emails from the health centers to their employees will be somewhat different for the pilot compared to the national rollout. For the pilot study, no data will be given back to individual health centers. We will describe for pilot participants how the eventual national data collection will provide results of the health center’s employees’ answers as group averages and that this information will be used to identify issues to be addressed locally for workforce quality improvement efforts and technical assistance provided by HRSA. In the actual national rollout, the invitations to employees to participate in the national study will describe how participation in the survey will benefit health center staff and the health center overall through the identification of areas of need within workforce well-being and satisfaction, and how technical assistance strategies to address these needs will not only improve workforce well-being among staff members but will also promote improved patient quality of care.
Those participating in the pilot survey will have about a month to complete the survey and can do so at a time and place that works best for their schedule. After two weeks, a reminder will be sent to all invitees to complete the survey within the next 10 days and again at 20 days after the initial invitation if they have not already done so.
4. Tests of Procedures or Methods to be Undertaken
Pilot survey data will be inspected to both further refine the questions and to ensure that the survey can be easily administered. To realize these goals, JSI will review the following indicators in the following paragraphs with a particular emphasis on comparing survey completion and response rates under Approach A versus Approach B.
In terms of learning whether any further modifications to the survey instrument itself are needed, the pilot testing will identify questions that respondents may decline to answer at unacceptably high rates; it will identify questions that do not produce acceptable levels of variation in answers; it will identify questions that cannot be answered by certain occupation groups and therefore require adding into the program skip logic; it will allow the opportunity to calculate reliability coefficients for intended scales to identify low reliability scales; it will identify questions that weaken the reliability of scales in this population; and it will help to identify questions that are unnecessarily redundant and therefore candidates for dropping from the national survey.
In terms of learning about administration issues that may need to be addressed, the pilot test will provide a measure of voluntary response rates in the absence of any explicit incentives; it will provide evidence of the proportion of survey participants who respond to a reminder message to complete the survey; it will provide a measure of the number of respondents who fail to complete the survey after beginning it; it will test employees abilities to use the web address of the Qualtrics survey software; it will provide an opportunity to test any problems with storing the data in the Cloud or downloading the information for analysis purposes; it will provide a measure of the average length of administration of the survey as well as lower and upper bounds of time to complete the questionnaire; and it will provide an opportunity to ask a few questions about participants reactions to the survey - including perceived length, perceived usefulness of the information, perceived willingness to participate in the future rollout of the survey and confidence in the privacy of their individual answers both in this pilot survey and in the eventual national rollout.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The following individuals will be consulted on data collection, quantitative analysis for the pilot survey.
Jayne Berube, MS, RD
Team Lead, Care Integration and Workforce Team
Bureau of Primary Health Care
Health Resources and Services Administration
jberube@hrsa.gov
Meresa Stacy
Project COR
Health Resources and Services Administration
MStacy@hrsa.gov
Larry
Horlamus, MS
Deputy
Director, Quality Division
Office
of Quality Improvement
Bureau
of Primary Health Care
HRSA
Thomas Mangione
Project Director, Senior Research Scientist
JSI Research & Training Institute, Inc.
tom_mangione@jsi.com
Ann Keehn
Project Manager, Senior Consultant
John Snow, Inc.
Laura Steere
Research Associate
JSI Research & Training Institute, Inc
Laura_steere@jsi.com
Tabeth Jiri
Senior Epidemiologist/Statistician
JSI Research & Training Institute, Inc.
Mihaly Imre
Data Analyst
JSI Research & Training Institute, Inc.
Eric Turer
Health Systems Consultant
John Snow, Inc.
Pamela Byrnes, PhD
Co-Learning Collaborative Lead
John Snow, Inc.
Pamela_Byrnes@jsi.com
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Laura Steere |
| File Modified | 0000-00-00 |
| File Created | 2023-12-13 |