HPOG-NIE Follow-Up Phone Call Protocol for the Stakeholder/Network survey

Supporting Statement for OMB Clearance Request

Appendix C: HPOG-NIE Follow-Up Telephone Call Protocol for the Stakeholder/Network Survey

National Implementation Evaluation of the Health Profession Opportunity Grants (HPOG) to Serve TANF Recipients and Other Low-Income Individuals and HPOG Impact Study

0970-0394

April 19, 2013

Revised July 5, 2013

Submitted by:

Office of Planning,
Research & Evaluation

Administration for Children & Families

U.S. Department of Health
and Human Services

Federal Project Officers:

Molly Irwin and Mary Mueggenborg

Appendix C: HPOG-NIE Follow-Up Telephone Call Protocol for the Stakeholder/Network Survey

Note to reviewers

The following protocol will be online, with callers inputting responses to the questions to update an online Sampling Form for Stakeholder/Network survey.

• Instructions to the caller are in red font

• Background and text for caller is in standard black font.

• Prefilled text from previous responses is denoted in green font

• Programming text is in blue font.

Follow-Up Telephone Call Protocol for the Stakeholder/Network survey

The following is a protocol for Abt Associates and Urban Institute researchers charged with making follow-up telephone calls of organizations and contacts that the Study Liaisons of each HPOG grantee or local HPOG program have identified for the HPOG-NIE Stakeholder/Network survey. These calls are intended to identify additional organizations and internal contacts to complete this survey. The calls can also help confirm the appropriateness of the organizations and internal contacts proposed by the Study Liaisons for this survey in the sampling questionnaire, and the accuracy of the provided contact information.

Background

Each HPOG grantee is implementing its HPOG grant together with a broad network of local HPOG stakeholders. These are organizations with varying levels of interest in, and involvement with, the HPOG program and its efforts to help low-income individuals to secure healthcare training and employment. The Stakeholder/Network survey seeks the following information about these organizations:

• Information about the roles and responsibilities of organizations that have assisted [name of grantee institution] in the design and implementation of [name of local HPOG program], either with a formal contract or memorandum of understanding (MOU), or on an informal basis. (Examples of activities and services are provided later in the protocol.)

• Perceptions that these and other stakeholder organizations have about the effectiveness of [name of local HPOG program] (e.g., “To what extent do you agree with each of the following statements about the effectiveness of [name of local HPOG program] in accomplishing the following goals? a. Meeting area healthcare needs, b. Developing career ladders for HPOG participants, etc.”)

• Perceptions of the effectiveness and sustainability of the collaboration on behalf of HPOG beyond the grant period (e.g., “Which of the following represent challenges to the sustainability/future of HPOG-related activities after the end of the HPOG grant? a. Unfavorable economic conditions, b. Excess of labor supply (e.g., too many new low- to mid-skilled healthcare graduates), etc.”)

For the Stakeholder/Network survey, the Study Liaisons, appointed by each HPOG grantee, were asked to complete a “Sampling Form for Stakeholder/Network survey,” with a list of organizations or agencies that have been involved with [name of local HPOG program] at some point over the entire program, starting with the design or early implementation phases.

• You are asked to make telephone calls to each of the stakeholder organizations and internal contacts recommended by the Study Liaison for the Stakeholder/Network survey, with the purpose of soliciting additional organizations and internal contacts to respond to the survey.

• Please follow the suggested script starting with Part A and continuing with Part B, following the prompts for inputting responses to your questions.

[INFORMATION COLLECTED ONLINE BY THE TELEPHONE CALLS WILL POPULATE THE SAMPLE FORM FOR THE STAKEHOLDER/NETWORK SURVEY – SEE NEXT – THAT THE STUDY LIAISONS HAVE BEEN COMPLETING. THE EXCEPTION IS THE COLUMN ASKING ABOUT WHETHER OR NOT THE STAKEHOLDER HAS A FORMAL CONTRACT OR AGREEMENT WITH THE HPOG GRANTEE OR PROGRAM; THE TELEPHONE RESPONDENTS MAY LIKELY NOT HAVE THE INFORMATION TO ANSWER THIS, AND THE SURVEY ASKS ABOUT IT ANYWAY].

Sampling Form for Stakeholder/Network survey

Organization	Contact Person	Phone number	Email	Contract/MOU with [name of local HPOG program]?
				Yes	No

• This protocol and its questions can also be used to follow up with the additional contacts you have generated from your calls, to solicit other organizations and internal contacts to complete the survey.

• Please use the suggested script in Part C to contact other organizations or individuals your calls have generated, following the prompts for inputting responses to your questions. You are asked to do so until the individuals you call repeatedly suggest the same organizations and internal contacts.

Suggested script for the telephone calls

[IF AT ANY POINT DURING THE CALL THE INDIVIDUAL SAYS HE/SHE DOES NOT HAVE TIME AND TRIES TO HANG UP, ASK HIM/HER THE FOLLOWING: “IF THIS IS A BAD TIME, CAN YOU TELL ME ANOTHER DATE AND TIME TO CALL YOU? IT SHOULD TAKE NO MORE THAN FIVE OR TEN MINUTES.”]

Part A: The First Stakeholder Suggested by Study Liaison

Good morning/afternoon/evening, Mr./Ms. [name from Sampling Questionnaire’s list of potential Stakeholder/Network survey respondents]:

I am calling on behalf of [Abt Associates/the Urban Institute]. You were recommended to us by [name of Study Liaison] as someone who might be able to help us learn more about how [name of local HPOG program] is working with area organizations to provide and expand healthcare education and training opportunities for low-income persons. We have been asked by the Administration of Children and Families, of the U.S. Department of Health and Human Services, to evaluate [name of local HPOG program] as part of an important national study. We will be conducting an online survey of local organizations like yours that help [name of local HPOG program] to prepare low-income people for healthcare careers. We have spoken with [name of local HPOG program] and collected a list of local partner and stakeholder organizations that are involved in the program. We are interested in identifying any additional local organizations involved, and we would like your perspective. This telephone call should take no more than ten minutes of your time.

According to the Paperwork Reduction Act (PRA), an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0394 and it expires xx/xx/xxxx. If you have comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, please send them to [Contact Name]; [Contact Address]; Attn: OMB-PRA (0970-0397).

1. Are you aware that [name of grantee institution] is running a program called [name of local HPOG program] that is funded by a grant from the U.S. Department of Health and Human Services’ Administration for Children and Families?

• Yes

• No

[WHETHER YES OR NO TO 1, KEEP GOING WITH SCRIPT BECAUSE HE/SHE MAY KNOW SOMEONE AT HIS/HER ORGANIZATION WHO CAN RESPOND TO THE STAKEHOLDER/NETWORK SURVEY]

2. Is there anyone else at your organization who is able to provide us with information about your organization’s work with [name of local HPOG program]?

• Yes

• No

[IF YES, CONTINUE TO 3. ELSE, SKIP TO 6].

3. Please tell me the first and last name of this individual.

First Name __________________________________

Last Name __________________________________

4. What is their email address?

5. What is their phone number?

6. I am going to read you a list of organizations that [name of local HPOG program] identified already:

[INSERT LIST OF ORGANIZATIONS FROM SAMPLING SURVEY]

Are you aware of any other organizations that helped plan [name of local HPOG program] or that work with [name of local HPOG program], on marketing and outreach, curriculum development, vocational or occupational training, pre-training activities, basic academic skills education, counseling and support services, job development, job placement, or recruitment or hiring of graduates?*

• Yes,

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

[*NOTE: Definitions for these terms if prompted by the respondent:

• Marketing and outreach (e.g., printed materials available on-site, information available on partner’s website, mentions during presentations to stakeholders, mentions during orientation for organization’s services, mentions during assessment and counseling session)

• Curriculum development (e.g., offered examples of relevant curricula, provided feedback on draft curricula, wrote modules for curriculum)

• Vocational or occupational training (e.g., operation of training program, provision of faculty/instructors, provision of training space, provision of equipment, provision of learning technologies, provision of work-based learning opportunities—e.g., internships, clinicals)

• Pre-training activities (e.g., Prior to training, provision of workshops on healthcare occupations and educational requirements, reading or math refresher courses, computer skills; and/or provision of pre-training faculty/instructors, training space, equipment, and/or learning technologies)

• Basic academic skills education (e.g., education for foundational math, reading, and writing skills, such as General Equivalency Degree (GED) classes, pre-GED Classes, English as a Second Language (ESL) instruction, adult basic education)

• Counseling and support services (e.g., academic supports and counseling, personal supports and counseling, financial supports, other social supports)

• Job development activities (e.g., job readiness workshops, job search skills training, individual job search assistance, job coach navigator, group job search support, post-placement and retention support)

• Job placement activities (e.g., obtained and screened job listings for HPOG participants, screened HPOG participants for suitability for a position, scheduled interviews for a job candidate, provided interview space)

• Recruitment or hiring of [name of local HPOG program] graduates (e.g., guaranteed interviews for successful graduates or soon-to-be graduates, placed job listings with HPOG program, placed direct call(s) to HPOG program manager or other contact to learn about potential candidates)]

[IF YES, CONTINUE TO 7. IF NO, GO TO 9.]

7. Let me repeat that back: [INSERT LIST OF ORGANIZATIONS RESPONDENT PROVIDED IN 6]

For each of the organizations you listed, can you recommend anyone who can provide us with information about their work with [name of local HPOG program]?

• Yes,

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

[IF YES, CONTINUE TO 8. IF NO, GO TO 9.]

8. Do you have their email address or telephone number?

• Yes,

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

9. Are you aware of any other organizations that have helped plan [name of local HPOG program] or that work with [name of local HPOG program], as I described earlier, in recruiting, training, or providing case management or support services to low-income individuals for healthcare careers?

• Yes,

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

[IF YES, CONTINUE TO 10. IF NO, GO TO 12.]

10. Let me repeat that back: [INSERT LIST OF NEW ORGANIZATIONS RESPONDENT PROVIDED IN 9]

Can you recommend anyone at these organizations who can provide us with information about their work with [name of local HPOG program]?

• Yes

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

[IF YES, CONTINUE TO 11. IF NO, GO TO 12]

11. Do you have their email address or telephone number?

• Yes,

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

12. Finally, are there any organizations that have not been directly involved in the design or operation of [name of local HPOG program], but are aware of it or the HPOG grant, share the same constituencies or objectives, and may benefit from the program’s success? This might include community advocacy groups, labor unions, other community colleges or training providers, policymakers or their offices, local economic development agencies, or local healthcare industry professional groups.

• Yes,

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

[IF YES, CONTINUE TO 13. IF NO, GO TO 15.]

13. Let me repeat that back: [INSERT LIST OF NEW ORGANIZATIONS RESPONDENT PROVIDED IN 12]

Can you recommend anyone at these organizations who can provide us with information about their organization’s interest in [name of local HPOG program]?

• Yes

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

[IF YES, CONTINUE TO 14. IF NO, GO TO 15]

14. Do you have their email address or telephone number?

• Yes,

__________________________________________

__________________________________________

__________________________________________

__________________________________________

• No

15. In closing, do you have any questions for the HPOG research team? [ENTER COMMENTS BELOW.]

[Textbox – 1,000 character limit]

Thank you for your assistance with this important study.

Part B: Additional Stakeholders Proposed by Study Liaisons

[BEGIN CALLING THE NEXT STAKEHOLDER ORGANIZATION AND ITS INTERNAL CONTACT PERSON(S) PROVIDED BY THE STUDY LIAISON IN THE SAMPLING FORM, REPEATING 1 THROUGH 15, UNTIL THE ENTIRE LIST OF STAKEHOLDERS PROVIDED BY THE STUDY LIAISON HAS BEEN CALLED. THEN GO ONTO PART C].

Part C: Additional Stakeholders Generated by Telephone Calls

[BEGIN CALLING THE STAKEHOLDER ORGANIZATIONS AND THEIR INTERNAL CONTACT PERSON(S) THAT YOU HAVE GENERATED THROUGH THESE TELEPHONE CALLS, REPEATING 1 THROUGH 15, UNTIL YOUR LIST HAS BEEN EXHAUSTED AND/OR YOU ARE BEGINNING TO RECEIVE THE SAME SUGGESTED ORGANIZATIONS AND CONTACT PEOPLE.].

The Paperwork Reduction Act Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is xxxx-xxxx, and it expires xx/xx/xxxx. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to [Contact Name]; [Contact Address]; Attn: OMB-PRA (xxxx-xxxx).