Previously Submitted Appendix_Information on the Full ISIS Sample Frame

Supporting Statement for OMB Clearance Request

Appendix M: Information on the Full ISIS Sample Frame

National Implementation Evaluation of the Health Profession Opportunity Grants (HPOG) to Serve TANF Recipients and Other Low-Income Individuals and HPOG Impact Study

0970-0394

April 24, 2013

Submitted by:

Office of Planning,
Research & Evaluation

Administration for Children & Families

U.S. Department of Health
and Human Services

Federal Project Officers:

Molly Irwin and Mary Mueggenborg

Appendix M: Information on the Full ISIS Sample Frame

Study Overview: Innovative Strategies for Increasing Self Sufficiency (ISIS)

March 2013

ACF conceived of the ISIS project as a test of promising interventions for improving the economic prospects of low-income individuals and families. After extensive outreach to the program and policy community (conducted under OMB clearance No. 0907-0343), ACF determined that the focus of ISIS would be programs that fit into the career pathways framework.

ISIS study sites target low-income adults who are interested in occupational skills training. The sites conduct random assignment of individuals to one of two groups: a treatment group that is offered the innovative career pathways interventions, or a control group that is able to access only a set of “business-as-usual” or other services in the community. The target sample size in eight of the nine sites is roughly 1,000—with about 500 in each of the two research groups (in one of these sites, the sample will come from three subsites). The ninth site has an estimated sample of 2,700 across eight subsites, with 1,800 in the treatment group and 900 in the control group.

The nine programs selected all promote completion of certificates and degrees in occupations in high demand and, to this end, incorporate multiple steps on the career ladder, with college credit or articulation agreements available for completers of the lower rungs. While varying in specific strategies and target populations, they all provide some level of the core career pathways services (assessment, instruction, supports and employment connections) although the emphasis placed on each varies by program.

Three of the nine sites—Pima County Community College, San Diego Workforce Partnership, and WDC of Seattle-King County—are also HPOG grantees. An additional ISIS site, Instituto del Progresso Latino, is a subgrantee to the Will County WIB HPOG grantee. Data collected by ISIS on participant impacts in these three sites will be included in the impact data collected by HPOG-Impact on the 20 HPOG sites participating in that study.

Research Questions

The impact evaluation of ISIS will address the following research question:

What are the net impacts of career pathway programs on educational outcomes (program completion, attainment of credentials and degrees) and economic outcomes (earnings, employment levels, and wage progression)?

The impact study will rely primarily on data from a 15-month follow-up survey, which includes questions that overlap with those in the HPOG-Impact follow-up survey, to address impact study research questions. Additionally, as with the HPOG-Impact Study, data on study participants’ wages will be collected from the National Directory of New Hires (NDNH). Additionally, the research team plans to use the National Student Clearinghouse (NSC) to gather information about college persistence and degree completion for study participants. NSC is currently the only data source that tracks postsecondary student enrollment across states.

ISIS Study Participants

The universe of potential respondents is low-income adults (age 18 or older) who are interested in occupational skills training and who reside in the geographical areas where ISIS sites are located. The target enrollment for the study is an average of 1,000 individuals in eight of the nine sites, including the three sites that are HPOG grantees and included in the HPOG-Impact analysis, and 2,700 in the remaining site, for a total of 10,700 individuals in the study as a whole.

As with HPOG-Impact, program staff recruit individuals, determine eligibility, and if the individual is eligible, obtain informed consent from those who volunteer to be in the study. The specific steps are as follows: Program staff inform eligible individuals about the study. Staff then administer the participant agreement form, which describes the study, and require individuals to sign the form if they wish to participate in the evaluation. Those who refuse to sign the consent form are not included in the study and are not eligible for the career pathways program. They will receive information about other services in the community.

For those who consent, program staff collect baseline data, which includes the Basic Information Form (BIF) and Self-Administered Questionnaire (SAQ). Program staff enter information from the BIF into a web-based system developed specifically for the evaluation. Staff then use the system to conduct random assignment to the treatment or control group. Those assigned to the treatment group are offered the provided services while those assigned to the control group are not able to participate in the program but can access other services in the community. Exhibit 1 summarizes the general process above.

Exhibit 1 ISIS Study Participant Recruitment and Random Assignment Process

A follow-up survey will be administered 15 months following enrollment in the study and random assignment, using procedures similar to those for the HPOG-Impact follow-up survey.