DRAFT HPOG 2.0 Memo to OMB_Post-Training Changes to Instrument 12_09272018

DATE: September 27, 2018

TO: Steph Tatham

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)

FROM: Nicole Constance and Hilary Forster

Office of Planning, Research, and Evaluation (OPRE)

Administration for Children and Families (ACF)

SUBJECT: Request for Non-Substantive Change to Health Profession Opportunity Grants (HPOG) Second Generation National and Tribal Evaluation (OMB Control Number 0970-0462)

This memo requests approval for non-substantive changes to the Short-term (15-month) Follow-up Survey for the Second Generation Health Profession Opportunity Grants (HPOG 2.0) National Evaluation’s impact study (Instrument #12 approved under OMB Control Number 0970-0462). First, it seeks approval for revisions to clarify some of the questions and improve the clarity of data collected. Second it seeks approval for the addition of one follow-up question to improve the quality of the respondent contact information. OMB recently approved nonsubstantive changes to Instrument 12; these additional revisions are based on questions that arose during the survey interviewer training conducted in mid-September 2018.

Non-Substantive Change Request #1—Short-term (15-month) Follow-up Survey

The Short-term Follow-up Survey captures data on the following outcomes of interest to the HPOG 2.0 impact study:

• Section A: training experiences and employment history from the point of random assignment through the interview date.

• Section B: school experiences, containing a loop for each school attended since random assignment.

• Section C: credential attainment

• Section D: conditions of current or most recent employment.

• Section E: household composition.

• Section F: income and financial well-being.

• Section G: 21^st century and cognitive skills.

• Section H: respondent and secondary contact information.

The survey was approved by OMB under OMB Control # 0970-0462 on June 8, 2018, with non-substantive changes approved on September 18, 2018.

The study team conducted interviewer training in mid-September 2018. During that training, interviewers asked for clarification on some of the survey items. The evaluation team determined that minor wording changes in a small number of questions would help clarify the intent of the question for respondents and ultimately improve the quality of the data collected.

The requested changes are summarized below. The exact wording changes are reflected in the revised survey instrument attached: Instrument 12 HPOG2.0 Short-Term15-Month Follow-up Survey_REV09262018_Trackchanges.

• A7a, A7b, A12a, and A12b: These questions ask about the number of preschool and school-aged children who were ever in child care between random assignment and the interview date (A7a and A7b) and the number of hours the children in each age group spent in child care (A12a and A12b) for each job or school spell. During interviewer training, several interviewers asked whether respondents should include child care provided by family members or friends or shared child care arrangements in their response. The questions are intended to capture paid child care situations only. We seek approval to include language to clarify that the child care should be paid.

.

• B2a and B2b: We seek approval to add text to clarify that these questions are specifically about the basic education classes the respondent took while attending a particular school rather than to all classes they took while attending that school.

• B5: There were questions from the interviewers about whether respondents should report the total costs per semester (either at time of interview or at enrollment); or per course (either at time of interview or at enrollment); or the total amount spent to attend the school for that spell. The intent is to measure the respondent’s total costs of attendance at that school for that spell. We seek approval to clarify that in the question text.

• F2 and F3: During training, interviewers asked whether personal income (F2) and household income (F3) should be answered before or after taxes. These questions are intended to capture income before taxes and any other deductions. We seek approval to add text clarifying this for respondents.

• F4: During training we noted that the amount of the income cap to qualify for the Earned Income Tax Credit (EITC) varies based on marital status and the number of dependents. We seek approval to make the language about the income cap more general. The current question states that EITC is “for people making less than $49,000 per year.” We seek approval to change the wording to “those that make less than a certain amount each year.” 

Expected Benefits

We expect these requested changes will improve the instrument flow and make administration easier for the interviewer and the respondent. Clearly worded questions are easier to comprehend and improve both the interviewer’s ability to administer the questions and the respondent’s ability to answer them. If the respondent is clear on the intent of the question, they are able to provide more accurate responses. This is particularly important for the questions that are the subject of this nonsubstantive change request as most are related to financial information.

Non-Substantive Change Request #2: Additional Question for Short-term (15-month) Follow-up Survey

We identified a critical data item that was omitted from the respondent contact information section (Section H) of the originally approved instrument. The evaluation team has collected permission to text participants at the time of enrollment under PAGES Participant Level Baseline Data Collection (Instrument #1, approved in August 2015), and as part of the contact update form (Instrument 5b, approved in June 2017). However, the question to confirm that the evaluation team has the respondent’s permission to contact them via text message was omitted from the Short-term Follow-up Survey (Instrument #12, approved in July 2018). Although most of the sample (91 percent) provided permission to contact them via text, there is a small group of study participants who did not previously provide us with permission to contact them via text. If the evaluation team reaches a participant for whom we do not already have permission to text and who provide a cell phone number in either question H3, H4, or H5 we would like to ask the same question we ask in Instrument 5b, the contact update form:

“Do we have your permission to contact you via text message to your cell phone? This could be regular text or automated text.” 

The requested additions are include in the attached revised: Instrument 12 HPOG2.0 Short-Term15-Month Follow-up Survey_ REV09262018_Trackchanges

Expected Benefits

This additional question will add a critical component for the evaluation team’s ongoing participant tracking and data collection efforts. We believe that the burden associated with the addition of this question is very small as 91 percent of the sample already gave their permission to receive text messages. This will only be asked of the remaining nine percent of the sample, if they are interviewed and if they provide a cell phone number.

2