Nonsubstantive Changes Memo_FaMLE

MEMORANDUM

TO: Steph Tathum; Office of Information and Regulatory Affairs (OIRA), Office of Management and Budget (OMB)

FROM: Seth Chamberlain; Office of Planning, Research and Evaluation (OPRE), Administration for Children and Families (ACF)

DATE: 02/23/2016

SUBJECT: Request for Non-substantive Changes to Fatherhood and Marriage Local Evaluation and Cross-Site Data Collection Instruments (OMB Approval Number 0970-0460)

The Office of Management and Budget (OMB) approved the initial information collection request for the Fatherhood and Marriage Local Evaluation (FaMLE) and cross-site data collection on 7/9/2015 (OMB Approval Number 0970-0460). OMB approved the collection and reporting of performance measures by the 2015 healthy marriage and responsible fatherhood (HMRF) grantees.

Since approval, we have identified the need for some minor changes and are submitting these revisions as a nonsubstantive change. In part A of this memo, we provide a brief background about the purpose of the performance measures. In part B, we describe the proposed changes, including the rationale. The proposed changes either reduce or have no effect on burden. None of the proposed changes increase burden for respondents.

A. Background

The FaMLE Cross-Site Project will gather information from the 90 grantees who received funding for HMRF programs authorized under Sec. 811 (b) Healthy Marriage Promotion and Promoting Responsible Fatherhood Grants of the Claims Resolution Act of 2010, Pub. L. No. 111-291, 124 Stat. 3064 (Dec. 8, 2010). The project is being undertaken by the U.S. Department of Health and Human Services, Administration for Children and Families (ACF), and is being implemented by Mathematica Policy Research.

The proposed non-substantive changes are to the Applicant Characteristics (Instrument DCS-1), the Self-Administered Questionnaires (SAQs) upon program entry (pre-test) and program exit (post-test)for RF programs (Instrument DCS-4RF_1, Instrument DCS-4RF_2, Instrument DCS-4RF_3, and Instrument DCS-4RF_4), Self-Administered Questionnaires (SAQs) upon program exit (post-test) for adults in HM programs (Instrument DCS-4HM_2), and the semi-annual program progress reports (PPRs) (Instrument DCS-5HM and Instrument DCS-5RF). These surveys and reports are described below, and in Section B of this memorandum we describe the proposed non-substantive changes to them.

B. Proposed Changes and Rationale

In this section, we identify each proposed non-substantive change to the performance measurement surveys and semi-annual progress reports. The first ten changes are to the client surveys and the last seven changes are to the PPRs.

1. Changes to Applicant Characteristics (Instrument DCS-1)

a. Items A3 and A5: Ask respondents about their ethnicity before asking about race.

Rationale: The Applicant Characteristics Survey originally asked applicants to identify their race followed by a question to identify their ethnicity. However, research shows that some Hispanic respondents will select “Other” as their race and enter in “Hispanic” when asked for their racial identity before their ethnicity (Bradburn, Sudman, & Wansink 2004). In contrast, asking about ethnic identity before racial identity reduces confusion that Hispanic respondents sometimes have with racial identity questions (Bradburn, Sudman, & Wansink 2004). Therefore, we propose changing the order of the race and ethnicity questions on the Applicant Characteristics survey to ask for each applicant’s ethnicity before their racial identity.

Please see Instrument DCS-1, items A3 and A5, which illustrate this change.

b. Items B1, B6b, B7, C1-C3, C6-C8, E1-E2: Change skip logic in Applicant Characteristics survey for youth respondents to skip social assistance, employment, and relationship status questions; the section about where the applicant learned about the program; and why they chose to enroll.

Rationale: We propose further refinement of the survey skip logic to tailor the Applicant Characteristics Survey to the youth populations served by HM programs. In particular, we propose that applicants under age 18 skip questions about receiving specific types of assistance (e.g., TANF, child support) because they are unlikely to be the beneficiary of the types of assistance applicants are asked about in the question.

We also propose that applicants who are in high school skip questions about their employment and marital/relationship status. Youth in high schools are unlikely to be in long-term romantic relationships and less likely to be employed. Moreover, the outcomes of interest for HM programs serving youth focus on attitudes and dating behavior rather than long-term relationship status and employment. Thus, we propose having youth applicants skip the questions in these domains.

Additionally, we propose that youth applicants who are currently in high school skip the question that asks applicants how they learned about the program and why they chose to enroll in it. Youth applicants currently in high school are likely to be part of HM relationship education programs provided in high schools; as a result, there is likely to be little meaningful variation in how the applicants heard about the program or why they enrolled. Therefore, we also propose removing the option, “School staff, such as a teacher or counselor” from question E1 because this option will not apply to applicants in other HM and RF programs.

Please see Instrument DCS-1, items B1, B6b, B7, C1-C3, C6-C8, E1-E2, which illustrate these changes.

c. Items B3 and B6bd: Remove redundant education status question from the Applicant Characteristics survey.

Rationale: Originally, the Applicant Characteristics survey included two questions asking applicants to identify whether they are in school or college:

B3. Are you currently in school or college?

B6bd. Are you in school or college full or part time?

We propose deleting item B6bd from the survey to remove the redundancy.

Please see Instrument DCS-1, items B3 and B6bd, which illustrate this change.

d. Item C4b: Change wording of question asking male respondents about pregnancy status on Applicant Characteristics survey.

Rationale: We propose revising the wording of item C4b from, “Are any women currently pregnant with your child?” to “Is anyone currently pregnant with your child?” Youth in high school may not refer to their partners as “women,” a term which may be reserved for adults or people older than the respondents. The wording should not adversely affect any respondents’ understanding of the question.

Please see Instrument DCS-1, item C4b, which illustrates this change.

e. Item C5/C5a: Tailor question wording to youth respondents.

Rationale: For applicants currently in high school, we propose removing the phrase, “who are under 21 years old,” from the question about how many children applicants have (“How many children do you have who are under 21 years old? Do not include current pregnancies.”). Applicants currently in high school will themselves be under 21, so their children by default will be as well. This question wording change makes the item more applicable to youth applicants. The question tailored to youth applicants will appear as item C5a in the Applicant Characteristics survey. The web survey skip logic will be programmed so that this version of the question is only asked of applicants who report being in high school. The original item, C5, which includes the phrase, “who are under 21 years old,” will be asked only of those applicants not currently in high school.

Please see Instrument DCS-1, item C5/C5a, which illustrates this change.

f. Item C9: Change language about foster care status to capture whether the participant is or has ever been in foster care, and if so how long ago they left foster care.

Rationale: The original language for the response categories to this question (What is your current foster care status?) used the term “transition,” which is an official legal term relating to foster care. We are simply interested in knowing if and when a participant has ever been in foster care, so we propose removing the term “transition” and adding one additional response category to capture all possible foster care statuses.

The original response categories were:

I have never been in foster care

I recently (in last the past 6 months) transitioned out of foster care

I am preparing to transition out of foster care

I am currently in foster care, with no current transition plans in place

The new proposed response categories are:

I have never been in foster care  

I left foster care over six months ago

I recently (in the past 6 months) left foster care

I am currently in foster care

Not sure

Consequently, we propose changing the categories for this outcome in section C-02.3 of the HM PPR. Please see Instrument DCS-1, item C9, and Instrument DCS-5HM which illustrate these changes.

g. Item D1: Reverse the order of self-rated health response options.

Rationale: The Applicant Characteristics Survey originally asked applicants to rate their general health on a scale from “poor” to “excellent”. Methodological research, though, shows that Hispanic individuals tend to rate their health as poorer when the “poor” category is presented first in the list of options, whereas the order of response options have been found to have no significant effect on how other racial and ethnic groups rate their own health (Lee & Schwarz 2014; Shetterly, et al. 1996). Thus, to mitigate this measurement effect, we propose reversing the rating scale to present the “excellent” option first in the list of categories. The change will improve measurement for Hispanic applicants and should have no effect on applicants from other racial and ethnic groups. This ordering will also enhance the comparability of the question to other large health surveys, such as the Behavioral Risk Factor Surveillance System (BRFSS) and the National Health Interview Survey (NHIS).

Please see Instrument DCS-1, item D1, which illustrates this change.

2. Changes to Self-Administered Questionnaires (SAQs) upon program entry (pre-test) and program exit (post-test) (Instrument DCS-4RF_1, Instrument DCS-4RF_2, Instrument DCS-4RF_3, and Instrument DCS-4RF_4, and Instrument DCS-4HM_2)

a. Items A3, A4, A5, A10, A11, and A12 (Community Fathers Entrance and Exit Surveys): Change skip logic so that only fathers who have seen their children in the past month are asked about their interactions with their children.

Rationale: Originally, all fathers with children under 21 years old and who had a parenting agreement with the child’s (children’s) mother were asked how often they had specific feelings about or interactions with their child(ren) in the past month. We propose editing the skip logic for these questions so that fathers who have not seen their children in the past month are not asked these items. Changing the skip logic to be based on the amount of contact the client has with his children make these question consistent with the same questions asked of incarcerated fathers.

Please see Instruments DCS-4RF_1 and DCS-4RF_2, items A3, A4, A5, A10, A11, and A12 which illustrate these changes.

b. Items B6 and B7 (Community Fathers Entrance Survey); B9 and B10 (Community Fathers Exit Survey); and B5 and B6 (Incarcerated Fathers Entrance and Exit Surveys): Change skip logic so that only fathers with children under the age of 21 years are asked about legal arrangements or child support orders to provide financial support to their children.

Rationale: We propose changing the skip logic so that fathers who do not have children under the age of 21 years are not asked about having legal arrangements or child support orders to provide financial support for any of their children. Currently, all RF clients are asked whether they have such an arrangement or child support order and whether they know that they can request a change to it if their income changes. Because these questions do not apply to fathers without children under the age of 21 years, we suggest such clients skip these questions.

Please see Instrument DCS-4RF_1 and C, items B6 and B7, and Instruments DCS-4RF_3 and DCS-4RF_4, items B5 and B6, which illustrate these changes.

c. Item E3 and E4 (HM Adults Exit Survey): Fix question numbering typos.

Rationale: The exit survey for adults in HM programs originally had two items numbered as E3. We propose fixing this typo by renumbering the second as E4, and changing the last item on the survey to E5.

Please see Instrument DCS-4HM_2, items E3 and E4, which illustrate these changes.

3. Changes to Performance Progress Reports (PPRs) (Instrument DCS-5HM and Instrument DCS-5RF)

a. Section B-02: Change the order of the bullet point instructions for Program Enrollment in Section B-02 of the PPR (HM and RF).

Rationale: We propose reordering the bullet point instructions in Section B-02 of the PPRs to emphasize that grantees should provide a narrative about the degree to which they are enrolling their target population. The two other bullet points are follow-up items instructing the grantees to describe why they are or are not meeting their enrollment targets. Reordering the bullet points will improve the response flow for grantees.

Please see Instruments DCS-5HM and DCS-5RF, Section B-02, which illustrate this change.

b. Section C-02.1.4, Applicant Ethnicity: Remove “or of Spanish origin” from labeling of categories on the HM and RF PPRs.

Rationale: To make the PPR’s wording of ethnicity categories consistent with the wording of the Applicant Characteristics survey, we propose removing the phrase “or of Spanish origin” from the PPR tables that report the characteristics of a grantee’s applicants.

Please see Instrument DCS-5HM and Instrument DCS-5RF, Section C-02, which illustrates this change.

c. Section C-02 and C-09: Add Entrance Survey Outcomes to the PPR (HM and RF).

Rationale: Originally, the PPR only reported outcomes measured at program exit. We propose adding a subset of the same outcomes from the client surveys administered at program entry. The outcomes from the client surveys administered at program entry will auto-populate into the PPR. Specifically, we propose that outcomes from the program entrance surveys be reported in the PPR under new subsections within Section C-02:

• “Characteristics, at Beginning of Program, of Adult Couple and Individual Clients” and “Characteristics, at Beginning of Program, of Youth Clients” for the HM PPR and

• “Characteristics, at Beginning of Program, of Community Individual and Couple Clients” and “Characteristics, at Beginning of Program, of Incarcerated Clients” for the RF PPR.

For consistency, we propose renaming Section C-09 of the PPR which reports outcomes measured at program exit with “Characteristics, at Program Exit, of Adult Couple and Individual Clients” and “Characteristics, at Program Exit, of Youth Clients” for the HM PPR, and “Characteristics, at Program Exit, of Community Individual and Couple Clients” and “Characteristics, at Program Exit, of Incarcerated Clients” for the RF PPR.

Please see Instrument DCS-5HM and Instrument DCS-5RF, sections C-02 and C-09, which illustrate these changes.

d. Section C-03.1, Screening for Intimate Partner Violence: Change the metric about intimate partner violence screening from a survey question (based on a staff estimate) to a calculation (derived from data in the Management Information System) on the PPR (HM and RF).

Rationale: To increase accuracy of reporting on IPV screening, we propose to change the PPR to include an auto-populated summary calculation of how many clients were screened for IPV as reported in nFORM. Originally, this information was collected in Program Operations, Instrument DCS-2, that asked grantees to estimate the proportion of applicants that were screened for IPV. Including the actual calculation from the nFORM data will be more precise for reporting the number and percentage of applicants that were screened for IPV. Therefore, we propose removing question C5 from Program Operations Instrument DCS-2.

Please see Instruments DCS-5HM and DCS-5RF, Section C-03, and Instrument DCS-2, which illustrate this change.

e. Section C-04.2 Attendance at Workshops: Change the calculation of workshop attendance on PPR (HM and RF) from the percentage of sessions to the percentage of total hours of the workshop.

Rationale: The original PPR calculated client attendance at workshops as the percentage of sessions attended. We propose, instead, to calculate workshop attendance as the percentage of hours of workshop curriculum to be consistent with the way workshop curriculum is described in the nFORM system and to facilitate calculation of individually made-up workshop sessions.

Please see Instruments DCS-5HM and DCS-5RF, Section C-04.2, which illustrate this change.

f. Sections C-09.1.D and C-09.2.D (RF PPR), Awareness of Sources of Social Support: Reword category to be consistent with Exit Survey question.

Rationale: To make the PPR’s wording consistent with the question wording of the RF Exit Surveys, we propose changing the category wording from “Knows where to go for help if had money troubles” to “Doesn’t know where to go for help if had money troubles”. The item on the RF Exit Surveys asks clients how much they agree or disagree with the statement, “I wouldn’t know where to go for help if I had money troubles.”

Please see Instrument DCS-5RF, Section C-09, which illustrates this change.

g. RF PPR Order of Column Headings: Order Column Headings of RF Populations Consistently.

Rationale: For tables with data by RF populations, we propose changing the column headings of tables in the RF PPR to consistently be ordered left to right as Community Individuals, Community Couples, and Incarcerated Individuals. Currently, the order of column headings is inconsistent, which may be confusing when reviewing the data reported in the RF PPR.

Please see Instrument DCS-5RF, which illustrates this change.

An Affirmative Action/Equal Opportunity Employer