Justification

Volume I

National Household Education Surveys Program 2019 (NHES:2019) Focus Groups with Spanish Speakers for Development of Tailored Recruitment Materials

Round 2 – Recruitment

OMB #1850-0803 v. 229

April 2018

Attachments:

Attachment 1: Communication materials (Recruitment screener and consent form)

Attachments 2 and 3 will be submitted for OMB review as a separate submission by May 23, 2018:

Attachment 2: Study materials (Moderator’s guides, NHES letters for testing)

Attachment 3: Screener Survey (NHES screener survey to be tested)

Justification

The National Household Education Surveys Program (NHES) is conducted by the National Center for Education Statistics (NCES) and provides descriptive data on the educational activities of the U.S. population, with an emphasis on topics that are appropriate for household surveys rather than institutional surveys. NHES topics have covered a wide range of issues, including early childhood care and education, children’s readiness for school, parents’ perceptions of school safety and discipline, before- and after-school activities of school-age children, participation in adult and career education, parents’ involvement in their children’s education, school choice, homeschooling, and civic involvement. NHES uses a two-stage design in which sampled households complete a screener questionnaire to enumerate household members and their key characteristics. Within-household sampling from the screener data determines which household member is sampled for additional questions on the topical survey. NHES typically fields 2 to 3 topical surveys at a time, although the number has varied across its administrations. Surveys are administered in English and in Spanish.

Beginning in 1991, NHES was administered roughly every other year as a landline random-digit-dial (RDD) survey. During a period of declining response rates in all RDD surveys, NCES decided to conduct a series of field tests to determine if a change to self-administered mailed questionnaires would improve response rates. After a 5-year hiatus in data collection for this developmental work, NCES conducted the first full-scale mail-out administration with NHES:2012, which included the Early Childhood Program Participation (ECPP) and the Parent and Family Involvement in Education (PFI) surveys. The same two surveys, along with the Adult Training and Education Survey (ATES), were fielded in NHES:2016. In 2019, the NHES will field the PFI and ECPP surveys. This will be a two-stage study. In the first stage, households will be screened to determine if they contain eligible members. If eligible members are in the household, within-household sampling will be performed. Finally, topical surveys will be administered about the selected household members.

The ECPP, previously conducted in 1991, 1995, 2001, 2005, 2012 and 2016, surveys families of children ages 6 or younger who are not yet enrolled in kindergarten and provides estimates of children’s participation in care by relatives and non-relatives in private homes and in center-based daycare or preschool programs (including Head Start and Early Head Start). Additional topics addressed in ECPP interviews have included family learning activities; out-of-pocket expenses for non-parental care; continuity of care; factors related to parental selection of care; parents’ perceptions of care quality; child health and disability; and child, parent, and household characteristics.

The PFI, previously conducted in 1996, 2003, 2007, 2012, and 2016, surveys families of children and youth enrolled in kindergarten through 12^th grade or homeschooled for these grades, with an age limit of 20 years, and addresses specific ways that families are involved in their children’s school; school practices to involve and support families; involvement with children’s homework; and involvement in education activities outside of school. Parents of homeschoolers are asked about their reasons for choosing homeschooling and resources they used in homeschooling. Information about child, parent, and household characteristics is also collected.

The NHES redesign phase tested various design features related to increasing response to the screener and to second-stage surveys. For example, the NHES is administered by the U.S. Census Bureau, and the Census Bureau “branding” increased screener response rates by five percent in the 2011 NHES Field Test, leading NHES to adopt U.S. Census Bureau branding for all of its materials in 2012 and in 2016. Additionally, NHES experimented with prepaid incentive amounts for the screener and the second “topical” stage of the survey to find amounts that encouraged response without escalating collection costs.

In 2016, NHES continued to experiment with prepaid incentive amounts. In previous NHES collections, random samples were assigned to receive $0, $2, or $5 during the screener phase and screener response rates were compared. In 2016, NHES began experimenting with the use of a response propensity model to predict sample members’ likelihood of response. Applying the model allowed NHES to use $0 and $2 incentives (compared to $5 prepaid incentives) for addresses likely to respond, thus saving collection costs. For those addresses modeled to be unlikely to respond, NHES experimented with $10 prepaid incentives (compared to $5 prepaid incentives) but found that the higher prepaid incentive amount did not increase response enough to make it cost-effective.

The 2019 collection will again focus on attracting low-response propensity addresses to participate in the survey. For example, the 2019 collection will experiment with mode of administration to see how paper, web, and choice of paper or web perform across subgroups. It will also experiment with leveraging “tailored” or “targeted” materials that are designed to be attractive to a particular subgroup with historically low response rates. NHES already uses bilingual screener materials for addresses that are likely to have Spanish speakers, and Spanish topical surveys and letters for respondents who returned screener materials to the Census Bureau in Spanish. In 2019, NHES will build on this operation and will develop and send a set of materials that are designed to appeal specifically to the Spanish-speaking population, using unique messaging with tailored words, images, and contact strategies.

The first round of focus groups with Spanish-dominant respondents designed to better understand the barriers and benefits these respondents tend to associate with participation in surveys like NHES and to identify communication strategies that may help overcome those barriers to participation was conducted in April 2018 (OMB# 1850-0803 v.225). This request is to conduct the second round of focus groups with Spanish-dominant respondents to test materials and messaging developed from the first round. Spanish-dominant respondents share demographic characteristics with the occupants at addresses in previous NHES samples who responded at low rates. The focus groups will provide information for NCES to evaluate how well the newly adopted communication strategies targeting Spanish-dominant respondents help overcome barriers to participation in the NHES:2019 collection. The results will inform the tailored recruitment strategies and materials that will be used to encourage Spanish speakers’ participation in NHES:2019.

Design

We will conduct a total of three in-person focus groups in Spanish with Spanish-dominant adults from up to three market regions in three states (Denver, CO, San Antonio, TX, and Rockville, MD) to learn about their perceptions of revised language and visuals, such as images, that may be used in NHES contact materials (e.g. letters, envelopes, questionnaire covers). We will work with recruiters at local focus group facilities to select a sample of participants and ensure diversity of income and age of children. A recruitment screener (see attachment 1) will be used to determine eligibility of participants. We will address the following questions during the focus group:

All participants:

• Do the specific endorsements, colors, or photos used on the survey materials make participants in the focus groups feel included in the survey? Would they be likely to participate in the survey if they received these in the mail? What other reactions and perceptions do they have?

• Are there additional specific endorsements, colors, or photos for the front of the survey that would influence these household members toward participating?

Parent participants:

• What message do the materials convey to parents about what the survey is about? Do the materials help parents understand that it is important for them to complete and return the survey?

Non-parent participants:

• What message do the materials convey regarding what the survey is about?

• Do the materials help non-parents understand that it is important for them to complete and return the survey even though they do not have children?

Using the information collected from these conversations, we will incorporate the participants’ feedback into the development of materials and strategies for NHES recruitment. The following materials, which are drafts of materials planned for NHES:2019, will be presented to participants during the focus groups to elicit feedback and suggestions (attachments 2 and 3 will provide copies of all materials by May 23, 2018) after a more general guided discussion about survey participation.

Materials for participants’ review, to be discussed during the focus groups are under development at the time of this submission and will be provided for OMB’s review by May 23, 2018. These materials include:

• NHES advance letter/envelope—bilingual

• NHES screener cover letters (paper and web)—bilingual

• NHES screener survey—bilingual

• NHES second cover letters (web)—bilingual

• NHES third/FedEX cover letter (web)—bilingual

The focus groups will utilize draft NHES:2019 screener materials, and will not utilize topical materials. One reason for this decision is that NHES:2019 will use web as the primary mode of survey administration for most addresses, and the topical response rates in the web have been above 95 percent because the screener and topical surveys are administered in one stage online. The screener response rate, in contrast, was 66 percent in 2016. A second reason to focus on screener materials is that some participants in the focus groups will be non-parents, and the topical materials would not apply to these participants. A trained researcher working from a moderator’s guide (forthcoming in attachment 2-A by May 23, 2018) will conduct the focus groups.

Each focus group session will include 8-10 participants—a number that allows for in-depth collection of information. Topics of the discussion will focus on identifying the benefits participants associate with NHES participation, the barriers they perceive to participating, which NHES materials they believe would be most and least useful, and which elements of the NHES materials they like and dislike, and why. The focus group sessions will last approximately 90 minutes.

During the focus groups, using the moderator’s guide (forthcoming in attachment 2-A by May 23, 2018), a moderator will lead the participants through a discussion, and one to two Hager Sharp or American Institutes for Research team members will observe and take notes on the participants’ comments and suggestions, which they will later compile into a summary report. The session will also be video and audio recorded. The recordings will be destroyed as soon as the final report is approved, and no personally identifiable information will be included in the report. Following the focus groups, materials planned for the tailored materials experiment in NHES:2019 will be revised, if needed.

Recruiting and Paying Respondents

Hager Sharp will work with local focus group facilities in the Rockville, MD area (EurekaFacts), San Antonio, TX area, and the Denver, CO area (Fieldwork Denver) to recruit for and host the in-person focus groups. Focus group facility staff will be contacting people from their databases to look for respondents who meet the eligibility criteria. The recruitment screener in Attachment 1 will be administered to ensure that those contacted through databases meet eligibility criteria. No advertisements or email notices will be utilized.

The eligibility criteria for participating in these focus groups consist of:

• All participants must be Spanish-dominant and have not participated in Round 1 of these focus groups.

• Each parent focus group should include a balanced mix of parents of young children (ages 0-5) and parents of school-aged (K-12) children.

• One focus group will be with non-parents.

We will also gather information on age, income, and sex, and will attempt to recruit Spanish-dominant adults with a balance of ages and incomes. Furthermore, we will attempt to recruit both males and females. The proposed configuration of each focus group is as follows:

Market	Language	HH Income	Age of Children	Number of Participants
Rockville, MD	Spanish	Mixed, focus on low income	At least three parents of young children (ages 0-5); at least three parents of school-aged (K-12) children	Recruit 16 to have 8-10 show
Rockville, MD	Spanish	Mixed, focus on low income	No children	Recruit 16 to have 8-10 show
Denver, CO or San Antonio, TX	Spanish	Mixed, focus on low income	At least three parents of young children (ages 0-5); at least three parents of school-aged (K-12) children	Recruit 16 to have 8-10 show

To assure that we are able to recruit the select types of participants who are representative of typical Spanish-dominant NHES sample members and to thank them for their time and for completing the interview, during recruitment each participant will be offered a $75 incentive for participation in the focus group.

Assurance of Confidentiality

The statement below will be presented in all written materials (e.g., letters, emails) and read at the start of the focus group sessions. Participants will also be informed that they can leave the discussion at any time.

NCES is authorized to conduct this study by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C., § 9543). Your participation is voluntary and all of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

All contractor staff working on the NHES focus groups will sign the NCES Affidavit of Nondisclosure. Personal information (e.g., name, address) will be collected for recruitment purposes, but on the data file used for analyses, respondents will be identified only by a unique study ID number assigned to each participant. Within 48 hours of respondents’ participation in the focus groups, the discussion notes will be edited, organized, and cleaned, and all identifiers will be stripped from the data set. All computer files will be password-protected and hard copies will be locked in secure locations (e.g., data will be in locked file cabinets within locked offices). Only contract staff working directly on the data analysis portion of the project will have access to the data files. Once the final report is created, all personally identifiable information will be destroyed. All presentations of data in reports will be in aggregate form, with no links to individuals.

A consent form will be collected for all participants in the focus groups. The consent form is included in attachment 1.

Estimate of Hour Burden

Recruitment of participants is estimated to take about 10 minutes per participant, and we anticipate needing to contact up to 100 people to form the desired focus groups of a total of 24 participants. The focus groups will take approximately 90 minutes to conduct. There is no cost to participants beyond the participation burden time. Table 1 provides the burden estimates for this study.

Table 1. Burden estimates for NHES:2019 focus groups with Spanish-dominant adults

Respondent group	Number of respondents	Number of responses	Burden hours per respondent	Total burden hours
Participants – Recruitment	100	100	0.167	17
Participants ‒ Focus Group Participation	30*	30	1.5	45
Total Burden	100	130		62

*Subset of initial contact group, not double counted in the total number of respondents.

Estimate of Cost Burden

There is no direct cost to respondents.

Project Schedule

The schedule of activities for the NHES focus groups is provided in table 2.

Table 2. Schedule for the NHES:2019 focus groups with Spanish-dominant adults

Activity	Tasks	Date ranges
Data collection	Recruit participants	April 2018 (upon OMB approval)
	Conduct in-person focus groups	May/Early June 2018
Report-out meeting	Meet with NCES and AIR to discuss findings and recommendations	June 11, 2018
Report	Write report to document results and design decisions and include it in the NHES:2019 request 1850-(OMB# 1850-0768 v.14) submission to OMB	June 18, 2018
Submit revised materials to OMB for review with the NHES:2019 request 1850-(OMB# 1850-0768 v.14)	Finalize NHES:2019 materials	June 18, 2018

Cost to the Federal Government

The estimated cost to prepare for, administer, and report the results of the NHES focus groups round 2 is approximately $67,000. This cost includes salaried labor for contractor staff and other direct costs associated with organization of the sessions, incentives, materials design and revision for testing, and a final report. The cost for the recruitment portion of the NHES focus groups round 2 that is requested in this submission is $28,500.