NHES 2019 Types of Schooling Cognitive Interviews - Revision 2 Memo

MEMORANDUM OMB # 1850-0803 v.216

DATE: November 2, 2017

TO: Robert Sivinski, Office of Management and Budget

THROUGH: Kashka Kubzdela, National Center for Education Statistics

FROM: Sarah Grady, National Center for Education Statistics

SUBJECT: National Household Education Surveys Program (NHES) 2019 Types of Schooling Cognitive Interviews – Revision 2 [Revision to a previously approved package (1850-0803 v.212)]

The National Household Education Survey (NHES) is a data collection program of the National Center for Education Statistics (NCES) aimed at providing descriptive data on the educational activities of the U.S. population, with an emphasis on topics that are appropriate for household surveys rather than institutional surveys. Given a changing K-12 education landscape, and to improve NHES data quality, NCES is developing for 2019 a single Parent and Family Involvement in Education (NHES:PFI) questionnaire for all types of school-age children that, among other utilities, will identify whether a given child is a homeschooled student, virtual education student, enrolled student, or combination thereof. The request to conduct cognitive interviews to assess the extent to which draft NHES:PFI survey items for 2019 capture respondents’ perceptions regarding homeschooling, virtual schooling, and school choice phenomena, and to evaluate respondents’ ability to navigate the PFI questionnaire appropriately was approved in June 2017 with a revision approved in October 2017 (OMB #1850-0803 v. 200 & 212). The key points of interest for this study are: (a) respondents’ experiences with a combined PFI that includes questions for both families that homeschool and those that enroll their children in public or private schools and (b) performance of any new items that have not been part of PFI data collection in prior administrations, particularly new items related to virtual education, homeschooling, and school choice. Due to significant challenges encountered in recruiting participants, this request is to reduce the planned number of interviews, to create subgroup targeted advertising, and to increase the incentive to induce this hard-to-reach population to participate.

This request decreases the total estimated respondent burden and does not change the cost to the federal government.

Since the previous revision to the plan for this study (approved in October 2017), this study was scheduled to conclude in October 2017. However, we continue to encounter serious difficulties recruiting participants for this study. To date, recruitment ads have been placed on listserves and social media and posted in or distributed to over 135 locations including schools, libraries, and school choice organizations. These outreach efforts resulted in only 65 calls from interested individuals, which resulted in 25 completed interviews and 9 more interviews scheduled (some of which might result in no shows), out of the 85 desired interviews. The advertisements mentioned first the $40 and then the $50 incentive amount. We think that listing these incentive amounts may have led to the insufficient response to the outreach efforts. Given that, in order to meet NHES developmental schedule, we must conclude cognitive interviews this month (November 2017), this request is to expand the methods for recruitment, increase the incentive amount offered to prospective participants, and decrease the target number of interviews from the approved 85 to 48.

To reflect the desired changes, we made the following revisions to the approved clearance documents:

On page 3 of Volume 1, in the NHES:2019 Types of Schooling Cognitive Interviews section, the sentences:

This request is to conduct cognitive interviews to assess the extent to which draft NHES:PFI survey items capture respondents’ perceptions regarding homeschooling, virtual schooling, and school choice phenomena and to evaluate respondents’ ability to navigate the PFI questionnaire appropriately. Of particular interest will be (a) respondents’ experiences with a combined PFI that includes questions for both families who homeschool as well as families who enroll their children in public and private schools and (b) any new items that have not been part of PFI data collection in prior administrations, particularly new items related to virtual education, homeschooling, and school choice.

were replaced with:

The request to conduct cognitive interviews to assess the extent to which draft NHES:PFI survey items for 2019 capture respondents’ perceptions regarding homeschooling, virtual schooling, and school choice phenomena, and to evaluate respondents’ ability to navigate the PFI questionnaire appropriately was approved in June 2017 with a revision approved in October 2017 (OMB #1850-0803 v. 200 & 212). The key points of interest for this study are (a) respondents’ experiences with a combined PFI that includes questions for both families that homeschool and those who enroll their children in public or private schools and (b) performance of any new items that have not been part of PFI data collection in prior administrations, particularly new items related to virtual education, homeschooling, and school choice. Due to significant challenges encountered in recruiting participants, this request is to reduce the planned number of interviews and to create subgroup targeted advertising.

On page 3 of Volume 1, in the Design section, the sentences:

A total of 85 cognitive interviews will be conducted. At least 6 interviews regarding instrument navigation, 16 interviews with parents of homeschoolers, 16 interviews with parents of enrolled students who utilize some virtual education, and 6 parents of students whose school is not their assigned public school will be conducted in-person.

were replaced with:

A total of 48 cognitive interviews will be conducted. All interviews will look at instrument navigation, with 10 focusing on instrument navigation and the rest on testing new content. At least 5 interviews will be conducted with parents of homeschoolers, at least 5 with parents of students enrolled in a public or private school who utilize some virtual education, at least 2 parents of students whose school is not their assigned public school (i.e., utilizing school choice), and at least 5 interviews with parents of students enrolled in their assigned public school that does not utilize virtual education (target is 10 interviews in each group conducted in English and 2 interviews in Spanish; see Table 1).

In the same section on page 4, Table 1 was revised from:

Interview type and school arrangement	Language of parent		Total
	English	Spanish
Total	70	15	85
Interview type
Navigation	10	0	10
New content	60	15	75
School arrangement
Homeschool and virtual school	30	5	35
Public/Private school and virtual school	30	5	35
Public/Private school with school choice	10	5	15

to:

Interview type and school arrangement	Language of parent		Total
	English	Spanish
Total	40	8	48
Interview type
Navigation	10	0	10
New content	30	8	38
School arrangement
Homeschool	10	2	12
Public/Private school and virtual school	10	2	12
Public/Private school with school choice	10	2	12
Public school without virtual school or school choice	10	2	12

On page 4 of Volume 1, in the Recruiting and Paying Respondents section, $50 was replaced with $75 in the sentence: “To assure that we are able to recruit the difficult to recruit, select types of participants who are representative of homeschooling, virtual schooling, and school choice, who are like the parents that will take part in the 2019 NHES, and to thank them for their time and for completing the interview, during recruitment each parent will be offered a $75 incentive for participation.”

On page 5 of Volume 1, in the Estimate of Hour Burden section, the sentences:

We anticipate it will require 20 recruitment attempts per eligible participant (thus an estimated 1,700 attempts to yield 85 participants). This will result in an estimated total of 227 hours of respondent burden for this study.

were changed to:

We anticipate it will require 35 recruitment attempts per eligible participant (thus an estimated 1,680 attempts to yield 48 participants). This will result in an estimated total of 188 hours of respondent burden for this study.

In the same section, Table 2 was changed from:

Respondents	Number of Respondents	Number of Responses	Burden Hours per Respondent	Total Burden Hours
Recruitment	1,700	1,700	0.0833	142
Cognitive Interviews	85	85	1	85
Total	1,700	1,785	-	227

to:

Respondents	Number of Respondents	Number of Responses	Burden Hours per Respondent	Total Burden Hours
Recruitment	1,680	1,680	0.0833	140
Cognitive Interviews	48	48	1	48
Total	1,680	1,728	-	188

On page 5 of Volume 1, in the Project Schedule section, “October” was changed to “November” in the sentence: “The interviews are expected to be completed by November 2017.”

Throughout Attachment 1 and Attachment 2, the amount of the incentive was changed from $50 to $75.

Throughout Attachment 1, in order to speed up the process, letters will now be signed by the recruiter instead of Sarah Grady (NHES Project Officer at NCES).

In Attachment 1 on pages 2-7, in the Recruitment Phone Script, Email, and Advertisement sections, the sentence: “To improve the information NCES collects about experiences of K-12 students, we are looking for parents of children who are homeschooled or in school for grades kindergarten through 12 and are also enrolled in online, virtual, or cyber schooling to participate in a one-time voluntary interview.” was changed to “To improve the information NCES collects about experiences of K-12 students, we are looking for parents of children who are homeschooled, online/virtual/cyber schooled, or in school for grades kindergarten through 12 to participate in a one-time voluntary interview. ”

On pages 8-10 of Attachment 1, in the Letter to School Districts, Schools, and Parent Teacher Associations sections, the sentence:

“To improve the information that NCES collects about experiences of K-12 students, we are looking for parents of children who are in school for grades kindergarten through 12 and are also enrolled in online, virtual, or cyber schooling to participate in a one-time voluntary interview”

was replaced with:

“To improve the information that NCES collects about experiences of K-12 students, we are looking for parents of children who are in grades kindergarten through 12 to participate in a one-time voluntary interview”

On pages 11-14 of Attachment 1, in the Recruitment Flyers section, the original flyers have been replaced with target audience flyers (to parents of homeschoolers, parents of virtual schoolers, parents of school choice, and parents of regular K-12 students) that are also more visually appealing.

On pages 15-25 of Attachment 1, in the Recruitment Screener section, (a) references to “virtual or online schooling, homeschooling, and school choice” were taken out of the screener script (other than screener questions); (b) the screener has been revised to make parents of regular K-12 students eligible for the study (in the approved version, parents of regular K-12 students were deemed ineligible); and (c) the Screener Matrices were changed to match the targets presented in Volume 1.

On pages 28-30 of Attachment 1, in the Consent to Participate in Interview Research section, references to virtual schooling, online schooling, homeschooling, and school choice” were taken out.

On the title page of Attachment 1, the following note was added in red, bolded font: “Please note that in some cases the recruitment materials provided in this appendix will be utilized without listing the incentive amount.”

No edits were made to Attachment 3.

3