Justification Memo

NATIONAL CENTER FOR EDUCATION STATISTICS

(NCES)

Volume 1

Cognitive Testing of NCES Confidentiality Pledges

OMB# 1850-0803 v.162

July 28, 2016

Revised August 3, 2016

1. Introduction

The passage of the Cybersecurity Enhancement Act of 2015 requires the installation of the Department of Homeland Security’s Einstein cybersecurity protection system on all Federal civilian information technology systems by mid-December 2016. Combined with DHS’ stated policies, it also potentially compromises the absolute nature of the Federal statistical system’s (FSS) CIPSEA confidentiality pledges by no longer enabling statistical agencies to pledge that respondents’ data will be seen only by a statistical agency’s employees or its sworn agents. Consequently, the FSS needs to develop revised confidentiality pledges that inform respondents of this change in circumstances.

To optimize the effects of the revised confidentiality pledges, we need to conduct research to determine respondents’ comprehension of, and reaction to, the revised language. We will also explore wording options to determine which option may minimize the negative impact. Within NCES, the National Assessment of Educational Progress (NAEP) data collections currently uses the standard CIPSEA language to pledge confidentiality. NCES will focus on testing the revised CIPSEA pledges with educational institutional respondents, as they make up the NAEP respondent pool. NCES will be working with an interagency group, sharing research designs and findings to assist other statistical agencies in conducting similar testing for other respondent groups. It is possible that such a change may have differential effects on various classes of respondents such as households, small establishments, large establishments/enterprises, farmers and ranchers, educational or medical institutions, etc. Two versions of the revised CIPSEA language will be tested by NCES (Attachment A).

In addition, NCES sample surveys are governed by additional laws, most notable of which is the Education Sciences Reform Act of 2002 (ESRA 2002) (20 U.S. Code, Section 9573). This law permits the Attorney General to petition a court of competent jurisdiction for an ex parte order requiring the Secretary of Education to provide data relevant to authorized investigation or prosecution of an offense concerning national or international terrorism. As a result, NCES sample surveys use a modified version of the standard confidentiality pledge, henceforth referred to as the ESRA Pledge. NCES will test two versions of the ESRA Pledge that specifically refer to the new Cybersecurity Act (see Attachment B). Finally, because the ESRA Pledge was adopted without testing over a decade ago, NCES will also test a more conservative version of the ESRA Pledge to gauge the effect invoking the use of the term terrorism has on respondents (Attachment C).

2. Methodology

Participants will be recruited for the sole purpose of testing the CIPSEA and ESRA pledges. A standard cognitive interview approach will be used. Researchers from CRP, Incorporated (CRP), a Washington DC-based social science research and professional service firm, will conduct the cognitive interviews. Participants will be interviewed in-person, by telephone, and/or by Webinar, depending on the participant’s location and ability to travel to CRP’s cognitive laboratory in Washington DC. Other possible locations may include the participant’s school or an adjacent community center/site. The interview is expected to take an hour or less, and the protocols are included in Attachments D through F.

All participants will receive the current pledge first. When testing the CIPSEA pledges, a random half of the participants will receive Version A of the revised CIPSEA pledge second, and a random half of the participants will receive Version B of the revised CIPSEA pledge second. When testing the ESRA pledges with cybersecurity language, a random half will receive Version A of the revised ESRA pledge second, and a random half will receive Version B second.

Since there are only two ESRA pledges with terrorism language, all participants will receive the current pledge first, followed by Version A second.

3. Cost to Federal Government

The cost to the federal government to conduct this study is $ 80,600.

4. Participants

Up to 24 participants will be recruited to test the CIPSEA pledges. The goal is to obtain 8 interviews with principals, 8 with teachers, and 8 with parents. We will work with the NAEP Division to recruit participants through their Principal Panel. At the conclusion of each principal screening (Attachment I), we will enlist the help of the principal in identifying teachers for inclusion in this study. In addition to using the principal panel, we will also utilize personal and professional networks as well as advertisements and flyers to recruit participants that represent our populations of interest.

CRP will recruit potential participants in urban areas such Washington, D.C.; and Baltimore, MD, as well as suburban and rural areas in Maryland and Virginia. In addition to the aforementioned areas, CRP will also recruit in other geographical areas within the United States.

Recruitment Process will include:

1. CRP will send an email of introduction about the cognitive interview research to the NAEP Principals’ Panel and other recommended principals, using the NAEP State Service Center database. The email will include one of the attached fliers, depending on where the principal resides. CRP will phone those who reply to the email to confirm their willingness to participate in the interview, as well as to ask them to disseminate the flyers (section IX below) to potential parents, teachers, and postsecondary students (ages 18-25).

2. Simultaneously, CRP will use its pool of existing participants (e.g., existing participant databases, targeted telephone and mail contact lists, and school system research/assessment directors) to recruit additional participants. CRP will only discuss recruitment with interested participants that contact CRP.

3. If CRP receives an email or a phone message from a potential participant, CRP will call them and ask them to provide demographic information to ensure that a diverse sample is selected, per the screening checklist (Attachment I).

4. CRP will confirm participation and the date and time of the cognitive interview session.

5. Parents, postsecondary students (ages 18-25), teachers, and principals will be required to sign informed consent forms prior to the cognitive interview session.

6. All respondents with a signed consent will be asked to participate in cognitive interviews that may last up to 60 minutes. After participating in the cognitive interview, an incentive of a $40 Visa gift card will be provided to local in-person participants and to all participants outside of the metropolitan area (see section VI below) and a thank you letter/email will be sent.

In-person participants will be recruited from the Washington Metropolitan area (Maryland, Virginia, and DC) and asked to come to the CRP laboratory or an adjacent community center/site. To insure geographic diversity, telephone and Webinar respondents will be recruited from beyond the Washington Metropolitan area.

Up to 24 participants will be recruited to test the ESRA pledges with cybersecurity language. Using the same methods as discussed above, the goal is to recruit 8 interviews with teachers, 8 with parents, and 8 with postsecondary students.

Up to 24 participants will be recruited to test the ESRA pledge with terrorism language. Again, the goal is to obtain 8 interviews with teachers, 8 with parents, and 8 with postsecondary students.

In testing the language of both the CIPSEA and the ESRA Pledges, an attempt will be made to recruit at least a few participants who do not trust the federal government, given that the views of those who do not trust the government are likely to differ from those who do trust the federal government. As part of the recruitment and screening of participants (Attachment I), we will ask the Pew Research Center’s trust-in-government question (Pew Research Center, 2015). This question has been asked for nearly 60 years. Participants who choose the option a or b will be categorized as trusting the federal government and those who choose the option c or d will be categorized as not trusting government.

1. In general, how much do you think you can trust the government in Washington to do what is right?

   1. Just about always

   2. Most of the time

   3. Only sometimes

   4. Never

The other Federal agencies in the group did not think to recruit respondents who do not trust the federal government, but upon reflection, all of the agencies agreed that this was a good idea. Since NCES is submitting its package later than the other agencies, we will try to implement this idea during the screening process (Attachment I).

In addition, an attempt will be made to recruit at least a few participants who do not view the Department of Education favorably. To do this, we will ask the Pew Research Center’s favorability question.

2. Please tell me if you have a favorable or unfavorable impression of the Department of Education?

   1. Favorable

   2. Unfavorable

5. Burden Hours

The burden hours for this research are shown in Table 1.

Table 1. Estimated burden hours, by pledge and target population.

Pledge	Population	# of Partici-pants Screened	Minutes per participant for Screening	Total Screening Burden (minutes)	Max number of Partici-pants	Minutes per partici-pant for data collection	Total Data Collection Burden (minutes)	Total Screening + Collection Burden (hours)
CIPSEA Pledge
	Principals	16	5	80	8	60	480	10
	Teachers	16	5	80	8	60	480	10
	Parents	16	5	80	8	60	480	10
ESRA Pledge With Cybersecurity Language
	Teachers	16	5	80	8	60	480	10
	Parents	16	5	80	8	60	480	10
	Post-secondary Students	16	5	80	8	60	480	10
ESRA Pledge with Terrorism Language
	Teachers	16	5	80	8	40	320	7
	Parents	16	5	80	8	40	320	7
	Post-secondary Students	16	5	80	8	40	320	7
Total Burden*		144	-	144	72	-	-	81

* The total number of respondents is 144, the total number of responses is 216, and the total number of burden hours is 81.

6. Payment to Participants

To encourage their participation and thank them for their time and effort, all participants who live outside the Washington Metropolitan Area and who complete the study over the phone or by Webinar, and all participants who live in the Washington Metropolitan Area and elect to participate in an in-person interview will be offered $40 (a VISA gift card) for participating in the study. Participants who live in the Washington Metropolitan Area and who choose to complete the study over the phone or by Webinar will receive no monetary incentive.

7. Data Confidentiality

All in-person cognitive interview participants will have to sign the study consent form before the start of their cognitive interview (Attachment G), and all over-the-telephone or by-Webinar cognitive interview participants will be read the consent form and their verbal consent will be recorded prior to beginning the interview (Attachment H). The consent form includes the following statement:

“All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C., § 9573). The names of participants will not be included in the write-up of this study.

During this research you will be audiotaped and the interview may be observed. If you do not wish to be audiotaped, you still may participate in this research.”

On the consent form for those participating by phone or webinar (Attachment H), the sentence about voluntary nature of audiotaping (the last sentence above) instead reads:

“If you do not wish to be audiotaped, you still may participate in this research, but I must first record your consent to participate after I read to you the consent statement.”

8. References

Pew Research Center, November 2015, “Beyond Distrust: How Americans View Their Government.”

9. Sample Recruitment Flyers

Concerned about sharing YOUR data?

Interested?

Contact: CRP, Incorporated * Telephone: 1-866-277-4772 * Ask for: Terry Starke

Your input is needed to help evaluate new language ensuring that our pledge to YOU is clear and understandable. We want YOUR input!!

CRP, Inc., a local research company, is conducting cognitive interviews for the U.S. Department of Education’s National Center for Education Statistics (NCES).

The one-on-one interviews are designed to inform the public of changes to the confidentiality pledges that are used for assessments and surveys.

All responses will be audio recorded. No video recording will take place. Identities and responses will not be included in study reports.

The interviews will last no more than 1 hour.

Location of the Interview:

CRP’s Office in Washington, DC; virtual; or telephone

Tell us what you think!

Participate in an

NCES RESEARCH INTERVIEW

Scheduled to take place August 2016 through October 2016

Local participants will receive a $40 Visa gift

card for an

in-person interview.

Concerned about sharing YOUR data?

Long distance participants will receive a $40 Visa gift

card for taking part in a

virtual or telephone

interview.

Tell us what you think!

Participate in an

NCES RESEARCH INTERVIEW

Scheduled to take place August 2016 through October 2016

Your input is needed to help evaluate new language ensuring that our pledge to YOU is clear and understandable. We want YOUR input!!

CRP, Inc., a Washington DC research company, is conducting cognitive interviews for the U.S. Department of Education’s National Center for Education Statistics (NCES).

The one-on-one interviews are designed to inform the public of changes to the confidentiality pledges that are used for assessments and surveys.

All responses will be audio recorded. No video recording will take place. Identities and responses will not be included in study reports.

The interviews will last no more than 1 hour.

Location of the Interview:

Virtual or telephone

Interested?

Contact: CRP, Incorporated * Telephone: 1-866-277-4772 * Ask for: Terry Starke

Attachment A: Versions of CIPSEA Confidentiality Pledges to Test through Cognitive Interviews – cybersecurity language

Note: The text formatting (shading and bold font) shown below is for review purposes only, and will not be used in the materials shown to study participants. On this page, language highlighted in gray is the currently used CIPSEA confidentiality pledge. The bolded language is the expansion to the confidentiality pledge that will be tested through cognitive interviews described in this submission.

Current CIPSEA Pledge

The information you provide will be used for statistical purposes only. In accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107-347 and other applicable Federal laws, your responses will be kept confidential and will not be disclosed in identifiable form to anyone other than employees or agents. By law, every NCES employee as well as every agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of up to $250,000, or both if he or she willfully discloses ANY identifiable information about you.

Version A: CIPSEA Pledge

The information you provide will be used for statistical purposes only. In accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107-347 and other applicable Federal laws, your responses will be kept confidential and will not be disclosed in identifiable form to anyone other than employees or agents. By law, every NCES employee as well as every agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of up to $250,000, or both if he or she willfully discloses ANY identifiable information about you. Electronic transmission of your information will be monitored by Homeland Security in accordance with the Cybersecurity Enhancement Act of 2015.

Version B: CIPSEA Pledge

The information you provide will be used for statistical purposes only. In accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107-347 and other applicable Federal laws, your responses will be kept confidential and will not be disclosed in identifiable form to anyone other than employees or agents. By law, every NCES employee as well as every agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of up to $250,000, or both if he or she willfully discloses ANY identifiable information about you. NCES systems are further protected by Federal employees and contractors through security monitoring of systems that transmit your data.

Attachment B: Versions of ESRA Confidentiality Pledges to Test through Cognitive Interview – cybersecurity language

Note: The text formatting (shading and bold font) shown below is for review purposes only, and will not be used in the materials shown to study participants. On this page, language highlighted in gray is the currently used ESRA confidentiality pledge. The bolded language is the expansion to the confidentiality pledge that will be tested through cognitive interviews described in this submission.

Current ESRA Pledge

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C., § 9573).

Version A: ESRA Pledge with Cybersecurity Language

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C., § 9573). Electronic transmission of your information will be monitored by Homeland Security in accordance with the Cybersecurity Enhancement Act of 2015.

Version B: ESRA Pledge with Cybersecurity Language

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C., § 9573). NCES systems are further protected by Federal employees and contractors through security monitoring of systems that transmit your data.

Attachment C: Versions of ESRA Confidentiality Pledges to Test through Cognitive Interview – terrorism language

Note: The text formatting (shading and bold font) shown below is for review purposes only, and will not be used in the materials shown to study participants. On this page, language highlighted in gray is the currently used ESRA confidentiality pledge. The bolded language is the expansion to the confidentiality pledge that will be tested through cognitive interviews described in this submission.

Current ESRA Pledge

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C., § 9573).

Version A: ESRA Pledge with Terrorism Language

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except in the case of terrorism (20 U.S.C., § 9573).