Volume 1 - SSOCS-CRDC Incident Count Cognitive Interviews

Volume 1 SSOCS-CRDC Incident Count Cognitive Interviews.docx

NCES System Clearance for Cognitive, Pilot, and Field Test Studies 2019-2022

Volume 1 - SSOCS-CRDC Incident Count Cognitive Interviews

OMB: 1850-0803

Document [docx]
Download: docx | pdf





School Survey of Crime and Safety (SSOCS) Incident Count Check Cognitive Interviews Update




Volume I

Supporting Statement





OMB #1850-0803 v.262







Submitted by

National Center for Education Statistics (NCES)

U.S. Department of Education






February 2020






Appendixes:

Appendix A: Recruitment Materials

Appendix B: Protocols

Appendix C: Questionnaire Handouts

Submittal-Related Information

The following material is being submitted under the National Center for Education Statistics (NCES) generic clearance agreement (OMB# 1850-0803), which provides NCES the capability to improve data collection instruments by conducting testing—such as usability tests, focus groups, and cognitive interviews—to improve methodologies, survey questions, and/or delivery methods.

The request to conduct Phase 1 and Phase 2 of the School Survey of Crime and Safety (SSOCS) Incident Count Check Cognitive Interviews 2018 was approved in June 2019 (OMB #1850-0803 v.250). Based on the results of Phase 1, which was conducted from October to December of 2019, this request includes updates for Phase 2 data collection and includes revisions to: (1) recruitment procedures, (2) communication materials, and (3) Phase 2 protocols.

1. Background and Study Rationale

The School Survey on Crime and Safety (SSOCS) is a nationally representative survey of public elementary and secondary schools and is one of the nation’s primary sources of school-level data on crime and safety in public schools. Conducted by NCES, within the U.S. Department of Education (ED), SSOCS has been administered seven times starting in the 1999–2000 school year. The next administration is planned for the 2019–20 school year.


SSOCS is designed to produce nationally representative data on violence and safety practices in public schools. SSOCS questionnaires are completed by school principals or other school personnel designated by the principal as the person who is “most knowledgeable about school crime and policies to provide a safe environment.” SSOCS is the only recurring federal survey that collects detailed information on the incidence, frequency, seriousness, and nature of violence affecting students and school personnel, as well as on other indices of school safety, directly from school-level respondents.


Topics covered by SSOCS include school programs and practices, parent and community involvement, school security staff, school mental health services, staff training, limitations on crime prevention, the type and frequency of crime and violence, and the types of disciplinary problems and actions used in schools. Other federal surveys obtain information about school crime from individuals other than those who have the school-level perspective of principals.


One such survey is the Civil Rights Data Collection (CRDC). The CRDC is administered by the U.S. Department of Education’s Office of Civil Rights (OCR) and has collected data on education and civil rights issues in U.S. public schools since 1968. The CRDC collects a variety of information from superintendents and staff at local education agencies (LEAs), most of which is disaggregated by race/ethnicity, sex, limited English proficiency, and disability. Information collected by the CRDC is used by ED offices as well as policymakers and researchers outside of ED. The most recent collections were census collections of all LEAs. CRDC is currently finishing collection of 2017–18 data.


Between SSOCS and the CRDC, there is overlap in the counts of crime incidents reported, disciplinary actions, and harassment/bullying data. When SSOCS experienced a lapse in funding after the 2009–10 data collection, ED deemed it crucial to continue collecting counts of the key metrics traditionally collected in the SSOCS, and they were incorporated into the CRDC. When SSOCS resumed in 2015–16, incident counts remained in both SSOCS and the CRDC. Consequently, there are incident counts in both SSOCS and the CRDC for the 2015–16 and 2017–18 school years. This was intentional to assess how the counts obtained through both data collections compare.


Initial analyses show discrepancies in the information reported for schools that participated in both SSOCS and the CRDC. To gain a better understanding of the context around the differences in the reporting of the items in the two surveys, this request is to conduct a two-phase cognitive interview study to explore respondents’ understanding of the incident count questions—in particular, what respondents include in, and exclude from, their calculations and what records respondents reference to answer the questions—in order to determine whether respondents are providing the intended information. The goals of the study are to obtain information to improve the survey items, reduce the burden of future data collections, and ensure that the resulting data are more accurate for schools, districts, policymakers, and other data users. NCES contracted the American Institutes for Research (AIR) to carry out the cognitive interviews.


2. Study Design and Sample Characteristics

Taking into consideration data collection timelines, as well as the need to test and refine interview procedures and materials to ensure that the data being collected will answer the research questions, this study will be conducted in two phases. Phase 1 will consist of a small number of interviews in which the recruitment methods and study materials will be tested; the results of these interviews will be used to inform the conditions for Phase 2, which will be similar to Phase 1 but on a larger scale. With participant’s permission, each interview will be audio recorded to aid in data analysis and report writing.


The sample will be composed of public schools and districts; interview respondents will be the staff who were responsible for filling out the school crime incident counts in the 2017-18 SSOCS and 2017-18 CRDC questionnaires (per the contact information provided in each data submission). SSOCS respondents will include elementary, middle, and high school principals or other school personnel designated by the principal as the person who is “most knowledgeable about school crime and policies to provide a safe environment.” The CRDC respondents will be the staff at the respective LEA for that school. For each school in the sample, the SSOCS and CRDC respondents will create an “interview pair”, but each respondent will be interviewed individually. On behalf of NCES, AIR will recruit respondents, administer interviews, and provide to NCES reports and recommendations based on analyses of findings for both phases of this study. A brief description of the sample characteristics and objectives is provided below.

Phase 1

Phase 1 was designed to consist of approximately 10 interview pairs―one SSOCS respondent and one CRDC respondent, for a total of approximately 20 interviews—in July and August of 2019. These pairs would represent a range of: (a) incident types, (b) which data collection(s) the incidents were reported on, and (c) school level. A purposive sample would be drawn to ensure appropriate coverage of possible scenarios and school characteristics.


The interviews in Phase 1 were designed to evaluate the feasibility of scaling up the study methods and materials to conduct Phase 2 of the study. The Phase 1 interviews would allow us to assess the success of the recruitment tactics and to test the interview protocols to determine if the questions produce the information necessary to be able to determine validation issues/discrepancies across the two surveys. The findings from Phase 1 would provide recommendations for revisions to the study materials for Phase 2. A summary of the results from Phase 1 and recommendations for Phase 2 would be developed by late August 2019. If revisions were needed to materials for Phase 2, a revised request with the revised materials and a memo listing the changes would be submitted to OMB at that time for approval before Phase 2 begins.

Phase 1 Results

Phase 1 of the cognitive interview study launched on October 16, 2019 and lasted for eight weeks, ending on December 13, 2019. AIR staff invited 120 principals and four district administrators to participate in an hour-long interview. Recruitment efforts encountered a number of challenges, and after eight weeks of recruitment, only one SSOCS respondent and one CRDC respondent were interviewed.

Central goals of Phase 1 were to determine the ease of pinpointing the correct contacts for both surveys and provide a better idea of the time frame required to successfully recruit both contacts. Locating the correct respondent who completed 2017-18 SSOCS had implications on operational activities and study eligibility. Given high principal turnover in the last two academic years among the sampled schools, AIR had limited success in reaching 2017-18 SSOCS respondents. During recruitment, AIR reviewed automatic email bounce backs and verbal communication with gatekeepers and determined that at least 38 of 120 (about 32 percent) SSOCS respondents are no longer with the school. Additionally, some SSOCS respondents who AIR was able to reach were interested in participating, however, they expressed reservations with how the SSOCS interview related to the paired CRDC district respondent interview and whether they needed approval from the district level to participate in the study. These cases ended up resulting in no interviews because of district nonresponse to communication about the study (originating from both the SSOCS respondent and AIR). Of the remaining sample members, the majority were nonrespondents after the full recruitment cycle of six outreach attempts (four emails and two phone calls) was completed. Due to the number of ineligible respondents and nonrespondents, AIR was required to release additional sample members on two separate occasions but was still unable to increase participation during Phase 1.

Phase 2

Phase 2 will consist of up to 200 interviews in spring 2020. Based on the results of conducting Phase 1, Phase 2 will pursue interviews with SSOCS respondents and CRDC respondents separately; they will not consist of interview pairs representing the same school. Respondents will represent a range of school and district characteristics, including schools in different locales, schools serving different grade levels, and schools with varying enrollment sizes. Note that while the sample will include a mix of characteristics, the results will not explicitly measure differences by these characteristics.


The Phase 2 interviews will be used to assess the validity of the SSOCS and CRDC incident items and will focus on the following research questions:

  • Who is the respondent (principal, superintendent, other staff member, etc.) who completed the incident counts for SSOCS and the CRDC?

  • How did the respondent collect the incident count data? For example, did the respondent estimate the counts or use records? If the respondent used records, what records?

  • Are there differences by respondent type in the interpretation of the incident count items in SSOCS and the CRDC?

  • Did the respondent verify the incident count data? If so, how did the respondent do this?

  • How are the procedures for providing incident counts for SSOCS and the CRDC similar? How are they different? In what ways do the similarities and differences vary by whether the SSOCS and CRDC counts match/do not match?

Upon completion of the Phase 2 interviews, the interview notes will be analyzed to identify trends in the interview data and a summary report presenting the results of the study will be prepared. The report will include a description of key findings on schools’ and districts’ interpretations of the incident items in SSOCS and the CRDC, staff members responsible for completing the survey items, data storage procedures, and methods for providing and validating incident counts for the surveys. The summary report will provide recommendations for future administrations of SSOCS, specifically pertaining to the questions collecting incident count data.

3. Data Collection

3.1 Recruitment

In both phases of the study, recruitment is anticipated to last approximately 4 weeks and the recruitment process is expected to take 5 to 20 minutes per entity. A central goal of Phase 1 was to determine the ease of pinpointing the correct contacts for both surveys and provide a better idea of the time frame required to successfully recruit both contacts. AIR developed and maintained a detailed tracking sheet of recruitment efforts for each sampled district and school; this will aid in understanding the true level of effort required for recruitment prior to Phase 2 of the study.


Prior to the start of recruitment, OCR provided AIR with a list of the points of contact for the 2017–18 CRDC. Recruitment will begin once a sample of linked SSOCS schools and CRDC districts has been drawn. After the sample of schools has been selected, a notification letter from NCES will be e-mailed to the schools and their districts informing them of the study activities and inviting them to participate. School and district contacts will be assigned an AIR staff member who will serve as their main point of contact and who will administer the interviews with the respondents.


During recruitment, if the SSOCS or CRDC contact indicates they were not responsible for filling out the school’s incident count data, AIR will request contact information for the correct school or district staff member and then reach out accordingly. Once an interview has been scheduled, AIR staff will send a meeting invite via e-mail to confirm the interview date and time. A consent form and non-disclosure agreement (for district administrators only, during Phase 1 only) will be attached for their signature.


AIR will use multiple outreach methods and resources to recruit participants. SSOCS and CRDC respondents will be contacted by e-mail and phone during recruitment, and AIR will confirm that interested individuals are eligible to participate. Our recruitment experience with cognitive interviews for 2017-18 SSOCS and the 2017–18 CRDC indicated that these respondents are a hard-to-reach population, and increased time and effort will be dedicated to meeting recruitment targets.


Phase 1

Recruitment began by contacting the staff member listed as the 2017-18 SSOCS point of contact for each school to determine whether that person filled in the incident data and is the appropriate person for the interview. Once the SSOCS respondent has agreed to participate in an interview, they were asked to sign a consent form that requests permission for the following:


  1. to share information from the school’s participation in SSOCS with the district contact.

  2. to participate in the interview.


Once the SSOCS respondent agreed to participate in an interview, a similar outreach process was used to contact the listed CRDC 2017–18 point of contact for that school’s district. If the school consented to share their responses on SSOCS with their district, the CRDC respondent was asked to sign a non-disclosure affidavit (NDA) and a consent form that they agreed to participate in the interview. If the school did not consent to share their responses on SSOCS, the interview would be adjusted so that this information is not shared with the district and a non-disclosure affidavit would not be required. For more information about consent and non-disclosure procedures, see the “Assurance of Confidentiality” section below.


If a SSOCS respondent agreed to an interview, but AIR was unable to schedule an interview with the CRDC district respondent―either AIR was unable to reach the correct respondent or the respondent declined to be interviewed―AIR proceeded with interviewing the school only.


Phase 2

Phase 2 will closely follow the same recruitment process as Phase 1; however, based on the challenges faced during Phase 1 recruitment the following changes to recruitment strategies will be made for Phase 2. These changes reflect the decision to pursue interviews with SSOCS and CRDC respondents independently (rather than as an interview pair):

  • AIR will no longer need to contact a SSOCS respondent prior to reaching out to the CRDC respondent. A similar outreach process will be used to contact both 2017-18 SSOCS and 2017-18 CRDC respondents simultaneously throughout the recruitment period.

  • SSOCS data will no longer be shared with CRDC respondents. Schools will no longer be asked to consent to sharing their data and CRDC respondents will no longer be required to sign a non-disclosure affidavit (NDA).


Recruitment for Phase 2 will last for approximately 8 weeks. The increased recruitment window is based on the limited success in recruiting participants during the shorter window in Phase 1 and the higher number of target interviews for Phase 2. This timeframe will provide AIR the opportunity to recruit from a larger sample of schools and will allow for additional follow-up outreach to be conducted with each potential participant over a longer window.


Appendix A contains the full set of recruitment materials, including a visual representation of the recruitment process (see figure 1 in Appendix A). Screening and recruiting participants for qualitative studies such as this one is a dynamic process. The recruitment email templates and scripts will be used in Phase 11. Materials for Phase 2 were adapted based on feedback received during Phase 1. The contact materials in Appendix A are grouped separately for each phase: Phase 1 and Phase 2.

3.2 Qualitative Interview Methods

Qualitative interview practices will be used to gather information from SSOCS and CRDC respondents to learn about their experiences reporting information for the school crime incident count items that are included in each survey. In each qualitative interview, trained interviewers will provide an overview of the study goals and procedures and then ask participants to answer questions about their experiences responding to either SSOCS or the CRDC. The interviewers will use a semi-structured protocol (see Appendix B) drawing on methods from cognitive science to investigate the process that respondents use to answer the incident count questions in each survey. The main goal of this approach is to explore respondents’ understanding of the incident count questions—in particular, what they include in and exclude from their calculations when answering the questions—in order to determine whether respondents are providing the intended information.


The semi-structured interviewing methods will consist of two key components: think-aloud interviewing and verbal probing techniques (also known as concurrent and retrospective recall probing, respectively). With think-aloud interviewing, respondents are explicitly instructed to think aloud (i.e., describe what they are thinking) as they work through items. With verbal probing techniques, the interviewer asks probing questions, as necessary, to clarify points that are not evident from the think-aloud process or to explore additional issues that were identified a priori as being of particular interest.


Cognitive interview studies produce qualitative data in the form of verbalizations made by participants during the think-aloud interviewing and in response to interviewer probes. Both the think-aloud approach and probing techniques will be applied to all participants during the qualitative interviews. Interviewers will refer to the protocol to guide the content of the interviews but will also be free to deviate from the guide should participants have difficulty answering questions that do not have scripted probes or prompts.


The questions in the protocol will directly address the study’s research questions by gathering data on: how districts receive incident data from schools to report for the CRDC (and what processes they use to verify these data); what formats districts and schools use to store these data; and how district and school respondents understand and respond to the incident items in the two surveys, including any cognitive issues with definitions or question format. In order to compare the reporting processes between schools and districts for the same incident count items on the two surveys, the protocols include some questions that refer to estimates for specific incident count items that the school or district respondent provided in their respective survey (SSOCS or CRDC; Phase 1 only). For example, questions in the protocol for school interviews will discuss the incident counts from SSOCS and whether certain counts were higher, lower, or equal to the counts provided on the CRDC. Given time constraints, questions comparing reporting on SSOCS and the CRDC will only focus on the most serious incident for which the school or district reported data. No exact counts reported by the school on SSOCS will be shared with the district respondent. See the “Assurance of Confidentiality” section below for additional details. Given the decision to pursue interviews with SSOCS and CRDC respondents independently (rather than as interview pairs) during Phase 2, protocols for CRDC interviews will not include any school specific incident count data from the CRDC or SSOCS.


Following each interview, the digital audio recording will be archived for qualitative analysis. AIR staff will organize their observations and summarize the common themes, insights, and ideas emerging from each of the interviews into a report.


For the complete set of questions that are provided in the protocols, see Appendix B. Excerpts of the incident items taken directly from the data collection questionnaires (and the way in which items will be clustered for comparison) are provided in Appendix C.

3.3 Interview Logistics

The qualitative interviews will last 60 minutes to give participants sufficient time to reflect on the more open-ended questions included in the protocol and allow time for interviewers to probe for additional information as necessary. The qualitative interviews will be conducted remotely via video conference (using the web conferencing platform GoToMeeting). If a respondent would prefer that the interview be conducted over the phone, that will be accommodated. Interviews will be conducted primarily during regular work hours (9 am–6 pm EDT) but will be scheduled after work hours, as needed.

4. Consultation Outside the Agency

NCES has also been in consultation with some of the nation’s top content experts in school crime, school safety, and student support services in schools, as well as experts in methodology related to establishment and administrative surveys. In September and October of 2018, two technical review panels (TRPs) were conducted with external experts and representatives from federal agencies to provide feedback on the content and methodological approaches of SSOCS. The first TRP focused on whether SSOCS currently covers the most appropriate content areas/items, discussed how to address critical gaps in content in the questionnaire, and considered the usefulness and prioritization of current SSOCS items. The second TRP focused on potential improvements to SSOCS sampling and data collection procedures, including how SSOCS could be better integrated with or supplemented by extant administrative data sources, including the CRDC.

Feedback from the TRPs included the recommendation to conduct a thorough study to validate the crime incident data for the SSOCS and CRDC. The TRPs proposed that such a study include cognitive interviews with school and district officials (for schools that responded to both surveys) to determine how they responded to the incident items and uncover potential reasons for the matches or mismatches in incident counts reported across the two surveys.

5. Assurance of Confidentiality

All participants will be assured that their participation is voluntary, and materials shared with the participants will include the following language:


All the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).


A consent form that explains the purpose and duration of the interview will be sent via e-mail to both SSOCS and CRDC participants, to be signed and returned prior to their interview.


Phase 1

At the time of recruiting SSOCS participants, AIR staff sought to obtain consent from the school to be able to share minimal information regarding the school’s participation in SSOCS during the interview with the school’s district. This would include identifying the school that responded to the survey and providing limited information about the school’s responses to select incident items. No exact counts reported by the school on SSOCS would be shared with the district respondent. Consent to share the school’s responses on SSOCS would be collected at the time of recruitment and prior to the interview with the CRDC respondent. If written consent was not received from the SSOCS respondent prior to the interview with the CRDC respondent, information on the school’s responses on SSOCS was not shared during the district interview.


Additionally, CRDC district respondents were asked to sign a non-disclosure affidavit prior to the interview to affirm that they would not reveal any identifiable information or responses from the SSOCS school(s) in their district that may be shared with them as part of the interview. If the district respondent did not sign the non-disclosure affidavit, information on the school’s responses on SSOCS would not be shared during the district interview. See Appendix A for recruitment materials, including the consent form and the non-disclosure affidavit.


Phase 2

During Phase 2, AIR staff will no longer attempt to share minimal information regarding the school’s participation in SSOCS during the interview with the school’s district. As a result of this change, it will not be necessary to obtain consent from the school to share their information and CRDC district respondents will not be asked to sign a non-disclosure affidavit prior to their interview. Both SSOCS and CRDC respondents in Phase 2 will still be required to sign the basic form to consent to participating in the interview. See Appendix A for updated Phase 2 recruitment materials, including the consent form.


Participants will be assigned a unique identifier (participant ID), which will be created solely for data file management and used to keep all participant materials together. The participant ID will not be linked to the participant’s name except in a file used to manage recruitment and interviews. The consent and non-disclosure agreement forms, which include the participant’s name, will be separated from the participant interview files and secured for the duration of the study. The interviews will be audio-recorded. All files will be secured for the duration of the study—with access limited to key AIR project staff—and destroyed at the conclusion of the study.

6. Estimate of Costs for Recruiting and Paying Respondents

District- and school-level participation are vital to the success of this study, and from prior experience we expect that it will be challenging to gain district- and school-level administrators’ cooperation to participate. It has been identified in similar projects that incentives are an effective approach to district- and school-level administrator recruitment – they communicate appreciation of a respondent’s time and participation, and this may be especially important for districts and schools at the very busy time of the school year during which the study will be fielded. To encourage participation in the interviews and to thank participants for their time and effort, during Phase 1, we offered each a $50 gift card that was sent with a thank-you note via mail within 10 business days of completion of the interview. We offered a $50 gift card for 2018 SSOCS cognitive interviews conducted remotely with this population (OMB# 1850-0803 v.171) and were able to successfully reach acceptable levels of participation.


Low response rates during Phase 1 recruitment indicate that the incentive did not outweigh the anticipated burden of participation for respondents. In an effort to increase participation in Phase 2, we will offer each participant a $75 gift card. We offered a $75 gift card for past cognitive interviews with this population (OMB# 1850-0803 v.171) and were able to successfully reach acceptable levels of participation within planned recruitment timelines.

7. Estimate of Hourly Burden

We aimed to recruit 10 interview pairs (approximately 20 participants) for the qualitative interviews in Phase 1, and up to 100 interview pairs (approximately 200 participants) will be recruited for the qualitative interviews in Phase 2. Each interview will take approximately 60 minutes. The initial contact and screening of potential participants is estimated at an average of 10 minutes, or 0.16 hours. On average, three recruiting attempts are expected to be needed for each SSOCS participant (thus, an estimated 30 attempts to yield up to 10 SSOCS participants for Phase 1) and five recruiting attempts for each CRDC participant (thus, an estimated 50 attempts to yield up to 10 CRDC participants for Phase 1). Based on the results of recruitment in Phase 1, we anticipate additional attempts will be needed to obtain participants during Phase 2 – approximately 10 recruiting attempts for each SSOCS and CRDC participant (thus 1000 attempts to yield up to 100 SSOCS participants and 1000 attempts to yield up to 100 CRDC participants for Phase 2). Table 1 shows burden estimates for (a) recruiting participants (the initial contact and screening); and (b) conducting the qualitative interviews by each Phase of the study.





Table 1. Estimate of hourly burden for SSOCS qualitative interviews: Phase 1 and Phase 2

Activity

Number of Respondents*

Number of Responses

Burden Hours per Respondent

Total Burden Hours

Phase 1

SSOCS Recruitment

Recruitment (initial contact and screening)

30

30

0.16

5

Qualitative Interviews

10

10

1

10

CRDC Recruitment

Recruitment (initial contact and screening)

50

50

0.16

8

Qualitative interviews

10

10

1

10

Phase 1 Subtotal

80

100

-

33

Phase 2

SSOCS Recruitment

Recruitment (initial contact and screening)

1000

1000

0.16

160

Qualitative Interviews

100

100

1

100

CRDC Recruitment

Recruitment (initial contact and screening)

1000

1000

0.16

160

Qualitative Interviews

100

100

1

100

Phase 2 Subtotal

2,000

2,200

-

520

Total

2,080

2,300

-

553

Note: The subtotals for the estimated Number of Respondents do not include duplicated counts of respondents.

8. Schedule

Recruitment for the cognitive interview study was scheduled to begin in June 2019, pending the availability of CRDC 2017–18 data for matching analyses and sampling. The data collection of Phase 1 was scheduled to begin in early July 2019, with a summary of results and recommendations for Phase 2 developed by late August 2019. Phase 2 was scheduled to begin in October 2019. Tables 2 and 3 below provide the overall schedule for Phase 1 and Phase 2.

Table 2. Initial proposed schedule of high-level activities for 2017-18 SSOCS cognitive interviews study, Phase 1

Activity

Start Date

End Date

Phase 1

Training

June 24

June 26

SSOCS Recruitment

June 26

August 9

CRDC Recruitment

July 10

August 21

Interviews

July 11

August 22

Analysis

August 19

September 13

Reporting

August 26

September 29


Due to delays in the availability of 2017-18 CRDC data, Phase 1 was conducted from October to December 2019 and Phase 2 is scheduled to begin in February 2020. Based on recruitment challenges faced during Phase 1, an additional 4 weeks of recruitment efforts will be added to Phase 2, and the data collection period will be 8 weeks total.

Table 3. Schedule of high-level activities for 2017-18 SSOCS cognitive interviews study, Phase 2

Activity

Start Date

End Date

Phase 2

Training

February 10

February 12

SSOCS Recruitment

February 17

April 10

CRDC Recruitment

February 17

April 10

Interviews

February 24

April 17

Analysis

March 23

May 1

Reporting

April 24

July 20


9. Cost to the Federal Government2

The estimated cost to prepare for, administer, and report the results of these cognitive interviews is approximately $364,000. The cost includes salaried labor for contractor staff and other direct costs associated with the organization of the interviews.

1 In all phases of the study, in cases when an email or a phone call follow a prior interaction with the addressee, short personalized statements may be interwoven into the email and/or phone script, adjusted to the tone and subjects of prior communications with the respondents.

2 Given the increased recruitment efforts and incentive costs associated with Phase 2; AIR will monitor the total cost of the cognitive interviews to ensure costs do not exceed the funded amount. If necessary, to maintain costs, Phase 2 will consist of fewer than the 200 interviews originally planned.


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorKaleem, Korantema
File Modified0000-00-00
File Created2021-01-14

© 2024 OMB.report | Privacy Policy