Volume I NHES 2017 ATES Concept Interviews 2015

Volume I

2017 National Household Education Surveys Program (NHES)

Adult Training and Education Survey (ATES)

Work-Related Education and Training Concept Interviews

OMB# 1850-0803 v.139

July 15, 2015

Justification

The NHES is a data collection program of the National Center for Education Statistics (NCES) aimed at providing descriptive data on the educational activities of the U.S. population, with an emphasis on topics that are appropriate for household surveys rather than institutional surveys. Such topics have covered a wide range of issues, including early childhood care and education, children’s readiness for school, parents’ perceptions of school safety and discipline, before- and after-school activities of school-age children, participation in adult and career education, parents’ involvement in education, school choice, homeschooling, and civic involvement. The NHES uses a two-stage design in which sampled households complete a screener questionnaire to enumerate household members and their key characteristics. Within-household sampling from the screener data determines which household member receives which topical survey. The NHES typically fields 2 to 3 topical surveys at a time, although the number has varied across its administrations.

One of these NHES topical surveys, the Adult Training and Education Survey (ATES), focuses on attainment of non-degree credentials and participation in adult and career education. Out-of-high-school adults ages 16 to 65 are eligible to receive this topical survey. This topical survey has undergone extensive redesign and development work over the past five years. Three pilot studies have been conducted, and in 2016, the NHES is expected to field the first full-scale, official administration of the ATES.

The purpose of this submission is to conduct concept interviews that will be used to refine the ATES instrument in preparation for the next NHES administration, the 2017 NHES web pilot. In particular, the objective of this research is to develop and improve upon the items used to ask respondents about their participation in work-related classes and training. The concept interviews will be used to determine respondents’ understanding of key characteristics of their work-related coursework and training experiences, as well as the typical terminology and phrases respondents use to describe these characteristics. These interviews will ensure that the characteristics and terminology included in the survey items capture the key information about work-related coursework and training experiences as understood by those who have actually participated in them.

The primary deliverable will be a report highlighting the key interview findings. These findings will then be used to draft and refine survey items that will be cognitively tested in 2016, and will then be included in the NHES 2017 web pilot.

Design

These “concept” interviews are intended to be relatively unscripted conversations in which the interviewers determine the key characteristics of each respondent’s work-related courses and training. Each interview will begin with an open-ended request for information about the respondent’s current participation in work-related courses and training, followed by more specific probes aimed at capturing any details that were not initially mentioned by the respondent.

Interviews are expected to last about 1 hour and will be conducted by trained interviewers. This submission includes the protocol that will be used to guide the interviews. The research is expected to be iterative, in that contact materials and the interview protocol may change during the interview period in response to early findings. Interviews will be conducted over the telephone and will be audio-recorded with the respondent’s permission.

To adequately explore individuals’ perceptions of these concepts, it will be necessary to conduct interviews with respondents who have had a variety of relevant experiences. We propose to conduct up to 60 interviews with adults ages 18 to 65, with a mix of gender, race/ethnicity, educational attainment, employment status and fields of work across respondents. Respondents will also be recruited from several locations across the United States to ensure broad representation by region. People who have participated in cognitive studies or focus groups in the past 6 months and employees of the firm conducting the research will be excluded from participating. In particular, respondents meeting the following descriptions will be recruited:

• 20 adults who are currently participating in classes or training that are not part of a degree program to prepare for a certification, license, or certificate;

• 20 adults who are currently participating in classes or training to maintain their work-related credential (e.g., certification, certificate), maintain or update their skills, or prepare for or qualify for a promotion; and

• 20 adults who are currently participating in classes or training to prepare for a different job or career.

Consultations Outside the Agency

The NHES-ATES has an ongoing consultation with a government interagency working group – Working Group on Expanded Measures of Enrollment and Attainment (GEMEnA) – that has been involved with the survey design since its inception. The GEMEnA group reviewed the goals of the concept interviews and the respondent selection criteria at its July 2015 meeting.

Recruiting and Paying Respondents

To assure that participants from all desired populations agree to take part in the interviews and to thank them for their time and for completing the interview, each respondent will be offered $25.

Participants will be recruited by AIR using multiple sources, including Craigslist ads, flyers on local community college campuses, and personal and professional contacts. We will advertise the interviews in several locations across the United States to ensure a broad representation by region (e.g., California; Michigan; Washington, D.C.). Example recruitment materials are included in the attachments. Interested individuals will be asked to complete an online screening questionnaire; the questions that will be used to screen respondents for participation are also included in the attachments. We anticipate it will take 3 minutes to complete the screening questionnaire.

Assurance of Confidentiality

Participation is voluntary. Interviewers will read a confidentiality statement to respondents, and respondents will be given the opportunity to provide or decline consent before any interview is conducted. The consent statement is provided in the attachments.

No personally identifiable information will be maintained after the interview analyses are completed. Data recordings will be stored on AIR’s secure data servers.

Estimate of Hour Burden

We expect the interviews to be approximately one hour in length. Screening potential participants will require 3 minutes per screening. We anticipate it will require 12 screening interviews per eligible participant (thus an estimated 720 screenings to yield 60 participants). This will result in 36 hours of burden for the screener, and an estimated total 96 hours of respondent burden for the full study.

Table 1. Estimated response burden for ATES concept interviews

Respondents	Number of Respondents	Number of Responses	Burden Hours per Respondent	Total Burden Hours
Recruitment Screener	720	720	0.05	36
Cognitive Interviews	60	60	1.0	60
Total	720	780	-	96

Estimate of Cost Burden

There is no direct cost to respondents.

Project Schedule

The project schedule calls for recruitment to begin as soon as OMB approval is received. Interviewing is expected to be completed within 2 months of OMB approval. After the interviews are completed, draft survey items will be developed for cognitive testing in 2016.

Cost to the Federal Government

The cost to the federal government for this study is approximately $116,000.