Part A PIAAC Cycle 2 2022 Operational Field Test

The Program for the International Assessment of Adult Competencies (PIAAC) is a cyclical, large-scale study of adult skills and life experiences focusing on education and employment. PIAAC is an international study designed to assess adults in different countries over a broad range of abilities, from simple reading to complex problem-solving skills, and to collect information on individuals’ skill use and background. PIAAC is coordinated by the Organization for Economic Cooperation and Development (OECD) and developed by participating countries with the support of the OECD. In the United States, the National Center for Education Statistics (NCES), within the U.S. Department of Education (ED) conducts PIAAC. NCES has contracted with Westat to administer the PIAAC Cycle II Field Test data collection in the U.S.

The U.S. participated in the PIAAC Main Study data collection in 2012 and conducted national supplement data collections in 2014 and 2017. All three of these collections are part of PIAAC Cycle I, in which 39 countries participated (24 countries in 2012, 9 new countries in 2014, and 5 more new countries in 2017), including the U.S., with close to 200,000 adults assessed across the 39 countries over the three data collections. A new PIAAC cycle is to be conducted every 10 years, and PIAAC Cycle II Main Study data collection will be conducted from August 2022 through March 2023. In preparation for the main study collection, PIAAC Cycle II will begin with a Field Test in 2021, in which 34 countries are expected to participate with the primary goal of evaluating newly developed assessment and questionnaire items and to test the PIAAC 2022 planned operations. This request is to conduct the PIAAC Cycle II Field Test in July-August 2021.

PIAAC 2022 defines four core competency domains of adult cognitive skills that are seen as key to facilitating the social and economic participation of adults in advanced economies: (1) literacy, (2) numeracy, (3) reading and numeracy components¹, and (4) adaptive problem solving. All participating countries are required to assess the literacy and numeracy domains [(1) & (2)], but the reading and numeracy components and adaptive problem solving domains are optional [(3) & (4)].

The U.S. will administer all four domains of the PIAAC 2022 assessment to a nationally representative sample of adults, along with a background questionnaire with questions about their education background, work history, the skills they use on the job and at home, their civic engagement, and sense of their health and well-being. The results are used to compare the skills capacities of the workforce-aged adults in participating countries, and to learn more about relationships between educational background, employment, and other outcomes. In addition, in PIAAC 2022, a set of financial literacy questions will be included in the background questionnaire.

As noted above, PIAAC is a multi-cycle collaboration between the governments of participating countries, the OECD, and a consortium of various international organizations, referred to as the PIAAC Consortium. Members of the consortium include representatives from Educational Testing Service (ETS), Westat, cApStAn, The Research Centre for Education and the Labor Market (ROA), gesis-ZUMA Centre for Survey Research, German Institute for International Education Research (DIPF), and the Data Processing Centre of the International Association for the Evaluation of Educational Achievement (IEA-DPC).

PIAAC is collaborative and international by nature and is developed through an extensive series of international meetings and workgroups assisted by OECD staff. At these meetings, the PIAAC framework for the assessment, the background questionnaire, and the common standards and data collection procedures are all developed. Expert panels, researchers, the PIAAC Consortium’s support staff, and representatives of the participating countries collaboratively develop all these aspects of PIAAC and guide the development of the software platform for uniformly administering the assessment on tablets. All countries must follow the common standards and procedures and use the same software when conducting the survey and assessment. As a result, PIAAC is able to provide a reliable and comparable measure of adult skills in the adult population across the participating countries.

In recognition of the continuing global pandemic OECD shifted the timeline of PIAAC Cycle II Field Test and Main Study. Originally, the Field Test was scheduled for 2020 and the Main Study for 2021. The first shift in timeline was to move the Field Test to 2021 and the Main Study to 2022. The second shift in the PIAAC Cycle II collection affects the field test only. The OECD developed a guidance paper entitled “Adapting the Technical Standards of the PIAAC Field Trial in the face of the COVID-19 Pandemic”. The guidance paper allows for testing critical operations aspects of survey implementation, functionalities of the tablet device, and ease of use of the device and interface used for data collection and the assessment. Therefore, the current change in PIAAC Cycle II allows for a reduced field test effort both in scope and sampling.

In terms of the change in sampling, after several months of careful monitoring and in consideration of OECD guidance, NCES has elected to conduct the Operational Field Test among volunteer participants. Monitoring involved tracking the new daily cases of COVID-19. To determine the safety of in-person interviewing in light COVID-19, the NCES data collection contractor, Westat, uses the average of new daily cases of COVID-19 per 100,000 people over the last 7 days to classify counties in green (less than one case), yellow (between 1 and 9 cases), orange (between 10 and 24 cases), and red (more than 24 cases) risk levels. Westat only considers counties with a green or yellow risk safe to conduct in-person interviewing. That is, counties with an average of new daily cases of COVID-19 per 100,000 less than 10. Currently, very few counties in the entirety of the U.S. are in the green or yellow zone and none of the 20 County Primary Sampling Units (PSUs) in the PIAAC field test sample are in the green or yellow zone, considered safe for field interviewers.

In addition, the reduced Operational Field Test shifts the timeline from April through June 2021 to June through August 2021. If local pandemic conditions exceed the safe limits for data collection as described in section A.2, the Opertaional Field Test will be delayed until August 2021 and a non-substantive change request will be submitted to notify OMB of the delay.

There are seven appendices included with this submission.

Appendix A1. Recruitment and Respondent Contact Materials. This document contains all of the materials used to recruit volunteers for the operational field test, the interviewer introduction guide for the field test and relevant recruitment and consent materials from the Main Study that are part of the testing.

Appendix A2. PIAAC Cycle II Screener used at randomly selected households in the Main Study

Appendix B. PIAAC Cycle II Case Initialization Items for Main Study

Appendix C. PIAAC Cycle II Background Questionnaire

Appendix D. PIAAC 2022 Cycle II Assessment Text

Appendix E. COVID Mitigation Protocol

By January 2022, we will submit another full package for 60- and 30-day public review. That package will contain the materials for the national sample main study to be conducted in Fall 2023.

A.1 Importance of Information

Over the past three decades, national governments and other stakeholders have been increasingly interested in an international assessment of adult skills to monitor how well populations are prepared for the challenges of a knowledge-based society. In the mid-1990s, the International Adult Literacy Survey (IALS) assessed the prose, document, and quantitative literacy of adults in 22 countries or territories, including the U.S. Between 2002 and 2006, the Adult Literacy and Lifeskills (ALL) Survey assessed prose and document literacy, numeracy, and problem-solving in ten countries, including the U.S.

PIAAC’s measurement of competencies in problem solving and of skills used in the workplace moved the survey beyond conventional measurements of literacy. These two features helped to assess the extent to which adults have acquired a generic set of skills and competencies. At the same time, PIAAC looks more closely than previous surveys at the extent to which people with low literacy levels have the basic building blocks that they need to read effectively. Data from PIAAC Cycle I increased the understanding of the relationship between adult skills and life outcomes in health, employment status, and civic engagement, which have social and economic impacts both within and across countries.

U.S. participation in PIAAC Cycle I and II is consistent with the NCES mandate. The enabling legislation of the National Center for Education Statistics [Section 406 of the General Education Provisions Act, as amended (20 U.S.C. 1221e-1)] specifies that “The purpose of the Center [NCES] shall be to collect and analyze and disseminate statistics and other information related to education in the United States and in other nations.” The Educational Sciences Reform Act of 2002 (HR 3801, Part C, Sec.153) also specifies that NCES:

shall collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations, including—(1) collecting, acquiring, compiling (where appropriate, on a State-by-State basis), and disseminating full and complete statistics (disaggregated by the population characteristics described in paragraph (3)) on the condition and progress of education, at the preschool, elementary, secondary, postsecondary, and adult levels in the United States, including data on…(D) secondary school completions, dropouts, and adult literacy and reading skills…[and] (6) acquiring and disseminating data on educational activities and student achievement…in the United States compared with foreign nations.²

In addition to being essential for any international perspective on adult literacy and reading skills, U.S. participation fulfills both the national and international aspects of NCES’ mission. By conducting PIAAC, NCES is able to provide policy-relevant data for international comparisons of the U.S. adult population’s competencies and skills, and help inform decision-making on the part of national, state, and local policymakers, especially those concerned with economic development and workforce training.

A.2 Purposes and Uses of the Data

The core objectives for PIAAC Cycle II are similar to those for Cycle I. NCES will use data from the U.S. participation in Cycle II to help continue to:

Identify factors that are associated with adult competencies;
Extend the measurement of skills held by the working age population;
Provide a better understanding of the relationship of education to adult skills; and
Allow comparisons across countries and, as PIAAC is cyclical, over time.

Additionally, information from the PIAAC Cycle II will be used by:

Federal policymakers and Congress to (a) better understand the distribution of skills within and across segments of the population and (b) be able to plan Federal policies and interventions that are most effective in developing key skills among various subgroups of the adult population;
State and local officials to plan and develop education and training policies targeted to those segments of the population in need of skill development;
News media to provide more detailed information to the public about the distribution of skills within the U.S. adult population, in general, and the U.S. workforce more specifically; and
Business and educational organizations to better understand the skills of the U.S. labor force and to properly invest in skills development among key segments of the workforce to address skill gaps.

PIAAC provides important trend information that allow for all of the aforementioned actors to track and understand changes over time, better informing future interventions and investments.

NCES will use information gathered from the Field Test to refine the screener, background questionnaire, assessment, and training and data collection materials and procedures, before their implementation in the Main Study. Specifically, the objectives of the PIAAC Cycle II Field Test are to:

Evaluate computer-assisted personal interviewing (CAPI) system applications and data processing procedures;
Test the screener and background questionnaire content, including question ordering and skip patterns;
Evaluate the reading, numeracy, and adaptive problem-solving assessment items to determine which field-tested items should be selected for the Main Study;
Test data collection procedures, including transitions between instruments and the direct use of the automated instruments by respondents;
Evaluate the interviewer training program;
Review and analyze the Field Test data, including the variability and sensitivity of questions, the consistency of edit checks, and the administration time associated with each aspect of the data collection;
Test the flow of materials and the sample data from sample selection to the delivery of the Sample Design International File (SDIF) at the end of the data collection.

PIAAC Cycle II Field Test Components

As noted in the prior section, NCES will use the results from the Field Test to improve all aspects of the content and processes of the PIAAC before the Main Study.

The PIAAC Cycle II Field test is operational and uses volunteers recruited through local agencies and online advertisements. Volunteers who meet the eligibility requirements (must be between the ages of 16 and 65) and are willing to come to Westat research facilities will participate in a test of the instrumentation and operations. To achieve a diverse sample, volunteers will be screened on age, gender, education, and race/ethnicity. Ensuring variation in education, English language proficiency and computer/tablet skills will help test functionality across volunteers with varying skill and education levels. Details of the recruitment design are provided in Part B of this submission.

Volunteer Recruitment

Volunteers will be recruited through public announcements about the opportunity to participate in the research. Interested volunteers will be instructed to call a toll free number to determine their eligibility to participate. Eligibility is determined by age and demographic characteristics used for the purposes of recruiting a diverse set of 250 cases. Field staff will schedule appointments for volunteers to come to Westat research facilities in the Washington DC area.

Field Test Administration

Once on site at the research facility, volunteers will be instructed on the nature of the study, that it is a test of survey operations. They will be given detailed information about how the session will proceed and offered a COVID mitigation summary flyer (See Appendix A). After these volunteers have been introduced to the purpose of the testing session and provide informed consent, interviewers will conduct the entire set of procedures as designed for the Main Study. As in Cycle I, PIAAC Cycle II instruments include a Screener (see Appendix A and section B.2), a set of verification items known as the Case Initialization (Appendix B), the Background Questionnaire (Appendix C) and the assessment.

Case Initialization

After administering the screener, the first step in the interview with the eligible respondent is the Case Initialization module. This module is used to verify information about the respondent before the Background Questionnaire begins, to ensure the correct person is being interviewed, that the person is eligible, and that we have the correct contact information for the person. Through several screens, the module will prompt the interviewer to ask the respondent to verify their name (or nickname), age, gender, address and telephone number. This information will be prefilled from the answers obtained in the Screener, but the interviewer can correct any answers on the screens if needed.

Background Questionnaire (BQ)

The PIAAC BQ is meant to identify (a) what skills participants regularly use in their job and in their home environment, (b) how participants acquire those skills, and (c) how those skills are distributed throughout the population. To obtain this information, the BQ asks participants about their education and training; present and past work experience; the skills they use at work; their use of specific literacy, numeracy, and information and computer technology skills at work and home; personal traits; and health and other background information. For Cycle II, the BQ will include a set of questions on financial literacy, particularly about the respondent’s financial habits and understanding of financial information. The BQ is administered as a computer-assisted personal interview (CAPI) by a household interviewer on a study tablet computer, and takes approximately 30 minutes to complete (see Appendix C for the English version of the BQ. This instrument will be administered in English only for the Operational Field Test).

Assessment

The PIAAC direct-assessment evaluates the skills of adults in four domains: literacy, numeracy, reading and numeracy components, and adaptive problem solving. These domains are considered to constitute key information processing skills in that they provide a foundation for the development of other higher-order cognitive skills and are prerequisites for gaining access to and understanding specific domains of knowledge.

The PIAAC is an adaptive computer-based assessment. Under this unique design, the computer-based instrument directs respondents to a set of easier or more difficult items based on their answers to a set of locator items. The locator items are designed to assess the respondent’s ability to complete the domain-based items and to assess his or her basic computer skills (e.g., the respondent’s capacity to use the stylus or their finger to highlight text on the tablet computer). As a result, some respondents may not complete all items in the assessment.

In this adaptive design, the instrument uses an algorithm to select assessment items for each participant. This algorithm uses a set of variables that include: i) the participant’s level of education, ii) the participant’s status as a native or non-native language speaker; and iii) the participant’s performance in the locator tasks and on the computer-based items as they advance through the assessment. The key advantage of such an adaptive design is to provide a more accurate assessment of participants’ abilities, while using a smaller number of items than a traditional test design.

At the completion of the BQ, the participant completes a brief tablet tutorial (see Appendix D) and then begins the reading and numeracy components. Upon completing the components, the participant completes the locator and, based on the locator score, the respondent is routed to the literacy, numeracy or adaptive problem solving items on the assessment.

COVID-19 Mitigation Procedures for Safe Face-to-Face Interviewing

To determine the safety of in-person interviewing, the NCES data collection contractor, Westat, uses the average of new daily cases of COVID-19 per 100,000 people over the last 7 days to classify counties in green (less than one case), yellow (between 1 and 9 cases), orange (between 10 and 24 cases), and red (more than 24 cases) risk levels. Westat ONLY considers counties with a green or yellow risk safe to conduct in-person interviewing. That is, counties with an average of new daily cases of COVID-19 per 100,000 less than 10. Westat will monitor the conditions in Montgomery County MD, where interviewing will take place. Interviewing will be suspended if rates are not within the green or yellow classification. This package proposes an Operational Field Test data collection window of July-August 2021. If local pandemic conditions exceed the safe limits described above, it will be delayed until September 2021 and a non-substantive change request will be submitted to notify OMB of the delay .

Westat has developed a detailed protocol manual safeguarding the implementation of project-specific field procedures for COVID-19 mitigation. To the extent federal and/or applicable state and/or local orders or guidance differ from these protocols, such orders and guidance will govern. Interviewers and supervisors will receive a hardcopy of this manual and a separate training session on these procedures will occur as part of field staff training. For recruitment to the operational field test, volunteers who provide an email address will be sent a a message to confirm the date and time of their interview (See Appendix A1, Section ID). Westat will provide a flyer that summarizes the Interviewer COVID-19 mitigation procedures as an attachment to this confirmation email (See Appendix A1, Section IE). The interviewer procedures manual will contain instructions for safe interviewing practices, social distancing, the use of masks and frequently asked questions on Westat’s COVID-19 mitigation protocols. Interviewers will describe to participants the COVID-19 mitigation protocols followed throughout the course of the interview. Onsite interviewers will also offer another copy of the flyer that summarizes Westat Interviewers’ COVID-19 procedures.

As originally designed, the PIAAC interview involves the exchange of the study materials between the interviewer and the respondent. For example, it would have been allowable for the tablet and show cards listing response categories to facilitate response to survey questions read aloud by the interviewer to be passed off to the respondent. The study has minimized the exchange of materials by providing paper copies of consent script to be provided to the respondent and retained by respondent (previously the consent script was on the tablet screen and the tablet was passed to the respondent to review the screen and then and handed back to interviewer). The new protocol limits the exchange of the tablet respondent to the time they complete the exercise only, and the pass will occur only after the tablet has been sanitized. Also, interviewers will now provide paper copies of show cards that remain with the respondent at all times during the interview and are discarded at the conclusion of the interview.³

Appendix E contains a copy of the Westat COVID Mitigation Procedures Manual. This document was prepared by Westat for their own in-house use across several projects. Although the PIAAC Operational Field Test does not included household visits and will instead invite people into Westat facilities to participate in the field test, much of this material will still be used to train those who will be administering the field test. It is included as a reference that will help describe to OMB and interested readers the kind of precautions that Westat is preparing interviewers to take during data collection. It will not be included in respondent contact materials.

In addition to the manual, Westat will provide the following supplies for use in the field:

nitrile gloves;
disposable masks;
sanitizing wipes or sanitizing spray with dry wipes (to use dry wipes and sanitizing spray, thoroughly dampen a clean dry wipe and use in same manner as sanitizing wipe);
hand sanitizer

A.3 Improved Information Technology (Reduction of Burden)

As in PIAAC Cycle I, technology is a large component of the PIAAC Cycle II study. The Background Questionnaire (Appendix C) is CAPI administered. The interviewer will read the items aloud to the respondent from the screen on a tablet computer and will record all responses on the tablet. Because the assessment is computer-based, the data capture system uses automated skip patterns to display the appropriate questions depending on the preceding responses. For the screener, the tablet is used to run a sampling algorithm to determine who, if anyone, in the household is eligible to participate in the study, and it will select a respondent or respondents for the BQ and the assessment.

A.4 Efforts to Identify Duplication

PIAAC Cycle I was the first study in the U.S. to incorporate a technology component among the measured skills. The international nature of the study allows comparisons of the prevalence of these skills in the U.S. versus other PIAAC participating countries. Although other international assessments examine a similar content area, PIAAC is unique in several ways, detailed here.

Content

PIAAC is a “literacy” assessment, designed to measure performance in certain skill areas at a broader level than school curricula, encompassing a broader set of skills that adults have acquired throughout life. The skills that are measured in PIAAC differ from those measured by other studies such as the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS), which are curriculum based. The TIMSS and PIRLS are designed to assess what students have been taught in school in specific subjects (such as science, mathematics, or reading) using multiple-choice and open-ended test questions.

Another international assessment, the Program for International Student Assessment (PISA), assesses literacy, functional skills, and other broad learning outcomes and is designed to measure what 15-year-olds have learned inside and outside of school throughout their lives. Besides its different target age group, PISA measures skills at a much higher and more academic level than PIAAC, with PIAAC being more focused on the extensive range of skills needed in both everyday life and in the workforce. PIAAC contains tasks not included in the school based assessments, and assesses a range of skills from a very basic level to the higher workplace level skills that adults encounter in everyday life.

PIAAC has also improved and expanded on the cognitive frameworks of previous large-scale adult literacy assessments (including the National Adult Literacy Assessment (NALS), NAAL, IALS, and ALL). The most significant difference between PIAAC Cycle II and previous large-scale assessments is that PIAAC will be administered on tablet computers and is designed to be an adaptive assessment, so respondents receive groups of items targeted to their performance levels. Because of these differences, PIAAC introduced a new set of scales to measure adult literacy, numeracy, and problem solving. Scales from IALS and ALL have been mapped to the PIAAC scales so that trends in performance can be measured over time.

Information collected

The kind of information PIAAC collects also reflects policy purpose different from the other assessments. PIAAC provides policy relevant data for international comparisons of the U.S. adult population’s competencies and skills and provides vital data to national, state, and local policymakers focused on economic development and adult workforce training.

A.5 Minimizing Burden for Small Entities

The PIAAC Cycle II Field Test will collect information from the 16-65 year-old population through households only. No small entities will be contacted to participate in the PIAAC Cycle II Field Test data collection.

A.6 Frequency of Data Collection

PIAAC is a cyclical study, intended by the OECD to be updated internationally every 10 years. During PIAAC Cycle I, three rounds of data collection were conducted in the U.S.: 2012, 2014, and 2017, as described in Table 1. The PIAAC Cycle II Field Test data collection is scheduled for 2021 while the Main Study data collection will take place in 2021-2022.

Table 1. U.S. PIAAC data collection efforts

International cycle	Data collection name	Sample	Time period
Cycle I	PIAAC 2012 Main Study	5,100 adults ages 16-65	2011-2012
Cycle I	PIAAC 2014 National Supplement	3,600 adults ages 16-74 and adults in prison (ages 16-74)	2013-2014
Cycle I	PIAAC 2017 National Supplement	3,660 adults ages 16-74	2017
Cycle II	PIAAC 2020 Operational Field Test	250 adults ages 16-64*	2021
Cycle II	PIAAC 2022 Main Study	5,500 adults ages 16-74	2022-2023

*Adults age 65-74 are not included in the 2021 Field Test due to the vulnerability of this age group to COVID-19.

A.7 Special Circumstances

The special circumstances identified in the Instructions for Supporting Statement do not apply to this study.

A.8 Consultation outside NCES

As noted above, all participating countries develop PIAAC as a cooperative enterprise. PIAAC was developed under the auspices of the OECD by a consortium of organizations. The following are the key persons from these organizations who are involved in the design, development, and operation of PIAAC Cycle II:

William Thorn, Senior Analyst, Education and Skills Directorate, Organization for Economic Cooperation and Development, 2, rue André Pascal, 75775 Paris Cedex 16, FRANCE;
Irwin Kirsch, Project Director for PIAAC Consortium, Educational Testing Service (ETS) Corporate Headquarters, 660 Rosedale Road, Princeton, NJ 08541; and
Jacquie Hogan, Study Director, Westat, 1600 Research Boulevard, Rockville, Maryland 20850-3129.

A.9 Payments or Gifts to Respondents

In recent years, in-person household-based survey response rates have been declining. Research indicates that incentives play an important role in gaining respondent cooperation. To meet PIAAC Cycle II response rate goals, as was done in U.S. PIAAC Cycle I, sampled respondents will be offered $100 via a cash card to thank them for their time and effort spent participating in PIAAC (including completing the background questionnaire and assessment).

A.10 Assurance of Confidentiality

Data security and confidentiality protection procedures have been put in place for the PIAAC Cycle II Field Test to ensure that all contractors and agents working on PIAAC comply with all privacy requirements including, as applicable:

The statement of work for the PIAAC contract;
Privacy Act of 1974 (5 U.S.C. §552a);
Privacy Act Regulations (34 CFR Part 5b);
Computer Security Act of 1987;
U.S.A. Patriot Act of 2001 (P.L. 107-56);
Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9573);
Cybersecurity Enhancement Act of 2015 (6 U.S.C. §151);
Foundations of Evidence-Based Policymaking Act of 2018, Title III, Part B, Confidential Information Protection;
The U.S. Department of Education General Handbook for Information Technology Security General Support Systems and Major Applications Inventory Procedures (March 2005);
The U.S. Department of Education Incident Handling Procedures (February 2009);
The U.S. Department of Education, ACS Directive OM: 5-101, Contractor Employee Personnel Security Screenings;
NCES Statistical Standards; and
All new legislation that impacts the data collected through the inter-agency agreement and contract for this study.

Furthermore, all PIAAC contractors and agents will comply with the Department’s IT security policy requirements as set forth in the Handbook for Information Assurance Security Policy and related procedures and guidance, as well as IT security requirements in the Federal Information Security Management Act (FISMA), Federal Information Processing Standards (FIPS) publications, Office of Management and Budget (OMB) Circulars, and the National Institute of Standards and Technology (NIST) standards and guidance. All data products and publications will also adhere to the revised NCES Statistical Standards, as described at the website: http://nces.ed.gov/statprog/2012/.

All study personnel will sign Westat and PIAAC Cycle II confidentiality agreements, and notarized NCES nondisclosure affidavits will be obtained from all personnel who will have access to individual identifiers. The protocols for satisfying the confidentiality protocols for the PIAAC Cycle II Field Test have been approved by the Institute of Education Sciences (IES) Disclosure Review Board (DRB). PIAAC Consortium organizations, including ETS and Westat, will follow the procedures set in the PIAAC Cycle II Technical Standards and Guidelines to support the data delivery, cleaning, analysis, scaling, and estimation. NCES will work closely with the DRB and the PIAAC Cycle II Consortium organizations to map out the details of the disclosure analysis plan for the PIAAC Cycle II.

The physical and/or electronic transfer of personally identifiable information (PII), particularly first names and addresses, will be limited to the extent necessary to perform project requirements. This limitation includes both internal transfers (e.g., transfer of information between agents of Westat, including subcontractors and/or field workers) and external transfers (e.g., transfers between Westat and NCES, or between Westat and another government agency or private entity assisting in data collection). For PIAAC Cycle II, the only transfer of PII outside of Westat facilities is the automated transmission of case reassignments and completed cases between Westat and its field interviewing staff. Westat will delete all PII from the Field Test files before the data is delivered to ETS and IEA for analysis. The only geographic identifiers on the files sent to ETS and IEA will be the Census region and the urban/rural classification. No data coarsening will be done with the Field Test data because these data will not be released further than ETS and IEA.

The transmission of this information is secure, using approved methods of encryption. All field interviewer tablet computers are encrypted using full-disk encryption in compliance with FIPS 140-2 to preclude disclosure of PII should a tablet be lost or stolen. Westat will not transfer PIAAC Cycle II Field Test files (whether or not they contain PII or direct identifiers) of any type to any external entity without the express, advance approval of NCES.

Specifically, for electronic files, direct identifiers will not be included (a Westat-assigned study identifier will be used to uniquely identify cases), and these files will be encrypted according to NCES standards. If these are transferred on media, such as a USB drive, they will be encrypted in compliance with FIPS 140-2.

All PIAAC Cycle II Field Test data files constructed to conduct the study will be maintained in secure network areas at Westat and will be subject to Westat’s regularly scheduled backup process, with backups stored in secure facilities on and off site. These data will be stored and maintained in secure network and database locations where access is limited to the specifically authorized Westat staff assigned to the project who have completed the NCES Affidavit of Non-disclosure. Identifiers will be maintained in files required to conduct survey operations that will be physically separate from other research data including responses to questionnaire and assessment items, sampling frame information, and log data, and accessible only to sworn agency and contractor personnel. Westat will deliver data files, accompanying software, and documentation to NCES at the end of the study. Neither respondents’ names nor addresses will be included in those data files.

The following protocols are also part of the approved System Security Plan: (1) training personnel regarding the meaning of confidentiality, particularly as it relates to handling requests for information and providing assurance to respondents about the protection of their responses; (2) controlling and protecting access to computer files under the control of a single database manager; (3) building-in safeguards concerning status monitoring and receipt control systems; and (4) having a secured and operator-manned in-house computing facility.

The laws pertaining to the collection and use of personally identifiable information will be clearly communicated to participants in correspondence and prior to administering PIAAC Cycle II Field Test BQ and assessment. Westat will provide a study introductory letter and brochure to households that will describe the study, its voluntary nature, convey the extent to which respondents and their responses will be kept confidential, and provide the NCES authorization and PRA statements. In addition, before beginning the BQ, the NCES confidentiality statement and authorization and PRA statements will be shown to the BQ respondent on the tablet screen. The introductory letter and screen shown to respondents before starting the BQ will include the following language:

The National Center for Education Statistics within the U.S. Department of Education is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002; 20 U.S.C. § 9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Individuals are never identified in any reports. All reported statistics refer to the U.S. as a whole or to national subgroups. According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this voluntary survey is 1850-0870. The time required to complete this survey is estimated to last about two hours, depending on the components applicable to household, including the time to review instructions, gather the data needed, and complete and review the PIAAC Cycle II questionnaire and exercise. If you have any comments concerning the accuracy of the time estimate, suggestions for improving this survey, or any comments or concerns regarding this survey, please write directly to: PIAAC, National Center for Education Statistics, Potomac Center Plaza, 550 12th Street, SW, Washington, DC 20202. Approval expires MM/DD/YYYY.

A.11 Sensitive Questions

The volunteer recruitment screener and background questionnaire for PIAAC Cycle II Field Test will include questions about race/ethnicity, place of birth, and household income. These questions are considered standard practice in survey research and will conform to all existing laws regarding sensitive information.

A.12 Estimates of Burden

For the PIAAC Cycle II Operational Field Test, the total response burden is estimated at two hours per respondent, including the time to answer the recruitment screener (5 minutes), background questionnaire questions (45 minutes), complete the locator and the orientation modules (10 minutes), and the assessment (60 minutes).

Table 2. Estimates of burden for PIAAC Cycle II Field Test

Data collection instrument	Sample size	Expected response rate	Number of respondents	Number of responses	Burden per respondent (minutes)	Total burden hours
Volunteers
Volunteer Recruitment Screener	500	100%	500	500	5	42
PIAAC Screener	250	100%	250	250	5	21
Background Questionnaire	250	100.00%	250	250	45	188
Locator and Orientation Module	250	100.00%	250	250	10	42
Assessment**	250	98.10%	245	245	60	245
Total	NA	NA	500*	1250	NA	293

* Duplicate counts of individuals are not included in the total number of respondents estimate.

** Assessments are exempt from Paperwork Reduction Act reporting and thus are not included in the burden total.

Table 2 presents the estimates of burden for the PIAAC Cycle II Field Test. The intended target number of assessment respondents for PIAAC Cycle II Field Test is 250, with a total burden time (excluding the assessment) of 293 hours.

Assuming an average hourly cost of $25.72⁴ for respondents, the 293 total burden hours are estimated to translate to $7,536 total burden time cost to PIAAC Cycle II Field Test respondents.

A.13 Total Annual Cost Burden

There are no additional costs to respondents and no record-keeping requirements.

A.14 Annualized Cost to Federal Government

The total cost to the federal government, including all direct and indirect costs of preparing for and conducting the PIAAC Cycle II Field Test is estimated to be $2,648,499 (see Table 3 for cost detail).

Table 3. Cost for conducting the PIAAC Cycle II Field Test

Item	Cost
Labor	$665,810
Other Direct Costs	489,252
Respondent Incentives	84,560
Overhead, G&A, and Fee	1,273,577
Salaries of Federal Employees	135,300
TOTAL PRICE	$2,648,499

A.15 Program Changes or Adjustments

Since PIAAC is a cyclical study, the PIAAC Cycle II Field Test and Main Study data collections are building on the Cycle I content and procedures. The most significant changes or adjustments result from the timing and design changes resulting from the inability to safely conduct in-home face-to-face interviewing during the ongoing novel coronavirus (COVID-19) pandemic. The original 2020 approved Cycle II Field Test featured a stratified purposive subsample of 20 PSUs selected from the PIAAC Cycle I 2017 sample of 80 PSUs. The target sample yield of 1500 was required to estimate item parameters for all newly developed assessment items and to test the stability of the trend item parameters for each tested language in a participating country. Achieving the target sample yield required a three month field period, April through June. This was true for both the original 2020 Field Test and the modified 2021 Field Test. This field period allowed for data processing and tabulation in time for inclusion in the International Consortium required deadline of August 15^th. The current Operational Field Test, reduced in both scope and number of respondents, in consideration of the ongoing COVID-19 pandemic will run June through early-August 2021. The current Operational Field Test approach tests survey implementation as well as tests the functionalities of the tablet device and the ease of use of the device and interface for adults with low levels of familiarity with tablet devices. As presented in more detail within Part B, a convenience sample of volunteers in the greater Washington DC metropolitan area will be recruited to achieve a total of 250 completed cases. Volunteers will be screened on relevant demographic characteristics: age, gender, level of education, English language proficiency and computer/tablet familiarity. Recruitment quotas will be set within each of these characteristics’ categories.

Additional changes to the program from PIAAC Cycle I to PIAAC Cycle II, beyond the changes in the Field Test, include the use of a tablet computer to administer the study instruments, modules added to the background questionnaire, and the elimination of paper booklets during the Field Test (to be included in the Main Study).

A.16 Plans for Tabulation and Publication

Because the Field Test is intended for testing operational procedures in the field only, the data from the PIAAC Cycle II Field Test will not be used to produce national estimates.

Technical Report

NCES will not produce a report for the Field Test and there are currently no plans to conduct statistical analyses of the Field Test dataset. The schedule reflects estimates of the ability to safely conduct face-to-face interviewing in light of the novel coronavirus (COVID-19). We will continue to closely monitor the average of new daily cases of COVID-19 cases in the local area. If local rates exceed the average of 10 per 100,000, the Operational Field Test will delay start until August 2021.

Table 4. PIAAC 2022 Cycle II Schedule

Dates	Activity
May-August 2021	Recruit volunteers for the Field Test
September 2020—May 2021	Prepare field test data collection manuals, forms, training materials, assessment materials, questionnaires, and train data collection staff
June 2021-August 2021*	Collect field test data – includes recruitment and assessment activities
September 2021*	Deliver field test results to international consortium
January—June 2022	Select sample for main study
August 2021—July 2022	Prepare main study data collection manuals, training materials, forms, assessment materials, questionnaires, and train data collection staff
August 2022—March 2023	Collect main study data – includes recruitment and assessment activities
May 2023	Deliver main study raw data to international consortium
January 2024	Receive data files from international consortium
January 2024 —August 2024	Produce reports

* Assumes an average of new daily cases of COVID-19 per 100,000 less than 10 in the greater Washington DC metropolitan area.

A.17 Display OMB Expiration Date

The OMB expiration date will be displayed on all data collection materials.

A.18 Exceptions to Certification Statement

No exceptions to the certifications are requested.

1 The numeracy core competency domain refers to the ability to access, use, interpret, and communicate mathematical information and ideas, in order to engage in and manage the mathematical demands of a range of situations in adult life. Examples of numeracy skills include reading an airport timetable or calculating the sale price of an item based on an advertisement. The numeracy components core competency domain, on the other hand, is aimed at whether an individual has the foundational skills to develop the higher numeracy abilities necessary for functioning in society. Examples of these basic numeracy skills include determining how many objects are shown and which number is the largest.

2 See http://www.ed.gov/policy/rschstat/leg/PL107-279.pdf for the full Education Sciences Reform Act.

3 Each respondent will receive a booklet of show cards, with the response categories printed for each applicable question in the Screener and Background Questionnaire. In the CAPI system, interviewers are prompted to provide direction to the appropriate show card. For example, see Appendix C, page 55 of the Background Questionnaire.

4 The average hourly earnings of adults derived from May 2019 Bureau of Labor Statistics (BLS) Occupation Employment Statistics is $25.72. If mean hourly wage was not provided, it was computed assuming 2,080 hours per year. Source: BLS Occupation Employment Statistics, http://data.bls.gov/oes/ datatype: Occupation code: All employees (00-0000); accessed on December 10, 2020.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	PIAAC OMB Clearance Part A 12-15-09
Author	Michelle Amsbary
File Modified	0000-00-00
File Created	2021-03-18