Part A Final

Title II of the 2014 Workforce Innovation and Opportunity Act (WIOA; Pub.L. 113-128) provides funding to states for local programs that are designed to help adults obtain the education credentials, knowledge, and skills necessary for employment and economic self-sufficiency.¹ WIOA mandates an independent national evaluation of adult education programs funded under Title II. The national evaluation is being conducted by the U.S. Department of Education’s (ED’s) Institute of Education Sciences (IES). This request for clearance is for the National Study of the Implementation of Adult Education Under WIOA, which is a part of the national evaluation. The study will provide Congress, ED and the field with implementation information on adult education programs and practices.

The study will be descriptive and primarily involve collecting and tabulating data from surveys of adult education state directors and local providers funded under Title II. Some key findings from the provider survey will be compared with findings from the Adult Education Program Survey, an earlier national survey of providers conducted in 2003. This comparison will allow for an assessment of the extent to which adult education programs have evolved since prior to the enactment of WIOA. The study also will include the use of extant data, such as from ED’s National Reporting System (NRS). The NRS collects information on adult education programs funded, populations served, and participant outcomes achieved.

Part A. Justification

A.1. Circumstances Necessitating Collection of Information

Sec. 242(b)(4) of WIOA requires that ED carry out an independent evaluation of programs and activities funded under Title II. The implementation study described in this package is part of a set of studies that will comprise the first national evaluation. The study is funded by ED’s Office of Career, Technical and Adult Education (OCTAE) under its national leadership activities and is being conducted by IES.

A.2. Purposes and Uses of the Data

The purpose of the study is to provide Congress, OCTAE, and other federal, state and local stakeholders with information on the implementation of adult education under WIOA.

We will use data from the surveys of state directors (Appendix A) and local providers (Appendix B) plus other extant data sources to address the study’s three main research questions:

To what extent are the reforms to adult education policy and practices promoted by WIOA being implemented at the state and local levels?
Beyond those promoted under WIOA, in what other ways has implementation changed since the last federal update to adult education (Workforce Investment Act of 1998)?
What challenges are state agencies and local providers experiencing in administering and delivering adult education programming under WIOA?

Each of the study’s main research questions was operationalized as a set of more detailed research questions. The detailed research questions and the data source(s) to be used in answering them are provided in Exhibit A.1.

Exhibit A.1. Detailed Research Questions and Data Sources

Research questions	State Director Survey	Local Provider Survey	Analysis of extant data^a
Research questions	State Director Survey	Local Provider Survey	Analysis of extant data^a
1. To what extent are the reforms to adult education policy and practices promoted by WIOA being implemented at the state and local levels?
1.1. How are state agencies and local adult education providers coordinating with workforce development, postsecondary and other partners?			
1.2. To what extent do states have policies designed to encourage programming connected to workforce development and postsecondary transition?	
1.3. To what extent do adult education providers offer programming connected to workforce development and postsecondary transition?		
1.4. To what extent are adult education learners participating in programming connected to workforce development?			
1.5. What activities intended to enhance the connection between adult education, workforce development, and postsecondary education are states conducting?	
1.6. How are WIOA performance data being used by states and local providers?		
1.7. To what extent are states identifying and disseminating promising models of practice?	
2. Beyond those promoted under WIOA, in what other ways has implementation changed since the last federal update to adult education (1998 Workforce Investment Act)?
2.1. To what extent have there been shifts in collaboration with other public and private community partners?			
2.2. Have there been changes in how instructional services have been scheduled or delivered?			
2.3. Have funding levels or sources for local adult education programs changed across time? Has funding allocation changed across time?			
2.4 Have there been changes in the types of individuals served, since prior to WIOA? What services are being provided to support individuals to participate in programs and transition to postsecondary or employment?			
3. What challenges are state agencies and local providers experiencing in administering and delivering adult education programming under WIOA?
3.1. What have been the challenges in developing and sustaining collaborations with workforce development, postsecondary and other partners?		
3.2. What have been the challenges in developing and/or implementing approaches to including workforce development and postsecondary transition programming in adult education?		
3.3. What have been the challenges in supporting approaches to including workforce development and postsecondary transition programming in adult education?	
3.4. What have been the challenges for states in identifying and disseminating information on promising models?	
3.5. What have been the challenges in implementing performance accountability reporting requirements?		
3.6. What have been the challenges for states in assessing provider performance under WIOA?	

^aExtant data sources will include data from the NRS, the Adult Education Program Survey of 2003, and the American Community Survey, among others.

The surveys have been developed to address the questions shown in Exhibit A.1. Specifically, the State Director Survey covers the following topics:

Funding and grantmaking practices
Partnerships and collaborations with state workforce development partners or local workforce development boards
Technical assistance and professional development provided to local programs
State policies and procedures for implementing adult education services and requirements
How states identify promising models of adult education and literacy activities
How states assess the performance of adult education providers

The Provider Survey includes items on the following topics:

Characteristics of adult education providers
Sources of funding and expenditures
Instructional services provided
Partnerships and collaborations with other organizations
Technical assistance and professional development received from the state
How providers use data for accountability, communications and decision making
Demand and waitlists for services

The study will also collect administrative records data from states through the Provider List and Updated Provider List data collections (Appendix C). The Provider List collection will provide the study team with the provider names and contact information needed to administer the Provider Survey. Through the Updated Provider list collection, the study will collect provider-level NRS data needed to address questions regarding program enrollment and funding.² These are data that states will not yet have for the program year of interest at the time that the Provider Lists are initially collected.

To inform Congress, ED and the field on how adult education is being implemented under WIOA, the study will produce two reports—one focused on findings from the State Director Survey and the other on findings from the Provider Survey, presented within the context of state policies and practices. Further detail about the reports is in Section A.16.

A.3. Use of Improved Information Technology

The recruitment and data collection plans for this project reflect sensitivity to issues of efficiency and respondent burden. The study team will use a variety of information technologies to maximize the efficiency and completeness of the information gathered for this study and minimize the burden on respondents at the state and local levels:

The State Director Survey will be administered through a Web-based platform to streamline the response process; for example, to move the participant automatically past sections that do not apply based on prior responses and to collect data digitally with e-mailed reminders.
The Provider Survey will be administered primarily through a Web‑based platform to streamline the response process, with follow-up for nonrespondents using a mailed questionnaire and, finally, computer-assisted telephone interviews.

A.4. Efforts to Identify Duplication

There are no other federal surveys of the implementation of adult education under WIOA. We will, however, use administrative data from the NRS to limit respondent burden and the duplication of data collection efforts. For instance, we will ask the state directors during the Provider List and Updated Provider List collections to share their existing provider-level NRS data on provider characteristics, aggregate participant characteristics, and program enrollment rather than ask 2,000 individual providers to supply this information on the Provider Survey. However, we will ask state directors during the first Provider List collection to notify the collection team if they will be unable to share providers’ NRS data. Should any states be unable or unwilling to share providers’ NRS data, we will collect those data as part of the Provider Survey in those states.

A.5. Collection of Data From Small Entities

Some local providers could be considered small entities, such as school districts or community-based organizations. We have designed the study to keep the burden on these entities as low as possible while still answering the study’s implementation research questions. We will use two strategies to limit burden: (a) We will keep the survey as brief as possible, and (b) we will ask the state directors for specific information that they can provide on behalf of the providers.

A.6. Consequences of Not Collecting the Data or Collecting Less Often

This study is designed to provide information on the implementation of adult education programs under WIOA. It is an important component of the national evaluation mandated under that legislation. Failure to collect the data proposed through this study would prevent Congress and ED from assessing the progress that administrators of state and local adult education programs are making in implementing adult education under WIOA and the challenges they face. A similar survey of state directors has never been conducted, and a survey of local providers has not been conducted in more than 15 years. Without this information, ED will not be able to describe the intersection of federal, state, and local policies and programs for adult learners. ED also will be missing a valuable source of information on the challenges faced and types of technical assistance needed by state and local programs.

A.7. Special Circumstances of Data Collection

None of the special circumstances listed apply to this data collection.

A.8. Consultations Outside the Agency

A technical working group (TWG) of researchers, state directors of adult education, and local program administrators was convened in October 2018 to provide input on the data collection instruments under development for this study. Exhibit A.2 lists the TWG members who have provided guidance and input on the data collection instruments.

Exhibit A.2. TWG Members

Name	Title	Affiliation
Carol Clymer	Associate Professor of Education, Adult Education Program	Pennsylvania State University
Daphne Greenberg	Distinguished University Professor of Educational Psychology, Special Education, and Communication Disorders	Georgia State University
Amanda Harrison	Chief, Division of Adult Education	Pennsylvania Department of Education, Bureau of Postsecondary and Adult Education
Sheryl Hart	State Director of Adult Education	Arizona Department of Education
Jon Kerr	Director of Adult Basic Education	Washington State Board for Community and Technical Colleges
Kristen Olsen	Associate Professor, Department of Sociology	University of Nebraska–Lincoln
Kathleen Porter	Executive Director, Career Technical, Adult and Alternative Education	Poway Unified School District
Elizabeth Zachry Rutschow	Senior Associate	MDRC
Reecie Stagnolia	Vice President for Adult Education	Kentucky Council on Postsecondary Education

A.9. Payments to Respondents

Respondents will not receive any payment or gift for participating in the study’s data collections.

A.10. Assurance of Confidentiality

The study team is vitally concerned with maintaining the anonymity and security of its records. The project staff has extensive experience collecting information and maintaining the confidentiality, security, and integrity of survey data. All members of the study team have obtained their certification on using human subjects in research. This training addresses the importance of the confidentiality assurances given to respondents and the sensitive nature of handling data. The team also has worked with the Institutional Review Board at the American Institutes for Research (AIR) to seek and receive approval of this study, thereby ensuring that the data collection complies with professional standards and government regulations designed to safeguard research participants.

A.10.1. Confidentiality Assurance Statement to Providers

We will include the following statement on the cover of the Provider Survey:

Information collected for this study comes under the confidentiality and data protection requirements of the Education Sciences Reform Act of 2002, Title I, Part E, Section 183. Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings by state and for the nation as a whole and will not associate responses with a specific provider or individual. We will not provide information that identifies you to anyone outside the study team, except as required by law.

A.10.2. Confidentiality Assurance Statement for State Directors

Because we plan to publish state-level findings in the study’s reports, it will be possible to identify the individual who provided data on the State Director Survey. Therefore, we will include the following statement on the State Director Survey cover:

Information collected for this study comes under the confidentiality and data protection requirements of the Education Sciences Reform Act of 2002, Title I, Part E, Section 183. Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will provide state-level information on implementation of adult education. Although your name will not be released, data displayed by state will be attributed to the state agency and could be attributed to an individual respondent by readers of the report.

A.10.3. Data Protection Procedures

The following procedures are in place to protect data from all collections:

The study team will protect the identity of individuals from whom we will collect data for the study and use it for research purposes only. Respondents’ names will be used for data collection purposes only and will be disassociated from the data prior to analysis.
Although this study does not include the collection of sensitive information, the study team will provide all respondents with information on the topics covered in the surveys, how the data will be used and stored, and how their confidentiality will be maintained. The study’s goals, data collection activities, participation risks and benefits, and uses for the data will be explained in the notification letter sent to the study participants (Appendix D). This information will be included as part of the Web survey programming and will be sent with paper questionnaires that are mailed.
All electronic data will be protected using several methods. The contractors’ internal networks are protected from unauthorized access, including firewalls and intrusion detection and prevention systems. Access to computer systems is password protected, and network passwords must be changed on a regular basis and conform to the contractors’ strong password policies. The networks also are configured so that each user has a tailored set of rights, granted by the network administrator, to files approved for access and stored on the local area network. Access to all electronic data files associated with this study is limited to researchers on the AIR data collection and analysis team.

A.11. Sensitive Questions

No questions of a sensitive nature are included in this study.

A.12. Estimated Response Burden

It is estimated that the total hour burden for the list of providers, survey data collection, and additional provider-level information from the states is 929.5 hours, or 309.8 hours annually during the 3-year clearance period. This results in an estimated cost of $41,706.01 based on the average hourly wage of participants, or $13,902.00 annually during the 3-year clearance period. Exhibit A.3 summarizes the estimates of respondent burden for the two surveys and list collections. Note that the burden hours are based on the assumption that the provider-level NRS data will be collected through the Updated Provider List collection and not through the Provider Survey.

Exhibit A.3. Summary of Estimated Response Burden

Data collection activity	Respondent	Total sample size (N)	Estimated response rate	Number of respondents	Time estimate per response (in hours)	Total hour burden	Hourly rate	Estimated monetary cost of burden
Initial Provider List collection	State directors	51	100%	51	0.5	25.5	$49.78	$1,269.39
Updated Provider List collection	State directors	51	100%	51	0.5	25.5	$49.78	$1,269.39
State Director Survey	State directors	57	100%	57	0.5	28.5	$49.78	$1,418.73
Provider Survey	Local adult education directors	2,000	85%	1,700	0.5	850	$44.41	$37,748.50
TOTAL		2,159		1,859	2.0	929.5		$41,706.01
Annualized basis		720		619.7	0.7	309.8		$13,902.00

A.13. Cost to Respondents

No additional costs for respondents are associated with this data collection beyond the burden estimated in item A.12.

A.14. Estimate of Annualized Cost to the Federal Government

The estimated annualized cost to the federal government is $466,319. This estimate is based on the total data collection and reporting cost of $1,398,956. It includes costs for the development of the data collection instruments, data collection, planning and conducting the data analyses, and preparing the study reports.

A.15. Reasons for Changes in Estimated Burden

This is a new data collection. No program changes or adjustments are reported.

A.16. Publication Plans and Project Schedule

The study’s data collections will begin in June 2019 and conclude in April 2020. The findings will be reported through two reports. The first report will focus on findings from the State Director Survey, and the second will present findings from the Provider Survey within the context of state-level findings. A set of restricted-use data files containing study data will also be produced. The timeline for data collection activities and data dissemination is summarized in Exhibit A.4.

Exhibit A.4. Timeline for Data Collection Activities and Reporting

Activity	Date
Begin collection of Provider List from state directors	June 2019
Complete collection of Provider List	August 2019
Notify Provider Survey participants	October 2019
Begin Provider Survey data collection	October 2019
Complete Provider Survey data collection	April 2020
Notify State Director Survey participants	January 2020
Begin State Director Survey data collection	January 2020
Complete State Director Survey data collection	April 2020
Begin collection of Updated Provider List from state directors	January 2020
Complete collection of Updated Provider List	May 2020
State Director Survey report	May 2021
Provider report	December 2021
Restricted-use data files	January 2022

A.17. Approval to Not Display the Expiration Date for OMB Approval

The OMB authorization number and expiration date will be displayed on all data collection instruments.

A.18. Exceptions to the Certification Statement

No exceptions to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I are requested.

1 Title II of WIOA also is referred to as the Adult Education and Family Literacy Act.

2 The NRS provides public access to state-level data, but for the purposes of this study, we will seek provider-level data.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	AIR Proposal
Subject	AIR Proposal
Author	Sorensen, Diane
File Modified	0000-00-00
File Created	2021-01-15