Supporting Statement Part A 1830-0567 30 day

1830-0567 - Measuring Educational Gain in the National Reporting System for Adult Education (June 2023)

Tracking and OMB Number: 1830-0567

Revised: 06/15/2023

SUPPORTING STATEMENT

FOR PAPERWORK REDUCTION ACT SUBMISSION

1. Explain the circumstances that make the collection of information necessary. What is the purpose for this information collection? Identify any legal or administrative requirements that necessitate the collection. Include a citation that authorizes the collection of information. Specify the review type of the collection (new, revision, extension, reinstatement with change, reinstatement without change). If revised, briefly specify the changes. If a rulemaking is involved, list the sections with a brief description of the information collection requirement, and/or changes to sections, if applicable.

The Office of Career, Technical, and Adult (OCTAE) in the US Department of Education (ED) requests clearance for an extension of the information collection for the National Reporting System for Adult Education (NRS). Title 34 of the Code of Federal Regulations (CFR) part 462 establishes procedures the Secretary uses when considering the suitability of tests for use in the NRS. The regulations¹ further the Department's implementation of section 212 of the Workforce Innovation and Opportunity Act (WIOA), 29 U.S.C. 3292, which requires that programs and activities authorized in the Adult Education and Family Literacy Act (AEFLA) (title II of WIOA) are subject to the performance accountability provisions described in section 116 of WIOA. These regulations also include procedures that States and local eligible providers must follow when using suitable tests to measure educational gain of participants for purposes of WIOA’s performance accountability provisions.

The regulations: (a) describe the Department’s review process for identifying tests suitable for identifying educational functioning levels and measuring educational gain (§§462.10 and 462.12); (b) delineate the information test publishers must include in an application requesting the Secretary to determine the suitability of such tests (§462.11); (c) identify the criteria and specific information the Department uses to determine the suitability of tests (§462.13); (d) offer test publishers an opportunity to submit for review additional tests as long as an application for review is filed with the Department by April 1, 2017, October 1, 2017, April 1, 2018, October 1, 2018, and by October 1 of each year thereafter. (§462.10); and, (e) establish a seven-year period during which a test determined as suitable would remain suitable, unless otherwise indicated by the Secretary or unless the test was substantially changed by the publisher (§462.14).

WIOA makes accountability for results a central focus of the law. It sets out performance accountability requirements for States and local programs that measure program effectiveness on the basis of student academic achievement and other outcomes. In order to help States to validly demonstrate and accurately report their annual improvements in literacy skill levels and other core indicators of performance, the Department previously established the NRS on July 1, 2000, after extensive consultation with State directors of adult education, representatives from volunteer provider agencies, directors of local adult education programs, and experts on accountability systems. The NRS standardizes the measurement of the core indicators across States and establishes procedures for collecting and reporting student outcome data to enhance its validity and reliability.

Section 116(b)(2)(A)(i) of WIOA, 29 U.S.C. 3141(b)(2)(A)(i) of WIOA establishes six primary indicators of performance that must be used to assess State performance:

• The percentage of program participants who are in unsubsidized employment during the second quarter after exit from the program;

• The percentage of program participants who are in unsubsidized employment during the fourth quarter after exit from the program;

• The median earnings of program participants who are in unsubsidized employment during the second quarter after exit from the program;

• The percentage of program participants who obtained a recognized postsecondary credential, or a secondary school diploma or its recognized equivalent during participation in or within one year after exit from the program. For those program participants who obtained a secondary school diploma or its recognized equivalent, the participant must also have obtained or retained employment or be in an education or training program leading to a recognized postsecondary credential within one year after exit from the program;

• The percentage of program participants who, during a program year, are in an education or training program that leads to a recognized postsecondary credential or employment and who are achieving measurable skill gains toward such a credential or employment; and

• The indicator(s) of effectiveness in serving employers.

For AEFLA participants, measurable skill gains are defined in 34 CFR § 463.155(a)(1)(v)(A) and (B) as one of the following:

• Documented achievement of at least one educational functioning level of a participant who is receiving instruction below the postsecondary education level;

• Documented attainment of a secondary school diploma or its recognized equivalent.

Educational functioning level gain by pre- and post-testing (i.e. the achievement of at least one educational functioning level) describes students’ improvement in literacy skills during instruction. States are required to have their local programs assess gain by administering standardized pre-post assessments to students and follow valid administration procedures (e.g., use an appropriate assessment, use different forms of the test for pre- and post-testing). See 34 CFR §§ 462.41 and 462.42. The NRS allows states to select the assessments most appropriate for their state from among those determined to be suitable under 34 CFR part 462.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The information collected will be used by the Secretary to determine the suitability of assessments submitted by test publishers for measuring educational functioning level gain by pre- and post-testing, for purposes of WIOA’s performance accountability provisions. The suitability of the assessment will be determined by evaluating the information submitted by publishers against the published review criteria of 34 CFR part 462. Only assessments determined suitable by the Secretary can be used by state and local adult education programs funded under AEFLA to measure and report educational functioning level gain by pre- and post-testing under the NRS. To date, this information has been used by the Department to review tests submitted by publishers to determine the suitability of each test to measure and report an adult learner’s educational functioning level gain by pre- and post-testing under the NRS.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision of adopting this means of collection. Please identify systems or websites used to electronically collect this information. Also describe any consideration given to using technology to reduce burden. If there is an increase or decrease in burden related to using technology (e.g. using an electronic form, system or website from paper), please explain in number 12.

Test publishers submit their applications by email. There is no increase or decrease in burden related to using this technology.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The information to be collected by the Secretary is not available from any other source and the collection of information will not duplicate any existing collection efforts.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden. A small entity may be (1) a small business which is deemed to be one that is independently owned and operated and that is not dominant in its field of operation; (2) a small organization that is any not-for-profit enterprise that is independently owned and operated and is not dominant in its field; or (3) a small government jurisdiction, which is a government of a city, county, town, township, school district, or special district with a population of less than 50,000.

The information collected will not have an impact on small businesses or other small entities.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Accountability for performance is a cornerstone of AEFLA and WIOA. In establishing a performance accountability system under WIOA, Congress wanted not only to assess the effectiveness of State performance in assessing measurable skill gains but also to help States continuously improve their performance and to optimize the return on the investment of Federal funds. Educational functioning level gain is a key performance measure for the Federally-funded adult education program for purposes of determining whether an AEFLA participant has achieved a measurable skill gain. To provide standardized and comparable performance data within and among State and local programs, the assessments being used must validly and reliably measure the content areas of interest and report those gains using the standard definition of educational functioning level gain contained in the NRS, based on applicable regulations in 34 CFR §§ 462.41-462.42 and 463.155(a)(1)(v)(A).

Without identifying suitable tests, the Department cannot assure the validity and reliability of the data being reported on the key measure of educational functioning level gain and any standardization and comparability of this measure is lost. Additionally, states are required to evaluate the past effectiveness of eligible providers under AEFLA in meeting levels of performance for the primary indicators of performance and to use those evaluations to make funding decisions. Without valid and reliable assessments to measure educational functioning level gain, and thus, measurable skill gain, states would be unable to carry out this responsibility effectively.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

• requiring respondents to report information to the agency more often than quarterly;

• requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any document;

• requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;

• in connection with a statistical survey, that is not designed to produce valid and reliable results than can be generalized to the universe of study;

• requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or that unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

Test publisher applications may include proprietary information. The agency has procedures in place to protect the confidentiality of this information.

8. As applicable, state that the Department has published the 60 and 30 Federal Register notices as required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB.

Include a citation for the 60 day comment period (e.g. Vol. 84 FR ##### and the date of publication). Summarize public comments received in response to the 60 day notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden. If only non-substantive comments are provided, please provide a statement to that effect and that it did not relate or warrant any changes to this information collection request. In your comments, please also indicate the number of public comments received.

For the 30 day notice, indicate that a notice will be published.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instruction and record keeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years – even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

We have consulted with test publishers on their ability and efforts to comply with this information collection. The Department published 60-day Federal Register notice soliciting public comment on February 22, 2023 (88 FR 10883). More than 50 public comments were received during the comment period. A memo summarizing the responses to comments is included with this request. No changes were made in response to the comments received. The Department will publish a 30-day Federal Register notice soliciting public comment.

8. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees with meaningful justification.

There are no payments or gifts to respondents.

8. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy. If personally identifiable information (PII) is being collected, a Privacy Act statement should be included on the instrument. Please provide a citation for the Systems of Record Notice and the date a Privacy Impact Assessment was completed as indicated on the IC Data Form. A confidentiality statement with a legal citation that authorizes the pledge of confidentiality should be provided.² If the collection is subject to the Privacy Act, the Privacy Act statement is deemed sufficient with respect to confidentiality. If there is no expectation of confidentiality, simply state that the Department makes no pledge about the confidentiality of the data. If no PII will be collected, state that no assurance of confidentiality is provided to respondents. If the Paperwork Burden Statement is not included physically on a form, you may include it here. Please ensure that your response per respondent matches the estimate provided in number 12.

The Department will assure that proprietary information submitted by a publisher in the process of complying with this information collection will not be shared or published outside the review process. No personally identifiable information (PII) is being collected and there are no assurances of confidentiality.

Public Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number.  The valid OMB control number for this information collection is 1830-0567.  Public reporting burden for this collection of information is estimated to average 40 hours per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  The obligation to respond to this collection is voluntary.  If you have any comments concerning the accuracy of the time estimate, suggestions for improving this individual collection, or if you have comments or concerns regarding the status of your individual form, application or survey, please contact the Office of Career, Technical, and Adult Education directly at NRS@ed.gov.

8. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. The justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

There are no questions of a sensitive nature in this application.

8. Provide estimates of the hour burden for this current information collection request. The statement should:

• Provide an explanation of how the burden was estimated, including identification of burden type: recordkeeping, reporting or third party disclosure. Address changes in burden due to the use of technology (if applicable). Generally, estimates should not include burden hours for customary and usual business practices.

• Please do not include increases in burden and respondents numerically in this table. Explain these changes in number 15.

• Indicate the number of respondents by affected public type (federal government, individuals or households, private sector – businesses or other for-profit, private sector – not-for-profit institutions, farms, state, local or tribal governments), frequency of response, annual hour burden. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable.

• If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burden in the table below.

• Provide estimates of annualized cost to respondents of the hour burdens for collections of information, identifying and using appropriate wage rate categories. Use this site to research the appropriate wage rate. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14. If there is no cost to respondents, indicate by entering 0 in the chart below and/or provide a statement.

The following burden calculations have been estimated based on OCTAE’s actual experience with test publishers submitting their assessments for review. Since 2008, OCTAE has published a notice inviting publishers to submit tests for a determination of suitability for use in the National Reporting System for adult education. Through interactions with various test publishers in the intervening years, OCTAE has learned practical information about the respondents’ burden with regard to the annual assessment review process. OCTAE used mean wage rates for General and Operations Managers (BLS occupation code 11-1021), published in the May 2021 National Occupational Employment and Wage Estimates from the U.S. Bureau Of Labor Statistics at https://www.bls.gov/oes/current/oes_nat.htm. We anticipate that no more than 15 test publishers per year will submit assessments for review during 2023-2025. Therefore, we estimate the total annual reporting burden for this collection to be 600 hours, 15 responses x 40 hours per response, as calculated below.

Estimated Annual Burden and Respondent Costs Table

Information Activity or IC (with type of respondent)	Sample Size	Respondent Response Rate	Number of Respondents	Number of Responses	Average Burden Hours per Response	Total Annual Burden Hours	Estimated Respondent Average Hourly Wage	Total Annual Costs (hourly wage x total burden hours)
Preparing and submitting assessment review package3 (private sector test publisher)	N/A	N/A	15	15	40	600	$55.41	$33,246
Annualized Totals			15	15		600		$33,246

Please ensure the annual total burden, respondents and response match those entered in IC Data Parts 1 and 2, and the response per respondent matches the Paperwork Burden Statement that must be included on all forms.

8. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14.)

• The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life); and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and acquiring and maintaining record storage facilities.

• If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of contracting out information collection services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

• Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government or (4) as part of customary and usual business or private practices. Also, these estimates should not include the hourly costs (i.e., the monetization of the hours) captured above in Item 12.

Total Annualized Capital/Startup Cost : 0

Total Annual Costs (O&M) : 0

Total Annualized Costs Requested : 0

The total for the capital and start-up cost components for this information collection is zero. This information collection will not require the purchase of any capital equipment nor create any startup costs. The information requested is routinely generated by publishers to meet the professional standards for assessment development and are therefore a part of their normal business costs. The cost of subsequent collections will vary based on the number of publishers submitting test information.

8. Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

Total annual cost to the Federal government is estimated at $91,660. Federal staff at the GS-13 and GS-14 level will work with this data for 40 hours per year at a cost of $2,660 per year, calculated in the first table below. The federal contractor cost is $89,000 per year. The cost estimate reflected in the second table is based on contractor support through which experts review the applications submitted by publishers to determine their suitability for use in the NRS accountability program. This is the primary cost to the Federal government associated with this data collection.

Year	Federal Staff	Estimated Number of Federal Staff Hours Per Year	Hourly Rate4	Estimated Annual Federal Staff Cost
2023	GS-14 GS-13	20 20	$72 $61	$1,440 $1,220
	Total 2023	40		$2,660
2024	GS-14 GS-13	20 20	$72 $61	$1,440 $1,220
	Total 2024	40		$2,660
2025	GS-14 GS-13	20 20	$72 $61	$1,440 $1,220
	Total 2025	40		$2,660
Total Estimated Federal Staff Cost for 3 Years				$7,980

Federal Cost for Contractor
Year	Total Cost
2023	$89,000
2024	$89,000
2025	$89,000

8. Explain the reasons for any program changes or adjustments. Generally, adjustments in burden result from re-estimating burden and/or from economic phenomenon outside of an agency’s control (e.g., correcting a burden estimate or an organic increase in the size of the reporting universe). Program changes result from a deliberate action that materially changes a collection of information and generally are result of new statute or an agency action (e.g., changing a form, revising regulations, redefining the respondent universe, etc.). Burden changes should be disaggregated by type of change (i.e., adjustment, program change due to new statute, and/or program change due to agency discretion), type of collection (new, revision, extension, reinstatement with change, reinstatement without change) and include totals for changes in burden hours, responses and costs (if applicable).

There are no program changes or adjustments to this data collection.

Provide a descriptive narrative for the reasons of any change in addition to completing the table with the burden hour change(s) here.

	Program Change Due to New Statute	Program Change Due to Agency Discretion	Change Due to Adjustment in Agency Estimate
Total Burden
Total Responses
Total Costs (if applicable)

8. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The information collected will not be published. It is to be used for an internal review to determine the suitability of assessments developed by publishers to measure educational gain under the requirements of the authorizing legislation (AEFLA).

8. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

We are not seeking exemption.

8. Explain each exception to the certification statement identified in the Certification of Paperwork Reduction Act.

We have not identified exceptions to the certification statement identified in the Certification of Paperwork Reduction Act.

1 These regulations can be found at the following link: https://federalregister.gov/a/2016-16049.

2 Requests for this information are in accordance with the following ED and OMB policies: Privacy Act of 1974, OMB Circular A-108 – Privacy Act Implementation – Guidelines and Responsibilities, OMB Circular A-130 Appendix I – Federal Agency Responsibilities for Maintaining Records About Individuals, OMB M-03-22 – OMB Guidance for Implementing the Privacy Provisions of the E-Government Act of 2002, OMB M-06-15 – Safeguarding Personally Identifiable Information, OM:6-104 – Privacy Act of 1974 (Collection, Use and Protection of Personally Identifiable Information)

3

4 OPM Salary Table 2023-DCB, Hourly Rates. https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/23Tables/html/DCB_h.aspx

8