Summary of Comments Submitted in Response to 60-day FRN and Agency Responses

Departments of Labor and Education

Information Collection Request

Workforce Innovation and Opportunity Act (WIOA)

Common Performance Reporting

Summary of 60-day Federal Register Notice (FRN) Comments and Responses

Executive Summary (Overview of Document)

The Departments of Labor (DOL) and Education (ED) (collectively, Departments) published a 60-day Notice in the Federal Register (FRN) seeking public comments on the Workforce Innovation and Opportunity Act (WIOA) Common Performance Reporting, Information Collection Request (ICR) (Joint Performance ICR) on January 19, 2017 (82 FR 6651) (OMB Control Number 1205-0526).

19 comments were received in response to the 60-day comment Notice on the Joint Performance ICR. 2 comments were duplicates, and not included. This document provides a breakup of the 17 sets of public comment’ and the Department’s responses to those comments. The Departments have organized this Summary of Comments and Responses by issues raised by the commenters, and the particular portions/forms of the ICR documents. This document primarily focuses on those comments and responses that raise substantive concerns for the WIOA performance ICRs (both joint and program-specific) and, therefore, we have not included comments expressing support or appreciation for the ICRs. In addition, the comments are mostly unique, and were not paraphrased, or shortened.

The comments are organized into 12 sections as outlined in the table of contents below. Each comment contains a comment number in the first column. The actual comment received appears in the second column. The third column provides the agencies’ response. Finally, the fourth column indicates the frequency of the comment (i.e., how many times the same or a similar comment was received) from the 60-day comment period.

TABLE OF CONTENTS (Hyperlinked to each section)

WIOA Reporting Template-9169

Annual Narrative Report

Effectiveness Serving Employers

Joint PIRL

JOINT Performance Specifications

Credential Attainment Rate

Measurable Skill Gains

Eligible Training Provider – 9171

Reportable Individual

MISC VR Comments

Miscellaneous

Irrelevant/No Action

AMENDED WIOA COMMON PERFORMANCE ACCOUNTABILITY ICR – COMMENT RESPONSES
#	COMMENT	RESPONSE	COMMENT FREQUENCY
WIOA Reporting Template-9169
	Three commenters expressed concerns regarding the funds expended. Comment as follows: Funds Expended for Career Services and for Training Services (#3 and #7): Our fiscal system collects funds expended for training services for DWG funds so we can report it on fiscal report ETA-9130. Except for this, we don’t have the information. Our workforce boards would like some guidance about what dollars are included. For example, is staff salary included, and if so, how would they determine what percentage of staff time was spent on Training vs. Career services? Reporting Element: 3 & 7 - Funds Expended/ Career Services: This may be difficult to obtain when fiscal management systems and program management systems are not the same. What is the source of the data for the report the PIRL has no cost elements Reporting Element: 4 & 8 - Funds Expended/Training Services: This may be difficult to obtain when fiscal management systems and program management systems are not the same. What is the source of the data for the report the PIRL has no cost elements?	Please refer to TEGL 10-16. Financial expenditures are not captured in the PIRL. State/Local must use their financial reporting system in order to calculate the funds expended on career and training services.	3
	Two commenters mentioned that the Funding Steam Tab: Adult and Dislocated Worker: PIRL #903 or #904 have no value = 4 One commenter asked about National Dislocated Worker Grants and whether they were only disaster recovery grants are counted?	The commenters are correct. Code Value 4 was left out of elements 903 and 904 and 918 in error. The Joint PIRL has been updated to reflect a value of 4 = Reportable Individual for Elements 903 – Adult and 904-Dislocated Worker and a value of 2 = Reportable Individual for 918 –Wagner-Peyser and 911-Job Corps. Received Services Through A Disaster Recovery Grant has been deleted from the funding stream definition for Dislocated Worker	2
	One commenter had a question to clarify Secondary & Post-Secondary education: If an individual is enrolled in both secondary and post-secondary education, which condition must they meet in order to qualify for the credential attainment rate numerator? For example, a Youth participant begins the period of participation in secondary education and achieves a secondary school diploma. The same participant becomes enrolled in post-secondary education later during the participant period. The post-secondary education period extends past the exit date but ends soon after the exit date without achievement of a recognized post-secondary credential. Does the participant need to receive a recognized post-secondary credential within 1 year after exit to qualify for the Credential Attainment Rate numerator? Does the achievement of the secondary school diploma and enrollment in education within one year after participation qualify for the Credential Attainment Rate numerator?	In this scenario, participant should be counted in the numerator since they earned a secondary school diploma and were enrolled in post-secondary education within 365 days of exit. The specifications indicate that this is a unique count for each individual that either attains a secondary diploma or recognized equivalent OR a post-secondary credential, not both. Thus if both secondary and post-secondary attainment requirements have been meet, the participant will only count once in the numerator and once in the denominator. Likewise, if neither set of requirements are met, they will only count once in the denominator. If one set is met and another not met, then the participant will count once in the numerator and once in the denominator. Participants that attain a recognized post-secondary credential during participation or within 1 year after exit are counted as positive outcomes in the numerator.	1
Annual Narrative Report
	Two commenters stated the Annual Narrative has an established 25 page limit attached. Comment as follows: Do the 25 pages include or exclude the Statewide and Local Area Reports? If these reports are included, the limit is basically exceeded. Please clarify.	The Annual Report Narrative limit refers to the State’s narrative and excludes the Statewide and Local Area Reports. The Reports may be considered addendums. The Departments are making no changes based on this comment.	2
	Two commenters stated the Annual Narrative will include the Employer Metrics, both the metric being piloted and any additional metric the State is using. It is not clear if a sentence detailing the measures and the rates are required or if the table listed under the Joint PIRL must be utilized.	The ICR indicates that additional Departmental guidance will be issued related to the Narrative report which will provide more detail regarding the requested information. However, states should anticipate that the specific items to be described by States in the Performance Report Narrative include both a brief explanation of any state specific metrics being used, as well as the employer metrics being piloted. States are also required to submit the data outlined in the Joint PIRL. The Departments are making no change based on this comment.	2
	The “Annual Statewide Performance Report Narrative” guidance in the supporting statement contains a lengthy list of potential data collection elements and an anticipated federal guidance document. Given the unknown nature of which data elements the agency will capture, the timing of federal guidance is critical in the provision of the requested items. Training and Employment Guidance Letter 07-15, which provided Workforce Investment Act (WIA) Program Year (PY) 2014 Annual Report Narrative guidance, was not published until November 4, 2015, although states had been preparing for an October 1, 2015 submission. We respectfully request that guidance be provided as early as possible so that states can provide USDOL the best possible information.	The Departments appreciate the states need for timely guidance. The passage of WIOA required the Departments to issue a higher than average volume of guidance which was prioritized based on urgency. Unfortunately, some routine guidance issuances were impacted. However, the Departments strive to issue timely information.	1
	We (commenter) don’t understand the intent in limiting the report to Titles I and III but yet requiring 8 other core partner programs to participate in its development. Please clarify this in the next publication of the JPA-ICR for a 30 day comment period to allow us to better evaluate the proposal.	There is a requirement for other core partner programs to participate in the development of the narrative. However, it is noted to participate as appropriate. Title II has an existing reporting requirement. The Departments will clarify that the narrative report required in this ICR is related to titles I and III. The core programs should provide title I and III state grantees with the relevant information and data as appropriate to satisfy the title I and III narrative reporting requirements. However, titles II is not otherwise required to develop or submit the annual narrative report specified for title I and III.	1
Effectiveness Serving Employers
	Effectiveness Serving Employers – The Departments are implementing these measures as “system measures” that cross the programs. While that is understandable at the system level, it is unclear how this will work at the local level, since these measures will eventually be used in evaluating local Board performance and they are only responsible 40 for Title I performance under WIOA §116. The Departments need to clarify how they envision reporting this data at the local level. Will it include services provided by Titles II and IV? If so, how do the Departments plan to ensure that a local Board is not subject to sanction because of lack of performance from Title II or IV partners, should that unfortunate situation arise?	The Effectiveness in Serving Employers is a shared indicator, with all six core programs sharing the outcomes of this indicator. As such, States and local areas are better positioned to provide a single point of contact to each employer, making it easier for the differences between specific core programs to become invisible and enable the programs to serve together as a unified front. Since States and local areas have adopted many different organizational and programmatic design models, the Departments recommends that States and local areas draft specific policies which determine “how” the Effectiveness in Serving Employers indicator will be tracked. When reporting outcomes, a State will report one single number for each of the two (or three) approaches it selects to report. While this shared outcome will include information from each of the core programs, the Departments recommend that States centralize the coordination of data collection and reporting into a single agency. The agency coordinating the collection and reporting is the only agency that will need to report this indicator to the Departments. States have flexibility in determining what agency is responsible for tracking these services, including the collection of the data and the setting of goals with the local workforce boards. At the completion of the pilot program, each approach will be evaluated in order to identify one standard approach to measure this indicator. The Departments plan to implement this approach by Program Year (PY) 2019. Due to the completion and evaluation of the pilot program and the anticipation of a standardized approach, this indicator will not be subject to sanctions until approximately two full program years after PY 2019. Consequently, the Departments will issue forthcoming guidance on sanctions.	1
Joint Participant Individual Record Layout (PIRL)
	One commenter made the following comment regarding a data element: Field 1401 under the Joint PIRL Tab Changes, list field 1401 to be renamed “Enrolled in Secondary Education Program” removing “at Program Entry”. We agree with the deletion of “at Program Entry” because the definition is allowed during participation. Because the new definition listed has GED and other High School equivalent indicated, the name of the field should reflect the new definition. How about, “Enrolled in Secondary Education, GED or High School Equivalent Program;” this more closely reflects the definition listed.	The term GED is proprietary (trade-marked) which provides a high-school equivalency just as other companies do. Therefore the name of this data element will not change because it already captures types of secondary education which includes a high school equivalency (e.g., GED).	1
	One commenter made the following comment regarding a data element: Data Element 923: Please clarify whether 07-Criminal Offender is only a valid option for Title II Adult Education, consistent with written guidance, or whether participants who are dual enrolled in this Title II program and another program may use this "Other Reason for Exit" code.	Yes, any program serving a person who is incarcerated may elect to use the exclusion. It’s important to note that this exclusion does not exclude a participant from all indicators.	1
	One commenter questioned the source documentation required for data elements in the comment below. What are the documentation requirements for reporting an individual with any of the barriers to employment? Individuals with barriers to employment affect the statistical adjustment model and should have a clear standard of documentation. Some indicators are self-attested and do not impact the decision to enroll the participant or which services are provided. These indicators should not require documentation. Other elements that impact decisions for eligibility for services and have performance accountability impacts should have clear documentation requirements.	Source documentation will be detailed in future data validation guidance.	1
	One commenter had questions regarding a data element as follows: Reporting Element: 905 – Youth: Why is the youth no longer a participant, but only a reportable individual? If the client failed to complete the program requirements it doesn't mean funds were not spent. Wouldn't this be a participant with a negative outcome? Don't they still want to know what kind of service funding was used?	Code value 4 was added to indicate if the individual was a reportable individual.	1
	Two commenters stated their concerns for tracking Type of Training Service Data Elements (Data Elements 1303, 1310, and 1315), comment as follows. For creating distinctions in allowable ABE/ESL training for TAA vs. WIOA DW for, we believe it is more useful to track ABE or ESL ONLY (which is allowable under Trade but not under WIOA). This would be in contrast to ABE/ESL that occurs as part of a plan that includes Occupation-based training (which is allowable under both Trade and WIOA). As a result, we'd recommend the following word changes to coding values 04 and 07: (1) coding value 07 - "ABE/ESL only - (contextualized or other) not in conjunction with Occupational Training; (2) coding value 04 - eliminating the parenthetical statement "(non-TAA funded)"; (3) Coding value 04 - adding the word "occupational" before Training / the TAA-specific Coding Value 07, we'd prefer to add the word "Occupational" to Coding Value 04(which is a global value), as well as Coding Value #7 (which is marked as specific to TAA) . It seems that adding the word occupational could have added-value in providing a useful distinction for coding value 04 (given that the element itself refers to training as a generic term), or at least causes no deviation in meaning. However, we respect limitations that may be in place for a Joint WIOA Element. If such wording change is not possible for this reason, we'd still like to; (1) remove the "non TAA funded" statement from 04 and (2) modify the coding value for 07 that is specific to TAA.	The Type of Training elements (1303, 1310 and 1315) have been revised and this suggestion was taken into consideration.	2
Joint Performance Specifications
	The Commenter listed the specifications and issues as detailed below. 15 - Employment Rate (Quarter 2) Denominator: There is nothing in the Technical Specifications to exclude self-service-only participants. 17 - Employment Rate (Quarter 4) Numerator: There is nothing in the Technical Specifications to exclude self-service-only participants. 26 - Median Earnings: There is nothing in the Technical Specifications to exclude self-service-only participants. 56 - Migrant and seasonal farmworkers: Includes dependents of farmworkers, which is different than the specs for ETA-9173. 59 - Youth in foster care or aged out of system: Only includes youth aged 14-24, this is different than the specs for ETA-9173.	In response to each issue, 15 – The Employment Rate (Quarter 2) Denominator only includes those who are participants where a valid Date of Program Entry exists. Since a reportable individual does not receive services that trigger participation and require a valid Date of Program Entry, they are not included in the denominator. 17 – The Employment Rate (Quarter 4) Denominator only includes those who are participants where a valid Date of Program Entry exists. Since a reportable individual does not receive services that trigger participation and require a valid Date of Program Entry, they are not included in the denominator. 26 – The Median Earnings Denominator only includes those who are participants where a valid Date of Program Entry exists. Since a reportable individual does not receive services that trigger participation and require a valid Date of Program Entry, they are not included in the denominator. 56- The Departments appreciate the comment. The ETA 9173 was previously revised to include dependents of farmworkers consistent with the revisions to the PIRL. 59 – The commenter is correct. The specification for the ETA 9169, Annual Report and the ETA 9173, Quarterly Report are different. The ETA 9173 will be corrected.	1
	One commenter had a concern regarding cost per participant: Cost Per Participant Career Service and Cost Per Participant Training, will include overlap. Cost per participant career service is defined as FUNDS EXPENDED (CAREER SERVICES) ÷ PARTICIPANTS SERVED (CAREER SERVICES), while Cost Per Participant (Training) is defined as FUNDS EXPENDED (TRAINING SERVICES) ÷ PARTICIPANTS SERVED (TRAINING SERVICES)). The Cost Per Participant Career Service will include any participant with a career service, however some may have also received training during this same time frame. How then would the funds be differentiated between Career and Training?	Specifications for “FUNDS EXPENDED (CAREER SERVICES)” and “PARTICIPANTS SERVED (CAREER SERVICES)” both exclude anyone who has received training so that the calculation is therefore for participants who only got career services and the training calculation includes everyone who got training (including those who also got career services).	1
	Percent Co-Enrolled is defined as Count of UNIQUE RECORDS where ((Funding Stream) and (Date of Program Entry <= end of the report period) and (DATE OF EXIT => beginning of the report period or is null)) and (FUNDING STREAM) = 1 for any other funding stream)) ÷ Count of UNIQUE RECORDS where ((Funding Stream) and (Date of Program Entry <= end of the report period and (DATE OF EXIT => beginning of the report period or is null))) X 100. Information collected on Rehabilitation, Adult Education, Youth Build, Job Corp, etc. listed in the DOL PIRL, basically these are partners not using the same reporting system, the program identification will be staff identified, not system sharing validated. This means the data are underreported for many states where systems are not shared. How are program files combined if these programs are not sharing the same 12-digit identifier, because it is not required? How are States to determine Number Co-Enrolled and Number of Participants with barriers to Employment served by each of the core programs?	The Departments understand the program identification will be staff identified through the PIRL. Grantees do not have to combine files.	1
Credential Attainment Rate
	Two commenters specified their concerns for the credential attainment rate specifications: 2 - Secondary School Diploma or Recognized Equivalent Denominator: should not include OTHER REASON FOR EXIT = 07. See TEGL 10-16, Attachment 10, page 4: “except those that exit for specified reasons listed in PIRL 923; code value #1, 2, 3, 4, 5, 6, or 7” 4 - Recognized Postsecondary Credential Denominator: same as above 5 - Numerator Totals: change “Sum of (1, 3, 5)” to “Sum of (1, 3)” 6 - Denominator Totals: change “Sum of (2, 4, 6)” to “Sum of (2 OR 4)” It’s not possible for a person to be counted more than once in the numerator because only the 1st credential is considered for this measure, but it IS possible to be counted in both denominators: The number of participants who exited and were in a secondary education program (at or above the 9th grade level) without a secondary education diploma or equivalent during the program The number of participants who exited and were in a postsecondary education or training program during the program It is not accurate to only consider the 1st credential attained. Although the PIRL does not specify that the 3 credentials be sorted by the date they were attained, logically, that is how most states will fill them in. Therefore, the 1st credential attained has a higher likelihood of being a secondary credential than the 2nd or 3rd credential. To get into the secondary numerator a person must also be employed or in postsecondary education within one year of exit. This is not true if the credential attained is postsecondary – they are automatically counted in the numerator.	The specifications have been revised to address this issue.	2
	One commenter had a question regarding the calculation of the credential attainment indicator as follows: If an individual is enrolled in both secondary and post-secondary education, which condition must they meet in order to qualify for the credential attainment rate numerator? For example, a Youth participant begins the period of participation in secondary education and achieves a secondary school diploma. The same participant becomes enrolled in post-secondary education later during the participant period. The post-secondary education period extends past the exit date but ends soon after the exit date without achievement of a recognized post-secondary credential. Does the participant need to receive a recognized post-secondary credential within 1 year after exit to qualify for the Credential Attainment Rate numerator? Does the achievement of the secondary school diploma and enrollment in education within one year after participation qualify for the Credential Attainment Rate numerator?	They may meet either condition in order to qualify in the numerator. Participants are only included in the denominator or numerator one time regardless of whether they are in both secondary and postsecondary. As it relates to a postsecondary credential, the participant does need to achieve the postsecondary credential within one year after exit in order for that to count in the numerator. Achievement of a secondary school diploma and enrollment in education within one year after participation would qualify for the Credential Attainment Rate numerator.	1
	One commenter had a question regarding the calculation of the credential attainment indicator as follows: #28 Credential Rate Denominator Calculations. The specifications should distinguish between the Adult and Dislocated Worker Program and the Youth program based on inclusion definitions discussed in guidance. Participants in the Adult and Dislocated Worker Programs in guidance can only qualify for the credential attainment rate if they receive a training service. In guidance, the option does not exist to be enrolled in secondary or postsecondary education. The definition in the report specifications includes inclusion of participants who are enrolled in secondary education or post-secondary education regardless of the program.	All participants (including those in the Adult, Dislocated Worker, and Youth programs) are included in the credential rate denominator if they are enrolled in secondary or postsecondary education regardless of program. The difference among programs comes in regarding what is considered “training.” And the specifications do indicate which types of training from PIRL element 1303 are included in the credential denominator.	1
	One commenter had a question regarding the calculation of the credential attainment indicator as follows: #28 Credential Rate Denominator Calculations. A participant's enrollment in short term training other than On-the-Job Training and Customized Training enters the participant into the credential attainment rate denominator. Many of these short term training options do not lead to a recognized postsecondary credential however meet an employer's need for talent quickly. Additionally, there are many vocational rehabilitation training services that are valuable for the individual but does not lead to a credential such as training designed to assist an individual cope with their disability. We recognize balancing the long term need for individuals through promotion of courses that lead to recognized post-secondary credentials however, see value in short term training that offer certificates recognized by employers. We propose the indicator counts only those participants who are in training or education that leads to a postsecondary credential.	Both the Act and the final rule require that individuals in training are included in the credential indicator (with the exception of OJT and customized training). Individual programs define in TEGL 10-16 what is considered “training” for their program. The Departments do not have the ability to define this indicator to only include those trainings that lead to a postsecondary credential.	1
	One commenter had a question regarding the calculation of the credential attainment indicator as follows: Credential Rate – The currently approved Credential Rate denominator specifications include anybody in secondary or post-secondary level education, including training. In Title II, where the intent of the program is generally to provide a credential, that makes sense. Likewise, it makes sense in Title IV where it is common for Participants to remain in the program for extended periods while finishing such education. However, it is very problematic in those instances of Title I where the system may not be paying for any training or education but the Participant happened to be in post-secondary education or training during the Period of Participation (POP). The issue arises with Title I Worker Participants who may be pursuing post-secondary education part-time while working full-time and who come to the system solely for job search assistance. In this situation, they are not seeking training or assistance in pursuing their education but rather are only seeking job search assistance. It is not reasonable to put these Participants in the Credential Rate denominator, when the services they receive are in no way connected to their education and when they might be well over a year away from achieving a recognized credential (and thus have no chance to be in the numerator). (commenter) has run data on this scenario and found that it is very common. Remember, Accountability is primarily a question of system intent and whether we are meeting the needs of the Participants. The law allows Participants to receive only career services under Title I and in the common scenario described above, the need of the Participant is job search assistance. If the system meets that need (i.e., helps them find a job), the system would be portrayed as having not meet a need that the Participant did not seek assistance with (i.e. education).There are a number of different ways to address this issue but we recommend adopting one of the following two approaches: 1) The easiest method is to add a third option to PIRL elements 1332: “Record 2 if the Participant was in post-secondary education during program participation but the Participant’s pursuit of this education was not being supported by the program.” Then make a similar addition to PIRL element 1401: “Record 2 if the Participant was in Secondary Education but the Participant’s pursuit of this education was not being supported by the program.” 2) A second method would be to modify PIRL elements 1303, 1310, and 1315 to include types of education such as “12 = Secondary Education supported by the program” and “13 = Post-Secondary Education supported by the program” and then modifying the Credential Rate denominator as follows: Count of UNIQUE RECORDS Where (Funding Stream) and DATE OF PROGRAM ENTRY is not null and DATE OF EXIT is within the report period and ((TYPE OF TRAINING #1 or TYPE OF TRAINING #2 or TYPE OF TRAINING #3) = (02 or 03 or 04 or 06 or 07 or 08 or 09 or 10 or 12 or 13)) or (Participated in POSTSECONDARY Education During Program Participation = 1) or (ENROLLED IN SECONDARY EDUCATION PROGRAM = 1) and OTHER REASON FOR EXIT is (00 or 07)) Both options would keep the Participant out of the denominator if their education was not directly supported by the program while preserving the useful information about the Participant’s education status so that it could be considered in the statistical models.	While the Departments recognize these concerns, the Departments believe it would overcomplicate the indicator and be difficult to differentiate between education and training supported and not supported by the program.  Education or training could be supported by the program and part of an individual’s service strategy even if it was not paid for with WIOA funds.  For example, it could be paid for with Pell grant funds. To avoid this issue, states should report on their programs’ provision of education and training as defined by the program of participation in the performance guidance issued by the Departments in TEGL 10-16.  The Departments have not incorporated the suggested changes and recommend the state take this into account as part of the negotiation process for setting expected levels of performance.	1
Measurable Skill Gains
	This commenter stated an issue specifically about the measurable skill gains indicator. 30 - Measurable Skill Gains Numerator: Includes Secondary School Diploma/or equivalency, which does not match TEGL 10-16, Attachment 10, page 6.	Attachment 10 in TEGL 10-16 is incorrect and should include secondary school diploma/equivalency.	1
	Four commenters submitted similar comments regarding field 31 for the measurable skill gains indicator. 31 - Measurable Skill Gains Denominator: Once a person enters an education or training program leading to a postsecondary credential, they will be in the MSG indicator until they exit the program. Literacy Numeracy had a limit of 3 years, will MSG have a similar limit? Measurable Skill Gains is listed as the Count of UNIQUE RECORDS from MEASURABLE SKILL GAINS DENOMINATOR where the most recent date of either (the Date of Most Recent Measurable skill gains: Educational Functioning Level (EFL) or the (DATE ATTAINED RECOGNIZED CREDENTIAL is within the reporting period and TYPE OF RECOGNIZED CREDENTIAL = 1) or the DATE OF MOST RECENT MEASURABLE SKILL GAINS: SECONDARY TRANSCRIPT/REPORT CARD or the DATE OF MOST RECENT MEASURABLE SKILL GAINS: POSTSECONDARY TRANSCRIPT/REPORT CARD or the DATE OF MOST RECENT MEASURABLE SKILL GAINS: TRAINING MILESTONE or the DATE OF MOST RECENT MEASURABLE SKILL GAINS: SKILLPROGRESSION) is within the reporting period. Divided by Count of UNIQUE RECORDS Where (Funding Stream) and ((DATE OF PROGRAM ENTRY is not null) and (DATE OF Program Entry <= end of report period) and (DATE OF EXIT is null or within the report period) and (DATE ENROLLED DURING PROGRAM PARTICIPATION IN AN EDUCATION OR TRAINING PROGRAM LEADING TO A RECOGNIZED POSTSECONDARY CREDENTIAL OR EMPLOYMENT (WIOA) in not null) and (OTHER REASON FOR EXIT = (00 or 07). If a participant is in a program for more than one program year, and the training ended during the first program year, a gain in the first and second program year is still needed in order to be a positive for this measure for each year the participant is active (not based on training dates, but participation dates). Also, if the most recent date is utilized, a date for one program year could overwrite a date for a prior program year. The extract would therefore not contain historical data of the first gain in the next program year reporting. A second issue with the MSG Denominator is that often Participants will complete their training/education and continue to receive career services. This is particularly the case for those Participants who were unemployed while in education/training or who gained significantly more valuable skills that should enable them to obtain better employment If a Participant completes their education or training in the late spring and continues to receive career services past the end of the Program Year (PY), the Participant will be included in the MSG denominator in the year that they completed their education/training (as they 1 should be) AND will again be included in the next PY Denominator even though they were not in education/training in the new PY. This is because the denominator specification does not contain a reference to completion of education/training, which would allow it to exclude those who were in education/training during their POP but not during the 2nd PY when only career services are needed. Having a Denominator that includes Participants who have no opportunity to be in the numerator undermines Performance Accountability and could have unintended impacts on service delivery. The Departments should modify the Joint PIRL to include a variable related to completion of education/training during the POP and modify the MSG Denominator specification to include this. This could be done as follows: 1) Add “Date of Completion of Education/Training” defined as the date that the Participant has concluded participating in any education or training supported by the program; and then 2) Modify the Denominator Specification to read as follows: Count of UNIQUE RECORDS Where (Funding Stream) and ((DATE OF PROGRAM ENTRY is not null) and (DATE OF Program Entry <= end of report period) and (DATE OF EXIT is null or within the report period) and (DATE ENROLLED DURING PROGRAM PARTICIPATION IN AN EDUCATION OR TRAINING PROGRAM LEADING TO A RECOGNIZED POSTSECONDARY CREDENTIAL OR EMPLOYMENT (WIOA) in not null) and DATE OF COMPLETION OF EDUCATION/TRAINING” is null or within the report period and (OTHER REASON FOR EXIT = (00 or 07).	Individuals will only be in the MSG denominator in the program year in which they are in education or training. This was an error in the ICR and will be addressed.	4
	One commenter had a question regarding the calculation of the Measurable Skill Gains indicator as follows: 6, 8, 10 - Measurable Skill Gains Denominator: 3 of the 5 denominators should include OTHER REASON FOR EXIT = 07. See TEGL 10-16, Attachment 10, page 6: “except those that exit for specified reasons listed in PIRL 923; code value #1, 2, 3, 4, 5 or 6”	The Measurable Skill Gains template has been revised only to include the count of those participants that achieved a gain, thus eliminating the issue.	1
	Five commenters made comments concerning the Measurable Skill Gains Denominator and calculation as follows: Report Item #8 for the denominator for "satisfactory or better progress report…" method of achieving a measurable skill gains requires participants to have a training type of On-the-Job Training or Apprenticeship Training or Type of Work Experience greater than 0. Work experience does not appear as a Joint PIRL data element. ETA Specific PIRL Data Element 1205 is Type of Work Experience. The data element should be added to the Joint PIRL or the indicator's specifications should differentiate applicability between DOL ETA WIOA programs and the Vocational Rehabilitation program. 4. The Measurable Skill- Gains denominator for educational function level gain does not limit the population to those who have received secondary education at or above the 9th grade level. Additionally, guidance in TEGL 10-16, TAC 17-01 and OCTAE Memo 17-2 do not include the prerequisites for participants to be an English Language Learner or Basic Skills Deficient. We request clarification between the guidance and the written specifications. The calculations provided in the Joint Guidance and the Statewide Performance Report, item description 30 & 31, do not align with the calculations provided in the WIOA Measurable Skill Gains Report Template items 1-10. #MSG 4 - Denominator for Attainment of a secondary school diploma or its equivalent -The definition does not specify that an individual needs to have been enrolled in a secondary education program at some point during the year in order to be included. #MSG 11-#-13 -This produces an overall outcome rate that will double count any individual eligible for more than one of the denominators during the reporting year. The language of WIOA clearly requires that an individual only need achieve one measurable skill gains during the year, not one per type of education/training.	The Measurable Skill Gains Report has been revised to no longer require the calculation of denominators.	5
	One commenter the following comment on the measurable skill gains indicator: The Measureable Skill Gains cohort should be based off of the participant's anniversary date to provide adequate time to make a measurable skill gain. The measurable skill gains timeframe parameters of the program year do not permit flexibility in training options for participants that may enter the program at various parts of the program year. Using an anniversary date as a deadline to achieve a measurable skill gains provides a more accurate assessment of the program's effectiveness to assist its participant's in achieving a measurable skill gains.	WIOA defines measurable skill gains as the percentage of participants who, during a program year, achieve a gain. Because WIOA defines the indicator based on program year, we do not have the flexibility to make the suggested change. States could account for participants entering late in the program year through negotiating expected levels of performance.	1
	One commenter the following comment on measurable skill gains indicator: Measurable Skill Gains is listed as a Primary Indicator of Performance, however, no guidance is provided on arriving at a reasonable negotiated goal for the measure or calculations to track actual results. 3141 (b)(2)(A)(i)(V) While Measurable Skill Gains was included in a recent webinar facilitated by WorkforceGPS, developing a reasonable goal to measure against was not among the topics included. Providing additional information on the Statistical Adjustment Model in relation to this indicator and/or guidance on developing goals is needed. 3141 (b)(3)(A)(iv)(II)(viii)	ETA will provide additional guidance on negotiating expected levels of performance for all indicators, including measurable skill gains, prior to the next round of negotiations.	1
	One commenter the following comment on measurable skill gains indicator: MSG #5-#6 & MSG #9-#10 Postsecondary Transcripts and Exams See also “Joint PIRL tab data element numbers 1807 and 1808items #108 and #110 -This appears to exclude completion of a BA or general studies AA as representing a measurable skill gains. This makes no sense. If satisfactory progress in such programs counts as measurable skill gains, so should completion of these credentials, even if completion occurs in a reporting year in which the participant never attempted enough credits during the year to reach the part-time threshold. Sometimes participant circumstances, or course offerings, or other scheduling issues will place participants in the position of needing to complete a course that are too few or too unfortunately scheduled to complete when they continue into a new reporting year.	The measurable skill gains indicator is intended to capture interim skill gains and not completion of degrees or credentials, which are captured in the credential attainment indicator. While there may be times that the attainment of a credential may provide proof that a measurable skill gains occurred, the credential can only be used to validate a measureable skill gains when it shows that one of the five skill gains types was achieved during the period of reporting.	1
	Two commenters had similar comments on: PIRL #1811- Date Enrolled During Program Participation in an Education or Training Program Leading to a Recognized Postsecondary Credential or Employment. Comment as follows: Once a date is entered in #1811, the participant is included in the Measurable Skill Gains (MSG) indicator. Issues: 1. A participant may only be in an education/training program for as little as 1 day and be expected to show a gain. 2. Once a participant is included in MSG, they're in it until they exit, regardless of whether or not they continued to be in education or training. Mixing those who've received as little as 1 day of training with those who've been in training a whole year or more, artificially lowers the MSG rate, making it a much less meaningful indicator than it could be. If we're going to all the effort to collect the data for this indicator, it should be as meaningful as possible. Suggestions, in order of effectiveness (and difficulty): 1. PIRL #1811 should be used as the beginning of the 1-year period when a participant is expected to show a MSG, and be used as the anniversary date for every 1-year period thereafter. 2. Add another PIRL element to indicate when the Education or Training Program was completed or ended, and factor this date into when a participant drops out of the MSG indicator. 3. Exclude participants who have been in an education or training program for less than a certain amount of time, perhaps 3 months. 4. Limit the number of years that a participant is held to the MSG indicator (similar to how the WIA Numeracy and Literacy measure was limited to 3 years).	1) WIOA defines measurable skill gains as the percentage of participants who, during a program year, achieve a gain. Because WIOA defines the indicator based on program year, we do not have the flexibility to make the suggested change. States could account for participants entering late in the program year through negotiating expected levels of performance. 2) Individuals will only be in the MSG denominator in the program year in which they are in education or training. This was an error in the ICR and will be addressed. 3) See comment on 1) above. 4) Participants will be included in the MSG indicator for any program year in which they participated in education or training.	1
	One commenter the following comment on measurable skill gains indicator: Measurable Skill Gains continues to be a highly problematic measure from a Performance Accountability perspective. The issue described above relating to people coming to the system solely for job search assistance but who happen to be in education not supported by the system, is also an issue with Measurable Skill Gains (MSG). Fortunately, the issue can be similarly addressed by modifying the MSG denominator specifications to either: 1) Exclude Elements 1331 and 1401 where the value = 2 (in education but not education supported by the system if the Departments go with Option 1 recommended above on the Credential Rate); or 2) Only use elements 1303, 1310, and 1315 (since they will now track secondary and post-secondary education supported by the system if the Departments go with Option 2 recommended above on the Credential Rate).	While the Departments recognize the concerns of the commenter. The Departments believe it would overcomplicate the indicator and be difficult to differentiate between education and training supported and not supported by the program. Education or training could be supported by the program and part of individual’s service strategy even if it was not paid for with WIOA funds. For example, it could be paid for with Pell grant funds. To avoid this issue, States should report on their programs’ provision of education and training as defined by the program of participation in the performance guidance issued by the Departments in TEGL 10-16.  The Departments have not incorporated the suggested changes and recommends the state take this into account as part of the negotiation process for setting expected levels of performance.	1
	Two commenters shared similar concerns regarding MSG type 3 as follows: MSG Type 3 (Transcript/Report Card) provides that a part-time student can only be considered to have achieved a Type 3 gain by successfully completing 12 hours over 2 consecutive semesters. However, if the Participant began participation in the spring, it is not possible to achieve a Type 3 gain by the end of the PY. As discussed, the purpose of a Performance Accountability system is to set expectations and determine if those expectations are met. Including people in the Denominator who have no opportunity to be in the Numerator undermines the effectiveness of the measure. Those who improperly delay enrollment until the beginning of the next PY would appear to have better results than those who better serve their Participants by getting them into appropriate training a soon as it is available. To address this, MSG Type 3 should be adjusted to be 12 hours across two semesters during the Program Year OR 6 hours during the PY if entry into the training/education program was in the 2nd half of the PY.	MSG gain type 3 was intended to capture gains over two consecutive semesters even if the semesters cross program years. The Departments will update the PIRL definition and specifications to count those who complete 12 hours over 2 semesters when the semesters cross a program year.	2
	MSG Type 2 (Achievement of Secondary Credential or its Equivalent) needs to be broadened to include other Recognized Credentials. Achievement of a Recognized Credential within the PY should always, always be considered a success even if none of the other MSG types apply. For example, if the Participant was only 3 hours from completing an Associate’s degree and the system helped them achieve it during the spring semester (i.e., before the end of the PY in June), that should be considered an MSG. It should not be possible to count as successful in the Credential Rate measure and yet not count as successful in MSG, since MSG is supposed to be an interim measure of progress towards the ultimate result (i.e., the Recognized Credential). If the intent is that achievement of a Training Milestone (MSG Type 5) is intended to address this situation, then it is not clear why MSG Type 2 would be needed since achievement of Secondary Credential or its Equivalent would also clearly also be a milestone. Even if that is the intent, we don’t support addressing the achievement of a non-secondary diploma/equivalent through the MSG Type 5 because it would not apply to AEFLA Title II participants who are in integrated AEFLA and occupational skills training and who achieve a Recognized Credential other than the secondary diploma/equivalent. This leads to a broader issue of misalignment in MSG as it relates to Title II, which limits the types of gains that it recognizes to Type 1 (Educational Functioning Level Gain) and Type 2 (Achievement of Secondary Credential or its Equivalent). While we certainly agree that the vast majority of the time, gains achieved by Title II Participants will be of the Type 1 and Type 2 variety, we note that WIOA allows for other AEFLA activities, such as workplace AEL which may be a shorter-term and more focused activity – one that does not use an assessment system that is compatible with the EFL gain type. Again, it is not reasonable to include a Participant in the Denominator by virtue of participating in an allowable activity consistent with the intent of the program (in fact supported by the program), if the activity has no way of resulting in a recognized gain. The solution is simple: the JPA-ICR MSG specifications must be equally applied to all programs. If a training/education activity is allowable under Titles I, II, and IV, its’ outcomes must be reported the same way for each program. Outcomes should be calculated at the Participant level and then aggregated by program. In the integrated system that WIOA provides for, the program that pays for the service should not dictate whether an outcome (even an interim outcome like MSG) is considered successful in the Common Performance Indicators provided for under WIOA §116. The law creates two exceptions to Common Performance Indicators: 1) It considers Youth enrolled in education Post Exit to be successful outcomes; and 2) It excludes Title III-only Participants from the Credential Rate and MSG outcomes. If Congress had intended MSG to apply differently based on funding source, they clearly showed that they knew how to make that happen by virtue of creating the Wagner-Peyser exclusions and Titles I Youth alternate measures.	The measurable skill gains indicator is intended to capture interim skill gains and not completion of degrees or credentials, which are captured in the credential attainment indicator. While there may be times that the attainment of a credential may provide proof that a measurable skill gains occurred, the credential can only be used to validate a measureable skill gains when it shows that one of the five skill gains types was achieved during the period of reporting.	1
Eligible Training Provider – 9171
	Three commenters shared the same concerns regarding the elements listed below. #101 Type of Entity - The reportable values are: 1 = Higher Ed: Associate’s Degree 2 = Higher Ed: Baccalaureate or Higher 3 = Higher Ed: Certificate of Completion 4 = National Apprenticeship 5 = Private Non-Profit 6 = Private For-Profit 7 = Public 8 = Other For Data Element 101, Type of Entity, duplicative code values are provided which is confusing. Community colleges can be appropriately recorded as code value 1, as the majority of the credentials awarded are Associate’s degrees. Community Colleges can also be appropriately recorded as code 3, as the majority of the credentials awarded are a community college certificate of completion. RTI would fall under National Apprenticeship, and often Community College. This situation expectedly will lead to inconsistent reporting by states. In addition, common occupational training providers are not specifically included among those listed for this data element, and their contribution to workforce development in this country should be identified and reported. Two such providers are private vocational/career schools and two-year private/technical schools. Also, WIOA makes allowance for secondary schools as potential eligible training providers when their diplomas are offered in conjunction with occupational skills training (et al), but they are not specifically captured via the code values. Below is a proposed revision to this data element that accounts for the above-cited situations. Data Element 101: Type of Entity (reference: 20 CFR 680.410) 1 = Private Vocational or Career School (awards below associate degree) 2 = Community College 3 = Two-year Private Technical School (awards associate degrees) 4 = Four-year College or University 5 = Registered Apprenticeship Sponsor/Provider 6 = Secondary School 7 = Public Adult School with Occupational Program 8 = Other Private Non-Profit Provider 9 = Other Private For-Profit Provider 10 = Other Type of Provider When an institution is a Private For-Profit that leads to an associate’s degree what value is to be reported?	The commenter suggested a revision to the types of training entities to further clarify entity types and requested clarity on how entities should report. The ETP performance report aligns with the types of training entity defined in WIOA Section 122 (a) (2) and 20 CFR 680.410 and therefore the Department declines to make adjustments to the categories. Providers should select the option that best represents their entity in the event that the institution could be in more than one category. Providers may report “10 = Other Type of Provider” if their category is not represented in the list. The Department will remind states in subsequent guidance that providers must submit a performance report by approved program. While the type of entity is unlikely to change, the type of approved program may differ such as community college being a provider of Apprenticeship instruction. The Department has made no changes as a result of this comment.	3
	Four commenters had the same comment for data element 103 as follows: Data Element 103, Program of Study – by potential outcome, multiple code values may apply to any one program. Of course, this situation is not problematic if states may submit multiple code values for this data element. Otherwise, the situation is problematic. For example, a community college certificate program (or an associate’s degree program, etc.) may also have the potential outcome of an industry-recognized certificate (or of a certification or a license recognized by the state or federal government, etc.) Further, some programs of study are not clearly captured via the code values, e.g., private vocational/career school training programs and community college non-credit course series (“programs”) and credit certificate programs. These are distinct offerings which are the most commonly utilized by WIOA ITA participants in Maryland and likely many other states. In light of the above, a proposed revision to this data element is provided below, which assumes Multiple code selection, but is not based upon it Community college non-credit completion certificate College/university credit certificate below a baccalaureate (e.g., lower division collegiate certificates) Private vocational/career school certificate Associate’s degree Baccalaureate degree Secondary school diploma/.GED concurrent or combined with occupational training, et al Industry-recognized certificate or certification State or federal license Measureable skill gains leading to a credential Measureable skill gains leading to employment Employment Registered apprenticeship certificate of completion	States are required by the WIOA statute and final rules to establish and implement ETPL processes to approve ETPs to provide specific programs of study. Each of these programs of study must be reported on. For providers that are providing more than one approved program of study, states must report on these separately in order to comply with the requirements and ensure that these programs are accurately represented. This specific element should have only one code entered, which is based upon the primary potential outcome for the program at the time of its eligibility determination.	4
	Three commenters the same comment regarding the data elements below. #115 - All individuals: Median Earnings – This is consistent with all reporting; however, #137 – All Individual Average Earnings (Q2) and #138 -- All Individual Average Earnings (Q4) – This is not statutorily required and this is a totally different calculation than #137. No other program has to report this, so why is this required for ETPL reporting? The general public is interested in whether or not a program leads to employment and the wages of that employment, the quicker the better. The clarity and quality of that information can be accomplished with Data Element 115, which displays the earnings of all individuals that have completed the training and obtained employment within six months’ time. Including additional wage information (Data Elements 120, 137 and 138) will cause confusion for potential students. To minimize the burden of collection and reduce confusion of the consumers, it is recommended to eliminate 120, 137 and 138 which are additional elements not required by law.	The Departments acknowledge that the average earnings calculation is not a required performance metric under WIOA. However, the Departments has determined that this additional information will assist participants in choosing employment and training activities, and providers. Moreover, this does not require any additional data collection as it is simply a different calculation with the same data. Under WIOA section 185(a)(2), recipients of WIOA title I funds must submit such records and reports “as the Secretary of Labor may require regarding the performance of programs and activities carried out under [WIOA title I].” Therefore, the Departments are making no revision in response to this comment. Element #120 has been changed along with the other “non-WIOA completer” outcome elements to request information on WIOA exiters in order to more directly align with the statute, and has also been renumbered #141.	3
	Two commenters had the same comment on data element #123 – Cost Per WIOA Participation Served (to be completed by states) – Can program accounting systems provide this information considering that this report is not by the provider it is by the program? This may be hard to obtain because fiscal management systems and program management systems are not the same.	The previous name was slightly unclear and the Departments have added the word “numerator” to the name in order to clarify that this is one part of the calculation of the Cost per WIOA participant. The Departments clarify that the definition of this element is the Total ITA funds spent on WIOA participants in this program, and is to be completed by the state.	2
	Three commenters had the same comment on data element #128 – URL of Training Program – This is a lot of maintenance at the program or provider level. This data will need to be updated at least quarterly or biannually to ensure URLs are correct. Each time a program is removed, the URL will need to be removed as well. This is burdensome and will take significant time to ensure that the data are correct.	The specifications provide options for recording the URL of a program. States may record the URL of the program-specific webpage for training seekers to find more information on program; if a program-specific page is not available, providers may record the URL of a list of all programs for the provider, or leave the field blank if no URL is available. The ETP report is provided annually; there is no requirement that the records be updated more frequently. States approve eligible training providers in accordance with WIOA Section 122 and 20 CFR 680, and manage the provider process on a regular basis. The Departments recommend that states develop a mechanism for capturing the required reporting information from an institution through the regular provider approval processes. The Departments are making no revisions as a result of this comment.	3
	Three commenters had the same comment on data element #131 – Program Length (Clock/Contact Hours) – How are clock hours and credit hours differentiated?	The Departments have added the following language to the definition: “Clock hours are the total number of actual hours per week a student spends attending class or other instructional activities that count toward completing a program of study.” This clarifies how clock/contact hours are different than credit hours, which are based on the number of credits a student receives for enrolling in (and successfully completing) a given course.	3
	Three commenters had the same comment on data element #135 – Name of Associated Credential –The 75 character limit is a concern. Some credentials have far more characters than the allowable limit.	The Departments are changing the character limit on this element to 250.	3
	Three commenters had the same comment on data element #136 – Reciprocal Agreements with Other States (to be completed by states) – Are all states with reciprocal agreements to be listed or only those in which the specific program is included in a reciprocal agreement? Needs to be clarified. How is this to be implemented?	These reports are submitted by the states to DOL, and therefore states can choose the manner in which they collect this data. Therefore, providers do not need to report this information to the states if the states choose to append the information themselves	3
	Three commenters had the same comment on data element #139 - O*NET-SOC Code Associated with Program #1, 140 - O*NET-SOC Code Associated with Program #2 and 141 - O*NET-SOC Code Associated with Program #3 –Concern is that for existing programs on the ETPL, an SOC code is needed. This is a huge time burden to obtain and input the missing SOC codes, not previously required.	The Departments recognize the concern related to collecting data retroactively. The Departments recommend that SOC codes, which are publically accessible, be collected prospectively during through the state’s provider approval process. WIOA Section 122, and 20 CFR 680 require that States develop a policy for approving eligible training providers. In accordance with 20 CFR 680.460 the Governor’s policy must provide for a continued eligibility determination at least every two years which requires the State to review the provider’s eligibility information to assess the renewal of eligibility. Renewal of eligibility affords the opportunity for providers to submit updated information regarding the programs under consideration for approval. The Departments are making no revision as a result of this comment.	3
	One commenter the following comment on functionality of WIPS and ETP reporting: On page 8 of the Supporting Statement, it is stated that the States will submit their ETP data through DOL WIPS in a comma delimited file. Is WIPS prepared to take the file? WIPS has failed to produce appropriate reports for both quarter 1 and quarter 2 reporting. If the system is not functioning in time, will states be held accountable for the information, while the system is being corrected?	The Department of Labor (DOL) officially launched the Workforce Integrated Performance System (WIPS) on October 1, 2016. The system is ready for users to submit data. DOL anticipates that there will be no issues related to reporting under WIPS however states will not be held liable in the event of a documented system issue which prevents reporting. DOL continues to update and revise WIPS. States should access https://www.doleta.gov/performance/wips/ for the most recent WIPS information. The Department is making no revision as a result of this comment. The department will provide additional guidance on ETP reporting once the amendments to this ICR are approved.	2
	One commenter the following comment on ETP data element as follows: We see great potential in the use of the CIP taxonomy (Data Element 104), first as a stand-alone for direct comparison of similar training programs, and ultimately as a comparative tool for outcomes when cross matched against ONet or SOC codes. The adoption of and improvements to the fidelity of CIP coding will be a key factor. Revisions and updates to CIP codes resulting in updated CIP tables may complicate updates to current training program data. We expect this to be a bumpy ride at first.	The Departments recognize the concern related to the challenges of incorporating a new data element. The Departments are available to offer technical assistance regarding reporting to states encountering any difficulties or concerns. The Departments are making no revision as a result of this comment.	1
	One commenter expressed their concern regarding the burden and challenge of collecting student data elements as detailed in the comment below: The All Student data elements present a challenge, specifically elements: 113, 114, 115, 116, 118, 119, 120, 121, 122, 137 and 138.	While the Departments recognize that some elements may be new and represent a challenge, the elements requested were developed to meet the requirements of the WIOA statute and final rule. Elements #118, 119, 120 have been changed so that “non-WIOA completer” outcome elements now are “WIOA exiter” outcome elements in order to more directly align with the statute. These elements were also renumbered 139, 140, and 141 to accommodate the updated format. Elements 121 and 122 were removed.	1
	One commenter suggested the following change for a specific data element as detailed in the comment below: We respectfully request that the word “individuals” in the definition for data element 117 be replaced by “WIOA participants” to align with #5 of the Primary Indicators of Performance which report the results of the indicators with respect to all WIOA participants in the program of study.	The Departments have revised the language to better reflect the individual being served.	1
	One commenter expressed concern regarding burden in the comment below: The included statement on page nine of the supporting statement for OMB Control No. 1205-0526, The ETP Performance Report must report the results of the above indicators with respect to all WIOA participants in the program of study, establishes that results for the listed indicators be reported for a second group consisting of WIOA participants. Data elements 118-122 create an unnecessary third group by singling out non-WIOA completers. The report now requires performance information on all individuals (combination of WIOA and non-WIOA), WIOA participants and non-WIOA completers.	These reports are required by both the WIOA statute and final rules to have outcomes on All Individuals and WIOA exiters. The Departments have made changes to promote consistency by changing the elements requesting information on “non-WIOA completers” (elements # 118, 119, and 120) to request data on “WIOA exiters” and removing elements #121 and #122. This collection further reduces the burden of reporting on these outcomes by using the number of exiters as the denominator rather than requiring separate denominators for each of the performance indicators.	1
	One commenter suggested the following change for specific data elements as detailed in the comment below: Pennsylvania’s training provider system is built around the actual physical training location of the program. However, data element 127 requests only the training provider's main location, which is not student friendly and will result in training choices to appear centralized rather than being available throughout the state as WIOA intends. The actual physical location is more helpful to students in making an informed choice regarding their training. Therefore, we respectfully request that data element 127 be changed to the physical location of the training.	As the system develops, we may consider adding the option to provide additional physical locations. At this time, states should report the primary location for each program of service offered by a training provider. The Departments are making no revision as a result of this comment.	1
	One commenter suggested the following change for specific data elements as detailed in the comment below: ETA 9171 data elements 139, 140 and 141 require three different occupational (O*NET-SOC) codes for which the program prepares students. The current data element names cause confusion by including Program #1, Program #2 and Program#3. These elements should be requiring the collection of 3 occupational codes that the one program of study leads to rather than identifying it as Program #1, Program #2 and Program #3.	The Departments agree that data elements 139,140, and 141 should be clarified to illustrate the ability to report up to three related occupations. The Departments will make the revision to add “Occupation.”	1
	Using the term “individuals” and consistent use of that term to address the combination of WIOA and Non WIOA students will provide clarity and eliminate confusion. Using the term “participants” and consistent use of that term to address WIOA funded students will provide clarity and eliminate confusion. Using the terms “completed/completer” instead of “exited/exiter” to address the completion of training will provide clarity and eliminate confusion of exiting from WIOA.	The Departments are changing elements 117, 115, and 120 to more clearly align these definitions.	1
	One commenter expressed concern regarding burdensome. Throughout the regulations, it is mentioned that the Departments recognize the contribution of ETPs that may serve smaller populations and acknowledge that suppression standards may limit data. It seems unnecessary and burdensome to expand the requirements for reports where many will not be able to share useful data.	The Departments note that standards relating to the suppression of data apply to data made publically available. All data, regardless of the ultimate application of suppression standards, must still be reported to comply with the statute.	1
	One commenter expressed concern regarding burden in the comment below: The All Student data elements present a challenge, specifically elements: 113, 114, 115, 116, 118, 119, 120, 121, 122, 137 and 138.Are early implementation states finding that providers do not apply when faced with ETP reporting challenges? Does ETA have data regarding loss of provider due to implementation of ETP reporting requirements?	These reports were developed to meet the requirements of the WIOA statute and final rule. Elements #118, 119, 120 have been changed so that “non-WIOA completer” outcome elements now are “WIOA exiter” outcome elements in order to more directly align with the statute. These elements were also renumbered 139, 140, and 141 to accommodate the updated format. Elements 121 and 122 were removed. Data has not been collected on the loss of providers due to ETP reporting requirements. For response on the collection of elements 137 and 138, see comment 55.	1
	One commenter suggested the following change for a specific data element as detailed in the comment below: For Data Element 101, Type of Entity, duplicative code values are provided which is confusing. Community colleges can be appropriately recorded as code value 1, as the majority of the credentials awarded are associates degrees. Community Colleges can also be appropriately recorded as code 3, as the majority of the credentials awarded are community college certificates of completion. Occupational training providers are not included among the list of data elements for example private vocational/career schools and two-year private/technical schools. WIOA allows secondary schools as potential eligible training providers when their diplomas are offered in conjunction with Occ Skills training but this is not captured via the code values. It should be noted the Occupational Skills is the number one type of training for Adult & DW participants here in Nevada. For example: If you review element #1303, 1310 & 1315 to and including the Code Value: Other Occupational Skills Training and NOTE: Code 06 (Other) should only be utilized in rare instances when other codes are clearly not appropriate..	The Departments are making edits to the type of training elements in the WIOA PIRL that will allow for the option of selecting “Occupational Skills Training” as type of training. In terms of the example below, the type of training elements are being revised and will be included in the amended ICR.	1
	One commenter suggested clarity on the reporting of a data element: Reporting Element 137 - All individuals Average Earnings (Q2) and #138 (Q4) – Why is this required for ETPL reporting? Q-4 median earnings is not a WIOA core measure.	The Departments acknowledge that the average earning calculation is not a required performance metric under WIOA. However, the Departments have determined that this additional information will assist participants in choosing employment and training activities, and providers. Under WIOA section 185(a)(2), recipients of WIOA title I funds must submit such records and reports “as the Secretary of Labor may require regarding the performance of programs and activities carried out under [WIOA title I].” Therefore, the Departments are making no revision in response to this comment.	1
	One commenter suggested clarity on the following data elements: Exiter Data Elements (106, 108, 110, 113, 114, 115, 137, 138) in ETA-9171 – how is the "withdrew" portion of this data element defined? Is this the first class frequency period (week, month, quarter, semester, year, etc.) that the enrollee is no longer enrolled or does this classification require a formal declaration and supporting documentation as a transfer or completion would? How, specifically, should community colleges – who have frequent departure and return of students in any given program – be asked to track and report this date when providing data on students for eventual reporting? Will the point of withdrawal be different by institution type?	The Departments recognizes the variations between institutions and programs. The provider should apply their own institution’s standards for withdrawal from the program to this element. The Departments are making no changes as a result of this comment.	1
	One commenter raised concern regarding earnings data elements and WRIS in the comment below: Completers: Median Earnings Data Elements (115, 120) in ETA-9171 – are WRIS out of state numbers to be included in these calculations? Counts of identifiers for completers are fine for employment rate calculations to simply add to the total, but when looking into median wages this would currently require manual insertion of wage amounts into median calculations to find the correct amount depending on the correct interpretation of WRIS regulations. Current interpretation of WRIS regulations prohibits the storage or usage of WRIS wage data from other states alongside in-state data to build a "national database." Manual inclusion of WRIS wage data would put an undue burden on the agency in order to complete accurate, verifiable calculations for eligibility determinations and reporting. In addition to other issues, the naming of data element 115's Data Element Name is not descriptive of what the element itself is looking for. Element 120 specifies "in the second quarter after exit," but 115's description name omits this.	Wage data obtained through the WRIS may be included in the determination of median wages for the purposes of required performance reporting. Allowable uses of confidential UC information, including performance reporting as required under WIOA are explained in 20 CFR 603, and guidance on the use of UI wage data is further clarified in Training and Employment Guidance Letter No. 7-16. Utilizing data obtained through the WRIS for the purpose of required reporting does not constitute building a “national database” and therefore does not violate the WRIS agreements. States must comply with storage, retention and destruction requirements of the WRIS data sharing agreement where they have obtained data through the WRIS for the purposes of performance reporting.	1
	One commenter expressed concern and a revision to the credential data elements in the comment below: Obtained a Credential Data Elements in ETA-9171 – Obtained a Credential (116, 117) data elements versus Credential Rate (121, 122) data elements. Is there a reason for the deviation from naming convention between these three sets of data elements? The definition fields seem to match. Are the individuals attending programs with the ultimate goal of receiving only a secondary school diploma or equivalent still to be counted in denominator values along with those who are planning to go on to post-secondary/career specific training that had the HSD/Equivalent prerequisite?	The Departments have made edits to align the credential elements to all use “credential attainment.” Individuals attending programs with the goal of receiving a secondary school diploma or equivalent are counted in the denominator for credential attainment, as the denominators used for this report are “Total Number of Individuals Exited” and “Total Number of WIOA exiters”.	1
	One commenter had questions regarding the reporting periods as detailed in the comment below: Reporting period (ETP) – the reporting period is presumably a default for wages and employment % to be a quarter period as there is a relative date difference of 2 and 4 quarters from a completion period and thus looking at a period greater than a quarter would add complexity the relative time calculation, expanding it to account for different quarters within the reporting period. What dictates the first required reporting period for an ETP program? Program application date? Program approval date? If the reporting period is or becomes (as in the case of yearly reporting as a combination of the quarters that make up that period) greater than one quarter and looking at quarters relative to that timeframe, are the periods examined treated as one reporting period with the end of that 6 month or 1 year etc. period dictating which quarters are used as the 2 and 4 quarter from exit/completion or are the relative quarters utilized to check for employment and wages to be relative to each specific quarter that makes up the reporting period?	Reporting periods for the indicators of performance (employment 2nd and 4th quarter after exit, Median Earnings, Measurable Skill Gains, and Credential Attainment) should align to those used in reporting for annual reports submitted for the WIOA Annual report (ETA 9169). This means that the data is reported in accordance with when each indicator becomes available. ETA has provided a chart that clarifies the periods for reporting on each of the indicators on the DOLETA Performance Reporting page.	1
	One commenter expressed concern regarding burden in the comment below: Reporting Burden – The extracts required for eligible institutions to apply for initial eligibility as well as maintain their eligibility through subsequent reviews and additional consumer and federal reporting will place burdens on institutions additional to their other federal reporting requirements.	WIOA Section 122, and 20 CFR 680 require that States develop a policy for approving eligible training providers; the Governor’s policies for eligibility determination are not addressed in this ICR. WIOA Section 116 and 20 CFR 677.230 require that the providers deemed eligible submit performance reports annually to the State. The Departments have determined that the additional information will assist participants in choosing employment and training activities, and providers. Under WIOA section 185(a)(2), recipients of WIOA title I funds must submit such records and reports “as the Secretary of Labor may require regarding the performance of programs and activities carried out under [WIOA title I].” Therefore, the Departments are making no revision in response to this comment.	1
	One commenter questioned the calculations of performance indicators regarding data collected by elements detailed in the comment below: Undeclared/dual credit/piecemeal course students at Community College institutions – How should dual credit high school students taking college credit courses be counted in the total counts and (denominator) data elements? Fields involving Total and Exiter component – including employment and wage elements – (105, 106, 113, 114, 115, 137, 138) could be interpreted to include undeclared, dual credit, or piecemeal course students, namely at community colleges, due to the phrasing in element 105: "any individual who was enrolled in a course that is part of a program of study within a quarter." These courses may be part of multiple programs, but the student not directly associated with one until the point that they declare their major's program. At what point are these students to be included in data provided by institutions, and thus affect performance calculations?	States should report those individuals who are enrolled in an approved program of study and have enrolled in at least one course in that program.	1
	One commenter requested clarity on the data elements in the comment below: #129/130 – Out-of-Pocket Costs – Variations in program requirements will require that this be calculated by individual program. Variations in preparation by student result in no true “normal time to completion.”	The Departments agrees that the Out-of-Pocket Costs should be calculated by individual program. Performance reports should be submitted by each approved program. The normal time to completion is the institution’s determination of the time the program will take to complete (i.e. four years). The Departments are making no revisions as a result of this comment.	1
	One commenter requested clarity on the data elements in the comment below: #137/138 - All Individuals: Average Earnings – Community colleges have great difficulty gathering employment and wage data for students, as numbers are often lower than the threshold allowing the Maine Department of Labor to provide the figures, particularly if broken down by program of study.	The Departments recommend that providers work with their States to develop solutions for obtaining employment and wage data for students. The Governor may designate one or more state agencies to assist in overseeing the ETP performance and facilitating the production and dissemination of reports, which includes facilitating data matches between ETP records and UI wage data, as detailed in 20 CFR 677.230. The Departments agree that in accordance with WIOA Section 116(d) (6) (C) data is not required to be disaggregated when “the number of participants in a category is insufficient to yield statistically reliable information or when the results would reveal personally identifiable information about an individual participant.” However, this restriction should not impact the state’s ability to obtain the data.	1
	Two commenters mentioned the requirement to collect data on individuals who are not WIOA participants puts a burden not only on the eligible training provider (ETP), but also on the agency tasked with collecting the data. Many ETPs are not currently reporting to a state agency, and as such would need to negotiate data sharing agreements, as well as develop systems by which this data may be reported to the state agency tasked with its collection. Additionally, we have concerns that many ETPs do not have access to labor data requiring them to use "self-reported", and therefore less reliable, wage data than would be available using data match. As others have mentioned, we believe that the workload required to report data would serve as a deterrent to many potential ETPs.	The Departments recognize these reporting requirements may be challenging for those who have previously not reported on ETPs. However, these reporting requirements are required by statute. The Departments have worked to develop these definitions to both meet the requirements of the law and reduce burden on states and ETPs. The Departments encourages States to work with ETPs to assist in reducing the burdens of these requirements in whatever ways they can, such as conducting wage matches and using information from provider’s eligibility determinations to complete reports when possible.	2
	One commenter had concerns regarding the burden of collecting certain data elements in the comment below: While we find many of the non-WIOA and All Student data elements challenging, we submit that the program specific data as exemplified by elements 129 and 130 are particularly burdensome. For example, many ETPs already report a general tuition and fee schedule. However, program specific book, tool and fee costs vary from program to program and can change multiple times per year. The sheer volume of data can present collection issues, but we are unclear how ETPs should report if costs have varied over the course of a year.	States approve eligible training providers in accordance with WIOA Section 122 and 20 CFR 680, and manage the provider process on a regular basis. The ETP performance report is required on an annual basis. The Departments recommend that States develop a mechanism for capturing the required reporting information from an institution through the regular provider approval processes. The Departments expect the data to be reported based upon the state policy. The Departments are making no revisions as a result of this comment.	1
	Additional definition of data to be included on the ETA 9171 (Eligible Training Provider report) is being requested. Is Commerce expected to use the definitions as our report specifications currently in use? 3141 (b)(4)	Data submitted in this report must follow the definitions and instructions in the final ETA 9171	1
	One commenter provided an overview of their concerns on the ETP as detailed below: Overall Comment – There are too many new data elements added that are of relatively little use and/or poorly defined. States and providers would have to bear a substantial added burden of modifying interfaces and databases, technical assistance and training, and periodic updating or validation of this information for thousands of separate programs in a state, without providing commensurate value to participants, caseworkers or managers. The WIOA statute presents a sufficiently long list of unfunded mandates or options without inventing more. Long before a national database of programs of study should come a national exchange system to automate bilateral agreements among states for reciprocal eligibility management across state lines. Given the paucity of additional resources available for WIOA, it must be seriously questioned whether the additional data elements in this ICR should be in competition for scarce resources with other WIOA implementation activities such as: - Meaningfully defining and systematically and consistently collecting data on the factors that are to be “take(n) into account” in establishing state criteria for eligible programs (WIOA sec. 122 (b)(1)( A thru J). - Identifying and setting metrics for initial eligibility criteria (WIOA sec.122 (b)(4)(D)) that are meaningful for a program of study that applies in its first year of operation – in particular under (D)(i). - What to do with programs that have so few exiters in their first year that there are not enough exiters to make and publish a statistically valid performance rating when their initial year expires (WIOA sec. 116 (d)(6)(C). - Developing program evaluation systems for other types of training services under sec. 122 (h) and Sec. 123.	The Departments acknowledge the concerns related to additional data elements. However, the Departments have determined that this additional information will assist participants in choosing employment and training activities, and providers. Under WIOA section 185(a)(2), recipients of WIOA title I funds must submit such records and reports “as the Secretary of Labor may require regarding the performance of programs and activities carried out under [WIOA title I].” Therefore, the Departments are making no revision in response to this comment. The Departments have sought to improve the clarity of the definitions and reporting requirements by reordering the data elements into related groups, updating element names and definitions, and providing data elements to report the disaggregation required under the previous approval of this ICR.	1
	One commenter had concerns regarding a data element in the following c comment: #105 - Total Number of Individuals Served Quarterly reporting of participant counts is not feasible for non-public programs, and a significant added burden for some public programs – to what end? Should be changed to read “the unduplicated sum across all four quarters” if quarterly data is available There is confusion between “program” and “course” information. A single course can be part of more than one program of study, and some participants enroll with no intent to take more than one or a few specific courses. For a significant number of participants the selection of a specific “program of study” occurs sometime after initial enrollment, and may change repeatedly.	The Departments clarify that this is not a quarterly report and are modifying this definition to clarify there are no required quarterly submissions associated with this report.	1
	One commenter had concerns regarding a data element in the following c comment: #106 - Total Number of Individuals Exited “or transferred from this program” – again this is a level of granularity that doesn’t always exist in reality	The Departments acknowledges that providers may not have information regarding whether a student transferred to another institution in all cases. However, if a student officially transferred from the provider to another institution and the institution has a record of that transfer this should be reported as a transfer in element 106. The Departments are making no revision as a result of this comment.	1
	One commenter had concerns regarding a data element in the following c comment: 139-141 - O*NET-SOC Code Associated with Program #1 (#2, #3) Few providers have familiarity with SOC codes. So either state staff are going to be trained to translate program CIPS or skill or occupation descriptions into SOCs, or an automated match with a CIP-SOC crosswalk will be used. And if it is the latter, there is no reason for this not to be done at the national level.	The Departments appreciate the recommendation to create an automated CIP-SOC crosswalk. The Departments will consider this recommendation in the future.
	One commenter suggested the following elements should be collected in a separate report as detailed below: ELEMENTS THAT SHOULD BE COLLECTED IN A SEPARATE REPORT #123 - Cost Per WIOA Participant Served (to be completed by States) All of the precedent elements are at the program of study level #136 - Reciprocal Agreements with Other States (to be completed by states) – also a state level not program level element Inserting an aggregate state level element here is very confusing. State level element s should be collected in a separate block.	This report is submitted by the States to DOL. States have the flexibility to choose which of the information they will complete and which will be submitted by providers to the state.	1
	One commenter expressed the following data elements to be burdensome: EXCESSIVE BURDEN TO VALUE ADDITIONS The following are “nice-to-have” data elements that are definitely not free. #125 - Description of Training Program #126 - Description of Training Provider #128 – URL of Training Program # 129 - Out-of-Pocket Cost for a Non-WIOA Participant: Tuition and Required Fees #130 - Out-of-Pocket Cost for a Non-WIOA Participant: Books and Supplies #131 - Program Length (Clock/Contact Hours) #132 - Program Length (weeks) #133 - Program Prerequisites #134 - Program Format	The Departments acknowledges the concerns related to additional data elements. However, the Departments have determined that this additional information will assist participants in choosing employment and training activities, and providers. Under WIOA section 185(a)(2), recipients of WIOA title I funds must submit such records and reports “as the Secretary of Labor may require regarding the performance of programs and activities carried out under [WIOA title I].” Therefore, the Departments are making no revision in response to this comment.	1
	One commenter raised the issue of clarity for SOC codes. Smaller providers, particularly smaller career/proprietary schools may have difficulty 13 understanding items like CIP and O*NET SOC Codes and contact hours.	The Departments appreciate that some providers may need additional information regarding new elements. Online tools are available to assist the public in understanding O*NET. Providers should reference https://www.onetcenter.org/overview.html and can get help connecting other systems (such as CIPs) to the O*NET-SOC taxonomy at https://www.onetonline.org/crosswalk/. The Departments have made no changes in response to this comment.	1
	Consumer Education is enhanced by having out-of-pocket costs for Non-WIOA Participants.. However, the reality is there is not always one out-of-pocket cost. Most community colleges have “in district” and “out of district” cost models. Those who live in the community colleges’ taxing district pay the “in district rate” while those who live outside the district pay a higher “out of district rate”. We’re not aware if a similar practice exists in other states but we recommend providing in district and out of district cost fields.	The Departments encourage States to provide this information on variations in costs to consumers when they deem it appropriate. However, the Departments are not making changes to this element in order to ensure comparability across providers and across consumers.	1
	One commenter requested clarity on reporting SOC codes. We have a CIP-O*NET SOC code crosswalk that we use for analysis and it shows a large number of CIP codes associated with more than 3 SOC codes. What basis do you expect Providers use to select 3 to report? Would it be possible to just report as many as applicable? After all, the proposal is to have one field to list Reciprocal Agreements with other states.	For each program, States should submit up to three SOC codes. For providers with multiple eligible programs, different SOC codes may be provided for each separate program.	1
	One commenter expressed concern regarding clarity of SOC codes. Eight (8) digit O*NET SOC codes may be too detailed for many providers to use and for prospective students (i.e. the consumers) to easily understand. Moving to 6 digit O*NET SOC codes would simplify reporting for providers and enhance understandability for prospective students.	The Departments have determined that the additional level of detail will be of use to the prospective students in providing more information about the expectations of the job. The Departments have made no changes in response to this comment.	1
	Regarding Reciprocal Agreements, isn’t that really state-level information and not program-by-program? If so, doesn’t it make sense to track that at the state level rather than the provider level?	These reports are submitted by the states to DOL, and therefore states can choose the manner in which they collect this data. Therefore, providers do not need to report this information to the states if the states choose to append the information themselves.	1
	One commenter expressed concern regarding the addition of average earnings in the comment below: We do not support adding Average Earnings in Quarter 2 and Quarter 4 to the data reported. WIOA moved from a focus on Average Earnings to Median Earnings. ETPs are required to report Median Earnings in Quarter 2 for all students and WIOA Participants under WIOA §116. Requiring ETPs to report both Median and Average Earnings and to present that information to consumers will not serve the needs of consumers or providers well. If DOL believes that it is imperative to provide earnings in the 4th quarter information to consumers, even though the statute does not require it, it would be better to make it median earnings to make it consistent with the required earnings data that will already be displayed as required by §116.	While the performance reporting under 116, records are reported for each individual, and therefore even though the indicator used for performance accountability purposes is median earnings, both median and average earnings can be calculated under that reporting. For ETP reporting, the reports are aggregated and therefore those calculations can only be done by those submitting the reports. However, this requirement does not add any burden from the perspective of data collection, as it is simply a different calculation with the same data.	1
	The JPA-ICR states that small entities will not be significantly impacted. We believe that there are many ETPs that are small businesses who will certainly be impacted by the requirements.	The Departments have worked to reduce the burden on all entities, including smaller entities, while still complying with the requirements in the statute.	1
	The opportunity to compare the effectiveness of identical training provided to WIOA and non-WIOA participants is a noble cause. That said we anticipate an overall decrease in the number of training programs either sustained on the ETPL or offered as new programs because of the limited value returned to the training provider compared against the cost of gathering and distributing the data. The consequence of a desire for comparative data manifests a cost shift to the training provider.	The collection of data on both WIOA participants, and All individuals in a program of study is required by WIOA sec. 116 (d) (4).  Elements #118, 119, 120 have been changed so that “non-WIOA completer” outcome elements now are “WIOA exiter” outcome elements in order to more directly align with the statute. These elements were also renumbered 139, 140, and 141 to accommodate the updated format. Elements 121 and 122 were removed.	1
	One commenter suggested the following in the comment below: The explanations from the list of new data elements (p. 10) mention the following: a. The description of the training provider "may include accreditation". We would benefit from a list of recognized and approved accreditation bodies. b. We presently gather and use program prerequisites in a narrative form. ETA-9171 shows coded values, but the six fields available are not a match for the specific prerequisites expressed in current practice by training providers. We also use this prerequisite field to explain the difference between generally-available training programs to any training seeker, and those training programs restricted only to Registered Apprenticeship candidates under contract. c. The specific name of a certificate is captured in a text field, but would certainly benefit from a coded list.	The Departments appreciate the commenter’s suggestion of a coded list for prerequisites and accreditation bodies, and a coded list of certificates. However, the Departments have determined that the current fields offer the most flexibility for providers. States are permitted to continue to use state specific tracking mechanisms to satisfy administration needs such as programs restricted to Registered Apprenticeship candidates. The Departments are making no change as a result of this comment.	1
	On the requirement to provide in the performance report the number of exits "disaggregated by the type of entity that provided the training" (p. 36), the three-year comparison won't be available until there are three years of exits for an initial cohort. The rationale to gather and report this field is unclear.	The three years of data will be collected on a rolling basis and the Departments acknowledge that the full three year data set will not be available in the first year. However, states are required to report the data from providers as it is received rather than waiting until a full three years of program data is available. The Departments will make no change as a result of this comment.	1
Reportable Individual
	One commenter sought clarification on reportable individuals in the comment below: Where are "Reportable Individuals" reported on ETA - 9169? How are unique "Reportable Individuals" who have multiple "Reportable Individual" episodes reported in the count? Do they count one time or multiple times as a reportable individual?	Reportable Individuals are not included in the ETA – 9169 report. ETA-9169 is for the data that is required for the annual state performance required under WIOA sec. 116(b)
MISC VR Comments
	Many recipients of Vocational Rehabilitation services are people who qualify for Social Security benefits, such as SSDI. If there is a way to collect data such that Departments may show the return on investment (ROI), that would be great. For example, is there a way to show that for every dollar spent in VR services, $X are saved in Social Security benefits paid out? We believe that the proposed collection of information does have utility, particularly in regard to our efforts to ensure that individuals with disabilities are served through WIOA programming. To that end, we encourage the collection of more specific performance information in regard to services for individuals with disabilities. In particular, we request that the Department of Labor consider collecting metrics regarding: Outcomes experienced by individuals with the most significant disabilities. Currently, 34 C.F.R. § 361.88 requires that a designated state unit (DSU) report the number of individuals, and the number of individuals with significant disabilities, that exited VR. Indiana Disability Rights is concerned that competitive, integrated employment outcomes may be much lower for those categorized as “most significantly disabled,” but there are no current reporting requirements regarding this metric. Supported employment usage. Through our CAP efforts, we have worked with several former VR clients that lost their job because VR did not offer sustained supported employment assistance. We are interested in learning more about the average number of months for which VR authorizes supported employment to its clients, and believe reporting on this metric would be useful. This interest is particularly pertinent now that extended supported employment is available. We would like to know whether VR agencies are making use of this new service opportunity. “Minority” interpreted more broadly. 34 C.F.R. § 361.88(a)(7) and (9) require the DSU to report specifically on those “individuals from a minority background.” Beyond racial or geographic minority status, we are interested in learning more about minorities within the disability population, including people who are deaf-blind. We are concerned that some disability minorities may not be successfully exiting the VR program due to the perceived burden in serving them. Most-provided VR services. We would be interested in learning the top service categories (i.e., transportation, postsecondary tuition, assistive technology) under which VR provides assistance to clients each year. We are concerned that some Individualized Plans for Employment (IPE) may be generated as boilerplates, and are interested in whether VR clients are forced to receive services they may not want. If, for example, we saw that 90% or more of a given state VR clients were receiving job placement services, we would suspect that IPEs are not being adequately tailored to individual clients. Conversely, if we saw that transportation assistance was rarely offered – especially in this relatively rural state – we would question whether VR was offering necessary services. Without such metrics, it is difficult to draw systemic conclusions about the adequacy of VR service provision. Sheltered workshop data. WIOA imposed new requirements that must be met before transition-aged students can enter sheltered workshops for subminimum wage. In addition to certifying that these requirements have been met, VR is also responsible for providing career counseling and referral services to current sheltered workers. We would like to see reporting requirements regarding the number of transition-aged students issued certification to enter subminimum wage work, as well as metrics on the number of sheltered workers reached by VR counseling and referral programs. Since Rehabilitation title I is included in all primary indicators for reporting purposes, what flexibility does the workforce agency have to ensure data exchanges are encompassing all participants for the measures? And, if measure(s) are not met, are the sanctions applied to both the reporting agency and the source data agency? 3141 (b)(2)(A)(i)	The Departments received a number of comments that were non-substantive and/or unrelated to the contents of this information collection request.  Consequently, we will not respond to those comments as they are outside the scope of this ICR
Miscellaneous
	The "WIOA PERIODS FOR REPORTING COHORTS" document from the previous Joint ICR should be added to this ICR because in written guidance a 2 quarter lag is referenced. The approved ICR has a 3 quarter lag following the time required outcomes to manifest. This discrepancy should be clarified in this amended CR.	The two-quarter lag takes place after the 90 days to confirm exit.	1
Irrelevant/No Action
	I tried to get help through this program to enroll in approved program AAS Business. How does this work. The representative took my name and number but never contacted me with a follow up. Data Element 1811: Please clarify this data element is only reported during a program year in which the participant is enrolled in education or training. This clarification will address situations when participants that receive education or training in one program year and continue participation through non-training services in subsequent program years. Data Element 907: "Incumbent Worker Training" Rapid Response state funded should be a category because Dislocated Worker reserve funds can be used to provide incumbent worker training. The Dislocated Worker Reserve (25% of a state's allotment) is a distinct fund source from the statewide (15%) category. We request a category for funding from Rapid Response related activities (Sec. 134 (a) 2.A.i.) is added as an option to Data Element 907. 8. Data Elements 203 through 209: We request the Department of Labor Employment Training Administration clarify the necessity for collecting these elements. Without a clear purpose we believe these elements regarding an individual's disability and services received are not necessary for collection for WIOA performance accountability system purposes. We feel that a number of clients will perceive these elements as excessively invasive and will not pursue our workforce programs. Reporting Element: 1608 - Employment Related to Training (2nd Quarter After Exit): How are states supposed to accurately report this? This would be a manual indicator as wage cross match will not be able to automatically set this flag. Obviously client's that stay in follow-up will be able to be reported; however, what about those that fall off the face of the Earth? Reporting Element: 808 - Migrant and Seasonal Farmworker Status at National Farmworker Jobs Program Entry (WIOA sec. 167): Does the value provided here affect reporting element 903 or 905? Can a 'youth' MSFW also be reported as an adult? Reporting Element: 100 – Unique Individual Identifier: How are states supposed to meet this requirement when title I & III are administered by a different agency than title II & IV. Each agency will have a different unique identifier as assigned by their MIS. Some clients receive services under multiple titles. Because there is the possibility of different identifiers participants will be counted more than once, rather than being counted once with multiple periods of participation. In PIRL, the SCSEP program selected collection of the TAA Application Date. I believe this is an error, as SCSEP and TAA are two separate programs. The SCSEP program selected Data Element 1324 - Current Quarter Training Expenditures, and Data Element 1325 - Total Training Expenditures. The data for these elements tie with TAA data as reported in the TAA 9130 Financial Reports. TAA Data Integrity compliance mechanisms will fail unless TAA expenditures are tracked separately from all other programs- if SCSEP expenditures are reported together with TAA in a co-enrollment scenario, there will be no opportunity to track TAA line item training expenditures between PIRL and 9130. Credential Attainment is listed as a Primary Indicator as well, however, collection of this information has been challenging in the past. The education agency will be providing the data for those participants completing a secondary or post-secondary program in order to be included in the performance. What flexibility does the workforce agency have to ensure data exchanges are encompassing all participants for the measure? And, if measure is not met, are the sanctions applied to both the reporting agency and the source data agency? 3141 (b)(2)(A)(i)(IV-V) When basic career services are provided in the one-stop to reportable individuals who are not participants in a specific program, or who are co-enrolled in multiple programs, what is the determining factor in deciding which program to assign the service for reporting and performance purposes? Is it the program funding the career counselor (staff) who provided the service? Is it the program that the customer is enrolled in, regardless of who provides the service? Some other criteria? Reportable Individuals: How are unique “Reportable Individuals” who have multiple “Reportable Individual” episodes reported in the count and where are they counted, do they count one time or multiple? In relation to data validation, how do the new measures (indicators) affect data validation? Particularly when the data being linked originates from other agencies like Education and Vocational Rehabilitation, will the originating agency validate the data or will the reporting agency be responsible? 3141 (b)(5) The Supporting Statement and corresponding reporting template (Statewide Performance Report) does not clearly identify if report is by individual Programs or jointly submitted across all 6 core programs? A checkbox is listed that allows Title1 and Title III to be submitted together. Are the funds to be captured by program, captured in a way that they can be combined to accurately determine the cost per service, especially given that participants are duplicated when crossing over programs? If we share data internally across programs, and each program has established data validation tools that differ, which barrier should be used? If WIOA Title 1 has a participant listed as a veteran, and Rehabilitation indicates that the same individual is not a veteran, which agency overrides the other in reporting? We (commenter) support the proposal to limit the narrative to 25 pages. One commenter would like to know if any of the following count towards a Measurable Skill Gains: Passage of 1 GED test Taking a college entrance exam Getting a food handlers certificate	The Departments received a number of comments that were non-substantive and/or unrelated to the contents of this information collection request.  Consequently, we will not respond to those comments as they are outside the scope of this ICR or require no response/action.