60D FRN Comment Response Summary

Upward Bound Program – 2017 Competition

Summary of Public Comments with Responses

Introduction

The U.S. Department of Education (Department) received over 300 comments from individuals interested in the Upward Bound (UB) Program competition, some of which commented on several topics: the commenters addressed four broad areas with the majority of comments expressing concern regarding the formatting requirements of the application narrative. Additional areas of some concern to the commenters were the competitive preference priority, performance measures and objectives. A number of commenters requested an increase in the page limit due to formatting requirements. We also received a number of miscellaneous comments, which suggest the need for clarification on several issues (e.g., the allowableness of stipends, providing services to disconnected youth, program funding and overlapping due dates of applications and the annual performance report).

Below is a summary of the comments and the Department’s responses to the comments including whether subsequent changes have been made to the Notice Inviting Applications (NIA) and the application package for new awards for FY 2017.

Formatting of Application Narrative—Double Spacing of Charts, Tables, Figures

and Graphs

Comments: The majority of commenters expressed concerns about the change in formatting requirements for the application narrative. The particular change in formatting requirements stipulates that all text in the application narrative including charts, tables, figures and graphs must be double spaced. Titles, headings, footnotes, quotations, references and captions may be single spaced. Several commenters stated that in previous UB grant competitions, UB applicants were able to single space charts, tables, figures and graphs.

Commenters expressed concern that the new formatting requirement would limit the amount of available space for adequately addressing the requirements in the UB application package. Several commenters with projects in rural areas stated that the formatting change is especially biased against rural programs that serve a larger number of target schools and thus have a lot more data to provide. Commenters also stated that single spaced charts, tables, figures and graphs may have greater clarity for reviewers of the UB application narrative. A number of commenters suggested alternatives to the proposed change in formatting requirements that included the following: increasing the page limit, allowing double spaced tables, charts, figures and graphs with a 10 point font and allowing the use of formatting approaches suggested in well-known style manuals like the Modern Language Association (MLA) style manual.

A majority of the commenters suggested that the Department increase the page limit of the application narrative to 75 so that applicants can adequately provide critical content. Commenters stated that with the current formatting requirement and the 60 page limit, it will be difficult to include all of the information and data needed for reviewers to assess.

Response: The Department is aware of the concerns of prospective applicants regarding the proposed change to the formatting requirements. However, the Department believes that the grant process is competitive and should be equitable; therefore, in order to ensure that applicants that include an inordinate amount of text in charts, tables, figures and graphs do not gain an advantage over applicants that do not engage in such practices, the Department is requiring double spacing of the application narrative, including charts, tables, figures and graphs.

Given the new formatting requirement and the need for applicants to provide clear and concise data for reviewers to assess, the Department will increase the page limit.

Change: The final NIA for the FY 2017 UB competition will reflect an increase to the page limitation of 5 pages for the Program Narrative section for a total of 65 pages.

Performance Measures

Comments: Several commenters suggested that three of the UB program performance measures be deleted because they are burdensome and not needed to meet a mandatory objective. The performance measures identified by commenters include: performance measure #1, the percentage of UB students who took two years of mathematics beyond algebra 1 by the 12^th grade; #4, the percentage of UB students who enroll in a program of postsecondary education will graduate on time – within four years; and performance measure #5, the percentage of UB participants who enroll in a program of postsecondary education will attain either an associate’s degree within three years or a bachelor’s degree within six years.

Response: The Department disagrees with the commenters. The UB performance measures were established by the Department to assess the extent to which the UB Program is adequately preparing students for success in postsecondary education. The Department must collect data to demonstrate that the UB Program is achieving its mission. UB performance measures and UB mandatory objectives serve similar but slightly different purposes. UB performance measures assess UB program performance. The mandatory objectives assess the performance of individual projects relative to target objectives established by those UB projects. Regarding performance measure #1, the Department believes that this is a reasonable requirement given the UB Program’s emphasis on assisting participants in completing a rigorous secondary school program of study. In addition, for performance measures #4 and #5, the Department utilizes the annual performance report to collect the data, thus eliminating the requirement and burden for UB projects to track the academic progress of students.

Change: None.

Comments: One commenter asked if performance measure #1 is asking if a student had completed two years of math beyond algebra 1 at the time that they entered the 12^th grade or by the time they completed the 12^th grade.

Response: Performance measure #1 is asking if the student completed two years of math beyond algebra 1 by the end of the 12th grade.

Change: None.

Competitive Preference Priority

Comments: Several commenters expressed concern regarding the competitive preference priority. One commenter stated that the Department should use an invitational priority instead of the competitive preference priority (CPP). Another commenter stated that assigning points to an ad hoc part of the application detracts from the intent of the TRIO legislation. The commenter further states that evidenced based research should be established in the reauthorization of the Higher Education Act (HEA).

Response: The Department does not agree that an invitational priority should be used instead of the CPP. The Secretary is sensitive to the adjustments that projects may need to make in terms of service delivery to address the CPP, but believes it is critical that grantees identify and implement strategies that effectively support disadvantaged students. The CPP was designed to incentivize applicants to provide evidence based programs that increase the likelihood that students complete high school, enroll into and complete postsecondary education. The CPP encourages applicants to focus on promising strategies that have been the subject of research and have demonstrated positive impacts in evaluations of such interventions. The CPP enhances the effectiveness of grant competitions and facilitates the improvement of outcomes for students participating in a UB project. Furthermore, the specific activities around which the Department is encouraging evidence based strategies are explicitly authorized in the UB legislation. Applicants will receive additional points for addressing some of the challenges that face UB-eligible participants in innovative ways with the ultimate goal of better meeting UB program goals. Lastly, the Office of Management and Budget (OMB) has issued guidance to federal agencies outlining how federal agencies might move forward to advance the use of evidence in programs. The Department’s action in this regard, is consistent with guidance provided by OMB and does not necessitate authorization by the reauthorization of the HEA.

Change: None.

Comments: Several commenters stated that studies could not be found in the What Works Clearinghouse (WWC) that address high school completion, college enrollment and college completion. One commenter asked if the priority could be modified to assess interventions that were effective in helping students to complete college.

Response: The focus of the CPP is the implementation of an authorized service, stated in 34 CFR 645.11 or 34 CFR 645.12, which is based on an evidenced-based intervention. The CPP does not specifically focus on high school completion, college enrollment and college completion. The CPP encourages applicants to propose activities (tutoring, counseling, mentoring, e.g., that are supported by the moderate evidence of effectiveness standard as defined in 34 CFR 77.1(c). In general, the Department is interested in promoting the use of evidence-based research to inform activities in order to improve student outcomes, and expects that applicants who address the CPP will use evidence-based research to identify strategies that most effectively meet the needs of UB participants. If the available studies already vetted by the WWC do not sufficiently address the needs of an applicant responding to the CPP, applicants also have the opportunity to increase the WWC research base by submitting studies for consideration under the WWC guidelines of moderate evidence of effectiveness. To assist applicants, the Department anticipates providing references to potential studies via the UB website.

Change: None.

Program Objectives

Project Objective #2: Academic Performance on Standardized Tests

Comments: A number of commenters expressed concern stating that the objective should be deleted due to the fact that no standardized test is used throughout the nation and that states use many different tests which can change from year to year. Commenters believe that major inconsistencies from year to year in state test and students opting out of testing, yield inconsistent data and inadequate baseline data.

Response: To facilitate the awarding of prior experience (PE) points, each grantee must have an objective that addresses each of the statutorily-prescribed outcome measures for the UB program. Therefore, the application package includes a standardized objective for each PE Criterion, including academic performance on standardized tests.

The regulation that addresses this requirement is in 34 CFR 645.32(e)(ii)B and states: “Whether the applicant met or exceeded its approved objective with regard to participants served during the project period who met the academic performance levels on standardized tests as specified in the approved objectives.”

In establishing this objective for the FY 2017 UB competition, the Department decided to continue to use the state assessments in reading/language arts and math for the following reasons:

• To measure if an Upward Bound participant is prepared for college, the participant should be proficient in both reading/language arts and math; and

• UB projects are familiar with this objective as it has been used to measure “academic performance” since the FY 2007 UB competition.”

Change: None.

Project Objective # 4: Secondary School Graduation (rigorous secondary school program of study) and Project Objective # 5: Postsecondary Enrollment

Comments: Several commenters stated that the two objectives on the Program Profile form make reference to the high school graduation cohort and that the language regarding the high school graduation cohort was revised in the 2012 UB application package and is no longer in use.

Response: The Department agrees with the commenters that the objectives were revised. The wording of the two objectives will be corrected.

Change: The objectives have been changed on the Program Profile form to read as follows:

Secondary School Graduation (Rigorous Secondary School Program of Study): X% of all current and prior-year UB participants who graduated from high school during the school year with a regular secondary school diploma will complete a rigorous secondary school program of study.

Postsecondary Enrollment: X% of all current and prior-year UB participants who graduated from high school during the school year with a regular secondary school diploma will enroll in a program of postsecondary education by the fall term immediately following high school graduation, or will have received notification by the fall term immediately following high school from an institution of higher education of acceptance but deferred enrollment until the next academic semester (e.g., spring semester).

Project Objective #4: Postsecondary Completion

Comment: One commenter stated that in addition to an associate’s and bachelor’s degree; certificates and diplomas should be included as a form of postsecondary completion.

Response: Due to the UB Program’s intensive, college preparatory nature, we do not agree with the commenter that a certificate or diploma should be included in postsecondary completion measurements.

Change: None

Miscellaneous

Comment: Several commenters asked if there are points awarded for the invitational priority.

Response: Under 34 CFR 75.105 (c) (1), the Secretary may invite applications that meet a priority. No points are awarded.

Change: None.

Comments: One commenter expressed concern that the due dates for the UB annual performance report (APR) and the 2017 UB application may overlap.

Response: The Department is aware of the possibility of the UB application and UB APR due dates overlapping; however, it may be unavoidable due to the necessity of completing the peer review process in a timely manner in order to issue awards to projects with a June 1, 2017 start date.

Change: None.

Comments: One commenter was concerned that stipends may not be allowed because the instructions for the Budget Summary and Itemized Line Item Budget state that training stipends are not applicable.

Response: In accordance with the UB regulations, stipends are applicable under UB. The Department will correct the reference to training stipends in the UB application package.

Change: The following change has been made in the application: “Training Stipends: Provide the cost for student stipends”.

Comments: One commenter asked if there is an expectation that projects serve a certain number of disconnected youth.

Response: There is no expectation that a UB project serve a certain number of disconnected youth.

Change: None.

Comment: One commenter asked that if the Federal TRIO Programs received a funding increase; a portion of those funds be used to fund additional projects in 2017.

Response: The Administration has requested $900,000,000 for the Federal TRIO Programs for FY 2017, of which we intend to use an estimated $273,000,000 for UB awards. The actual level of funding, if any, depends on final congressional action. If additional funds are appropriated by Congress, the Department will determine the allocation of the increase in funding at that time.

Change: None.

Comment: One commenter had a concern regarding the calculation of the prior experience points. The commenter stated that projects that serve at least 90% of the number of participants they are funded to serve but less than 100% are thrice penalized for not reaching their funded-to-serve number. The commenter believes that it is understandable that projects must forfeit the three points awarded to projects that served the number of participants agreed to under the approved application. However, the commenter states that projects are further penalized with respect to the prior experience points awarded under Academic performance--GPA and Secondary School retention and graduation, because the denominator in each calculation is required to be "the greater of the number of participants the project was funded to serve, or the number actually served during the assessment year." This is double (triple) jeopardy. The denominator in each should be the number of students actually served.

Response: The Department does not agree with the commenter. It would be unfair that a grantee that did not meet its funded numbers would be assessed the same as a project that met or exceeded its funded numbers. The denominator must include the “greater of the number of the participants the project was funded to serve, or the number actually served”, in order to ensure equity and fairness. If two projects have the same funded number and the same objective for the two objectives cited, in order to meet the objective, the project that met its objective would be required to have more students in the numerator than the project that did not meet its funded numbers, which is not fair. This policy of the denominator including the “greater of the number of the participants the project was funded to serve, or the number actually served” is followed in other TRIO Programs as well.

Change: None

Comment: One commenter requested that prior participants be removed from the Secondary School Graduation (Rigorous Secondary School Program of Study) and the Postsecondary Enrollment objectives. The commenter stated that it is difficult to track students who may have left the project in their freshmen and sophomore year.

Response: The Department does understand the concern of the commenter; however, in order to determine the success of UB participants and the program, it is important to track participants, new and prior, through postsecondary education. The Department encourages applicants to take into consideration extenuating circumstances when setting objective measures. This would include the difficulty in tracking participants who may leave the project.

Change: None

Comment: One commenter states that in the OMB standard form, Notice to All Applicants, references are made to gender. The commenter requested that terms and references to gender and sex be updated.

Response: The referenced form is an OMB standard form. The Department does not have the authority to change the content of an OMB standard form.

Change: None.

Comment: In reference to dual enrollment, one commenter asked if dual enrollment can be beneficial to populations of students that reside outside of the continental United States.

Response: Using data from a National Educational Longitudinal Study conducted by the National Center for Educational Statistics (NCES), the author of the study in the UB application package, found that dual enrollment positively influences college attainment. The author also concluded that first-generation college students that participated in dual enrollment were more likely to attain a college degree than non-participants in dual enrollment programs. Socioeconomic status and ethnicity were among the major variables cited in the study. The study did not disaggregate results due to the geographic regions where the students reside.

Change: None.

Comment: One commenter suggests that the competitive preference priority be discussed more fully in the Dear Applicant Letter.

Response: The more substantive introduction of the competition preference priority is in the notice inviting applications (NIA). The NIA is the official document within the application package; therefore, the Department believes that it is crucial to include pertinent information regarding the UB Program in this document.

Change: None.

Comment: One commenter questions the requirement to “examine and report on unanticipated outcomes” and suggests that the language be removed from the final version of the application package.

Response: The Department agrees with the commenter, the requirement to examine and report on unanticipated outcomes is not consistent with the selection criteria in the UB Program regulations.

Change: The language in the application package that refers to reporting on unanticipated outcomes will be deleted.

Comment: One commenter suggested that students that have a high risk of academic failure be added to the category of students that are listed under the Background section of the NIA.

Response: The Department does not feel that it is necessary to include this category of students under the Background section of the NIA. Projects are able to use the criteria of a high risk of academic failure as part of the eligibility requirement when selecting participants for their project.

Change: None.

Comment: One commenter noted that the Upward Bound Program Profile form did not provide a reference or place to include links to the citation(s) submitted for the competitive preference priority. The commenter also asked if the abstract was an appropriate place to provide the links to citation(s) submitted for the competitive preference priority.

Response: The Department identified two locations for the citation(s) for the competitive preference priority; the application abstract and the Upward Bound Program Profile form. The Department will add the information to the Upward Bound Program Profile form.

Change: The Department has added to the Upward Bound Program Profile form a specific location for the citation(s) for the competitive preference priority.