Final NFP

4000-01-U

DEPARTMENT OF EDUCATION

34 CFR Subtitle B, Chapter II

DOCKET ID ED-2009-OII-0012

RIN 1855-AA06

Investing in Innovation Fund

Catalog of Federal Domestic Assistance (CFDA) Numbers: 84.396A (Scale-up grants), 84.396B (Validation grants), and 84.396C (Development grants).

AGENCY: Office of Innovation and Improvement, Department of Education.

ACTION: Final priorities, requirements, definitions, and selection criteria.

DATES: These priorities, requirements, definitions, and selection criteria are effective [INSERT DATE 60 DAYS AFTER DATE OF PUBLICATION IN THE FEDERAL REGISTER].

SUMMARY: The Secretary of Education (Secretary) establishes priorities, requirements, definitions, and selection criteria under the Investing in Innovation Fund. The Secretary may use these priorities, requirements, definitions, and selection criteria in any year in which this program is in effect.

FOR FURTHER INFORMATION CONTACT: Telephone: (202) 453-7122; or by e-mail: i3@ed.gov; or by mail: (Attention: Investing in Innovation), U.S. Department of Education, 400 Maryland Avenue, SW., room 4W302, Washington, DC 20202.

SUPPLEMENTARY INFORMATION:

Purpose of Program: The Investing in Innovation Fund, established under section 14007 of the American Recovery and Reinvestment Act of 2009 (ARRA), provides funding to support (1) local educational agencies (LEAs), and (2) nonprofit organizations in partnership with (a) one or more LEAs or (b) a consortium of schools. The purpose of this program is to provide competitive grants to applicants with a record of improving student achievement and attainment in order to expand the implementation of, and investment in, innovative practices that are demonstrated to have an impact on improving student achievement or student growth (as defined in this notice), closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates.

These grants will (1) allow eligible entities to expand and develop innovative practices that can serve as models of best practices, (2) allow eligible entities to work in partnership with the private sector and the philanthropic community, and (3) support eligible entities in identifying and documenting best practices that can be shared and taken to scale based on demonstrated success.

Background: One of the overall goals of the ARRA is to improve student achievement and attainment through school improvement and reform. Within the context of the ARRA, the Investing in Innovation Fund focuses on four education reform areas that will help achieve this goal: (1) improving teacher and principal effectiveness and ensuring that all schools have effective teachers and principals, (2) gathering information to improve student learning, teacher performance, and college and career readiness through enhanced data systems, (3) implementing college-and career-ready standards and rigorous assessments aligned with those standards, and (4) improving achievement in low-performing schools through intensive support and effective interventions. The Department is using the Investing in Innovation Fund to support the overarching ARRA goal of improving student achievement and attainment by establishing four absolute priorities that are directly aligned with the four education reform areas under the ARRA. We are also establishing in this notice four competitive preference priorities that are aligned with Department reform goals in the following areas: (1) early learning, (2) college access and success, (3) serving students with disabilities and limited English proficient students, and (4) serving students in rural LEAs. Finally, we are requiring that all projects funded under this program be designed to serve high-need students (as defined in this notice).

Under this program, the Department is awarding three types of grants: “Scale-up” grants, “Validation” grants, and “Development” grants. Among the three grant types, there are differences in terms of the evidence that an applicant is required to submit in support of its proposed project; the expectations for “scaling up” successful projects during or after the grant period, either directly or through partners; and the funding that a successful applicant is eligible to receive. The following is an overview of the three types of grants:

(1) Scale-up grants provide funding to “scale up” practices, strategies, or programs for which there is strong evidence (as defined in this notice) that the proposed practice, strategy, or program will have a statistically significant effect on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates, and that the effect of implementing the proposed practice, strategy, or program will be substantial and important. An applicant for a Scale-up grant may also demonstrate success through an intermediate variable strongly correlated with these outcomes, such as teacher or principal effectiveness.

An applicant for a Scale-up grant must estimate the number of students to be reached by the proposed project and provide evidence of its capacity to reach the proposed number of students during the course of the grant. In addition, an applicant for a Scale-up grant must provide evidence of its capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to scale up to a State, regional, or national level, working directly or through partners either during or following the grant period. We recognize that LEAs are not typically responsible for taking to scale their practices, strategies, or programs in other LEAs and States. However, all applicants, including LEAs, can and should partner with others (e.g., State educational agencies) to disseminate and take to scale their effective practices, strategies, and programs.

Peer reviewers will review all eligible Scale-up grant applications. However, if an application does not meet the definition of strong evidence in this notice, the Department will not consider the application for funding.

Successful applicants for Scale-up grants will receive more funding than successful applicants for Validation or Development grants.

(2) Validation grants provide funding to support practices, strategies, or programs that show promise, but for which there is currently only moderate evidence (as defined in this notice) that the proposed practice, strategy, or program will have a statistically significant effect on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates and that, with further study, the effect of implementing the proposed practice, strategy, or program may prove to be substantial and important. Thus, applications for Validation grants do not need to have the same level of research evidence to support the proposed project as is required for Scale-up grants. An applicant may also demonstrate success through an intermediate variable strongly correlated with these outcomes, such as teacher or principal effectiveness.

An applicant for a Validation grant must estimate the number of students to be reached by the proposed project and provide evidence of its capacity to reach the proposed number of students during the course of the grant. In addition, an applicant for a Validation grant must provide evidence of its capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to scale up to a State or regional level, working directly or through partners either during or following the grant period. As noted earlier, we recognize that LEAs are not typically responsible for taking to scale their practices, strategies, or programs in other LEAs and States. However, all applicants, including LEAs, can and should partner with others to disseminate and take to scale their effective practices, strategies, and programs.

Peer reviewers will review all eligible Validation grant applications. However, if an application does not meet the definition of moderate evidence in this notice, the Department will not consider the application for funding.

Successful applicants for Validation grants will receive more funding than successful applicants for Development grants.

(3) Development grants provide funding to support high-potential and relatively untested practices, strategies, or programs whose efficacy should be systematically studied. An applicant must provide evidence that the proposed practice, strategy, or program, or one similar to it, has been attempted previously, albeit on a limited scale or in a limited setting, and yielded promising results that suggest that more formal and systematic study is warranted. An applicant must provide a rationale for the proposed practice, strategy, or program that is based on research findings or reasonable hypotheses, including related research or theories in education and other sectors. Thus, applications for Development grants do not need to provide the same level of evidence to support the proposed project as is required for Validation or Scale-up grants.

An applicant for a Development grant must estimate the number of students to be served by the project, and provide evidence of the applicant’s ability to implement and appropriately evaluate the proposed project and, if positive results are obtained, its capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to further develop and bring the project to a larger scale directly or through partners either during or following the grant period. As noted earlier, we recognize that LEAs are not typically responsible for taking to scale their practices, strategies, or programs. Again, however, all applicants can and should partner with others to disseminate and take to scale their effective practices, strategies, and programs.

Peer reviewers will review all eligible Development grant applications. However, if an application is not supported by a reasonable hypothesis for the proposed project, the Department will not consider the application for funding.

To summarize, in terms of the evidence required to support the proposed practice, strategy, or program, the major differences between Scale-up, Validation, and Development grants are (see Table 1): (1) the strength of the research; (2) the significance of the effect; and (3) the magnitude of the effect.

Table 1.  Differences Between the Three Types of Investing in Innovation Fund Grants in Terms of the Evidence Required to Support the Proposed Practice, Strategy, or Program

	Scale-up grants	Validation grants	Development grants
Strength of Research	Strong evidence	Moderate evidence	Reasonable hypotheses
Internal Validity (Strength of Causal Conclusions) and External Validity (Generalizability)	High internal validity and high external validity	(1) High internal validity and moderate external validity; or (2) moderate internal validity and high external validity	Theory and reported practice suggest the potential for efficacy for at least some participants and settings
Prior Research Studies Supporting Effectiveness or Efficacy of the Proposed Practice, Strategy, or Program	(1) More than one well-designed and well-implemented experimental study or well-designed and well- implemented quasi-experimental study; or (2) one large, well-designed and well- implemented randomized controlled, multisite trial	(1) At least one well-designed and well-implemented experimental or quasi-experimental study, with small sample sizes or other conditions of implementation or analysis that limit generalizability; (2) at least one well-designed and well-implemented experimental or quasi-experimental study that does not demonstrate equivalence between the intervention and comparison groups at program entry but that has no other major flaws related to internal validity; or (3) correlational research with strong statistical controls for selection bias and for discerning the influence of internal factors	(1) Evidence that the proposed practice, strategy, or program, or one similar to it, has been attempted previously, albeit on a limited scale or in a limited setting, and yielded promising results that suggest that more formal and systematic study is warranted; and (2) a rationale for the proposed practice, strategy, or program that is based on research findings or reasonable hypotheses, including related research or theories in education and other sectors
Practice, Strategy, or Program in Prior Research	The same as that proposed for support under the Scale-up grant	The same as, or very similar to, that proposed for support under the Validation grant	The same as, or similar to, that proposed for support under the Development grant
Participants and Settings in Prior Research	Participants and settings included the kinds of participants and settings proposed to receive the treatment under the Scale-up grant	Participants or settings may have been more limited than those proposed to receive the treatment under the Validation grant	Participants or settings may have been more limited than those proposed to receive the treatment under the Development grant
Significance of Effect	Effect in prior research was statistically significant, and would be likely to be statistically significant in a sample of the size proposed for the Scale-up grant	Effect in prior research would be likely to be statistically significant in a sample of the size proposed for the Validation grant	Practice, strategy, or program warrants further study to investigate efficacy
Magnitude of Effect	Based on prior research, substantial and important for the target population for the Scale-up project	Based on prior research, substantial and important, with the potential of the same for the target population for the Validation project	Based on prior implementation, promising for the target population for the Development project

In addition, the three types of grants differ in terms of the expectations to scale up successful projects during or following the grant period, either directly or through partners, and the level of funding that would be available. (See Table 2.)

Table 2. Differences Between the Three Types of Investing in Innovation Fund Grants in Terms of Expectations to Scale Up and the Funding to be Provided

	Scale-up grants	Validation grants	Development grants
Scale up	National, Regional, or State	Regional or State	Further develop and scale
Funding to be provided	Highest	Moderate	Modest

MAJOR CHANGES IN THE FINAL PRIORITIES, REQUIREMENTS, DEFINITIONS, AND SELECTION CRITERIA

The Department published a notice of proposed priorities, requirements, definitions, and selection criteria (NPP) for this program in the Federal Register on October 9, 2009 (74 FR 52214-52228). We received comments on the NPP from 346 commenters, including from LEAs, nonprofit organizations, professional associations, parents, and private citizens. We used these comments to revise, improve, and clarify the priorities, requirements, definitions, and selection criteria. In addition to minor technical and editorial changes, there are several substantive differences between the priorities, requirements, definitions, and selection criteria proposed in the NPP and the final priorities, requirements, definitions, and selection criteria that we establish in this notice. Those substantive changes are summarized in this section and discussed in greater detail in the Analysis of Comments and Changes that follows. We do not discuss minor technical or editorial changes, nor do we address comments that suggested changes that we are not authorized to make under the law.

Priorities

We are making the following changes to the priorities for this program:

● We are revising Absolute Priority 1--Innovations that Support Effective Teachers and School Leaders by substituting the term “principal” for the term “school leader” and clarifying that, to meet this priority, projects must increase the number or percentages of highly effective teachers or principals or reduce the number or percentages of ineffective teachers or principals; projects need not serve both teachers and principals to meet the priority. We are also revising the discussion of the teacher and principal evaluation systems that should be used in projects under this priority by stating that the measures used to determine effectiveness should be designed with teacher and principal involvement.

● We are revising Absolute Priority 3--Innovations that Complement the Implementation of High Standards and High-Quality Assessments to clarify that an eligible applicant must propose a project that is based on standards that are at least as rigorous as its State’s standards. Further, we are revising the priority to clarify that if the proposed project is based on standards other than those adopted by the eligible applicant’s State, the applicant must explain how the standards are aligned with and at least as rigorous as the eligible applicant’s State’s standards as well as how the standards differ.

● We are revising Absolute Priority 4--Innovations That Turn Around Persistently Low-Performing Schools to specify the schools for which reform projects may be implemented under this priority; as noted later in this section, we are removing the definition of persistently low-performing schools. In addition, we are revising the priority to include in paragraph (a) additional examples of the comprehensive intervention approaches to whole-school reform and to clarify in paragraph (b)(3) the examples for creating multiple pathways for students to earn regular high school diplomas.

● We are revising Competitive Preference Priority 7–-Innovations to Address the Unique Learning Needs of Students with Disabilities and Limited English Proficient Students by specifying that, to meet this priority, projects must focus on particular practices, strategies, or programs that are designed to improve academic outcomes, close achievement gaps, and increase college- and career-readiness, including increasing high school graduation rates (as defined in this notice), for these students.

Requirements

We are making the following changes to the requirements for this program:

● We are making clarifying changes to the requirements in order to better differentiate between eligible applicants (i.e., LEAs, under section 14007(a)(1)(A) of the ARRA; and partnerships between nonprofit organizations and (1) one or more LEAs or (2) a consortium of schools, under section 14007(a)(1)(B) of the ARRA) and the applicant (i.e., the single entity that applies to the Department on behalf of the eligible applicant, which could be itself or a section 14007(a)(1)(B) partnership).

● As discussed in the NPP, proposed paragraphs (1) through (4) of the eligibility requirements of this program repeated requirements prescribed by section 14007 of the ARRA. Section 307 of Division D of the Consolidated Appropriations Act, 2010 (P.L. 111-117), which was signed into law on December 16, 2009, makes several amendments to these statutory requirements, which we are incorporating in the final eligibility requirements. The major substantive changes include the following:

● Consistent with the amendments to section 14007(b) of the ARRA, we are revising proposed paragraph (1) of the eligibility requirements to require that, to be eligible for an award under this program, an eligible applicant must (A) have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the Elementary and Secondary Education Act of 1965, as amended (ESEA), or (B) have demonstrated success in significantly increasing student academic achievement for all groups of students described in such section. We are also eliminating proposed paragraph (2) of the eligibility requirements, which would have required that an eligible applicant have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years or have demonstrated success in significantly increasing student achievement for all groups of students described in that section through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress).

● Consistent with the amendments to section 14007(c) of the ARRA, we are revising the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization to specify that an eligible applicant that includes a nonprofit organization is considered to have met paragraph (1) and paragraph (2) (proposed paragraph (3)) of the eligibility requirements for this program if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. In addition, we are revising the Note to specify that an eligible applicant that includes a nonprofit organization is considered to have met paragraph (3) (proposed paragraph (4)) of the eligibility requirements if it demonstrates that it will meet the requirement relating to private-sector matching.

● We are establishing a requirement that, to be eligible for an award, an application for a Scale-up grant must be supported by strong evidence (as defined in this notice), an application for a Validation grant must be supported by moderate evidence (as defined in this notice), and an application for a Development grant must be supported by a reasonable hypothesis.

● We are revising the Cost Sharing or Matching requirement with respect to the timing of submission of evidence of the private-sector match. Selected eligible applicants are now required to submit evidence of the full 20 percent private-sector matching funds to support the proposed project following the peer review of applications. An award will not be made unless the eligible applicant provides adequate evidence that the full 20 percent private-sector match has been committed or the Secretary approves the eligible applicant’s request to reduce the matching-level requirement.

● Section 307 of Division D of the Consolidated Appropriations Act, 2010, amended the ARRA with respect to a grantee’s ability to make subgrants under this program. Under new section 14007(d) of the ARRA, in the case of an eligible entity that is a partnership between a nonprofit organization and (1) one or more LEAs or (2) a consortium of schools, the partner serving as the fiscal agent (i.e., the applicant applying on behalf of the eligible applicant) may make subgrants to one or more of the other entities in the partnership (referred to in this notice as official partners). We are revising the requirements for this program to reflect this statutory change.

● We are establishing limits on grant awards. No grantee may receive more than two grant awards under this program. In addition, no grantee may receive more than $55 million in grant awards under this program in a single year’s competition.

● We are revising the Evaluation requirement to establish that, in addition to making the results of any evaluation broadly available, Scale-up and Validation grantees must also ensure the data from their evaluations are made available to third-party researchers consistent with applicable privacy requirements.

Definitions

We are making the following changes to the definitions for this program. In addition to providing further clarity on the meaning of terms, these changes are intended to ensure consistency in the use and definition of terms in this program and other programs supported with ARRA funds where appropriate.

● We are removing the term persistently low-performing schools.

● We are replacing the term highly effective school leader with highly effective principal and revising the definition of this term.

● We are revising the definitions of the following terms: formative assessment, highly effective teacher, high-need student, regional level, and student achievement.

● We are adding definitions of the following terms: applicant, official partner, other partner, high school graduation rate, regular high school diploma, and well-designed and well-implemented (with respect to an experimental or quasi-experimental study).

Selection Criteria

We are making the following changes to the selection criteria for this program:

● Consistent with the Eligible Applicants requirement and the definitions of applicant, official partner, and other partner, we are revising the selection criteria for this program, where appropriate, to clarify the entities for which the criteria apply.

● We no longer intend to use a two-tier process to review applications for Development grants. Thus, we are removing, from the selection criteria for Development grants the discussion of a two-tier application process (including pre-applications) for those grants.

● We are revising Selection Criterion A (Need for the Project and Quality of the Project Design) for Validation grants to include, among the factors for which the Secretary will consider the quality of the proposed project design, the extent to which the proposed project is consistent with the research evidence supporting the proposed project, taking into consideration any differences in context.

● We are revising Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect) for all three types of grants to include college enrollment and completion rates among the student achievement and attainment outcomes for which the Secretary will consider the effect of a proposed project. In addition, we are revising the criterion for Scale-up and Validation grants to clarify that the strength of the existing research evidence includes the internal validity (strength of causal conclusions) and external validity (generalizability) of the effects reported in prior research. We are also revising the criterion for Development grants to clarify that the strength of the existing research evidence includes reported practice, theoretical considerations, and the significance and magnitude of any effects reported in prior research.

● We are revising Selection Criterion C (Experience of the Eligible Applicant) for all three types of grants to reflect the amendments to the authorizing statute discussed earlier in this notice. Under Selection Criterion C(2) (proposed Selection Criterion C(2)(b)), the Secretary now considers, in the case of an eligible applicant that is an LEA, the extent to which the eligible applicant provides information and data demonstrating that it has (A) significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA or significantly increased student achievement for all groups of students described in such section; and (B) made significant improvements in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and principals, as demonstrated with meaningful data. In the case of an eligible applicant that includes a nonprofit organization, the Secretary now considers the extent to which the eligible applicant provides information and data demonstrating that the nonprofit organization has significantly improved student achievement, attainment, or retention through its record of work with an LEA or schools. These changes are consistent with the changes to the eligibility requirements for this program discussed earlier in this notice.

ANALYSIS OF COMMENTS AND CHANGES

An analysis of the comments received on, and any changes to, the priorities, requirements, definitions, and selection criteria since publication of the NPP for this program follows.

Note about general comments: We received many comments expressing general support or making general recommendations for this program. In most cases, these comments were effectively duplicated by other comments expressing support or making specific recommendations for the program’s proposed priorities, requirements, definitions, or approval criteria, which we discuss in the sections that follow. We accordingly do not discuss those general comments here. In other cases, we interpreted a general comment as applying specifically to the priorities, requirements, definitions, or selection criteria and address the comment.

Note about comments on program issues not covered in NPP: We received a number of comments relating to program issues that may have been discussed in communications from the Department but were not proposed for public comment in the NPP for this program. These issues include: specific funding ranges or award amounts for the grant categories, the number of grant awards, uses of funds, length of grant periods, and technical assistance for applicants. We do not address comments on these issues here. We note, however, that information on these issues will be made available through other Department documents including the notice inviting applications for this program.

TYPES OF GRANTS

Comment: A number of commenters expressed support for this program’s three-tiered grant structure. Several commenters supported the Department’s attempt to balance the need to cultivate new programs with support for existing programs proven to be effective. However, a number of commenters recommended revising the grant categories or structure of the program. Many commenters recommended that the Department structure the program to include only two types of grants--Scale-up grants and Development grants--and to eliminate Validation grants. Similarly, one commenter recommended that the Validation and Scale-up grants be merged into a single category so that reviewers could consider the size of the target population, the complexity of the project, and other factors without restrictions on scaling targets. A number of other commenters recommended that the Department change the structure of this program to focus on funding a large number of small projects rather than larger projects that would be supported under the Scale-up grant category.

Discussion: The Department believes that the structure of this program and the use of three categories of grants present an appropriate balance between support for the development of promising yet relatively untested ideas and the growth and scaling of practices that have made demonstrable improvements in student achievement and attainment outcomes. In addition, we believe that the scaling targets provided for the three grant types are needed by applicants in developing their proposed projects. Consequently, we do not believe changes such as those recommended by the commenters are warranted.

Changes: None.

Comment: One commenter asked how the scale of implementation (State, regional, or national) differs between Validation and Development grants.

Discussion: Validation grants will be implemented on a broader scale than Development grants because of both the corresponding level of evidence and the funding provided for the practice, strategy, or program. The level of implementation for Validation grants is State or regional, but the level of implementation for Development grants would typically not extend to a statewide level.

Changes: None.

Comment: One commenter requested that the Department remove the term “new” from the description of the Development grants, noting that practices that are promising and untested (consistent with this category of grant) may not necessarily be new.

Discussion: We agree with the commenter that the practices proposed in projects for a Development grant need not necessarily be new. We are removing the term “new” from the description of the Development grants.

Changes: We are removing the term “new” from the description of the Development grants.

Comment: One commenter recommended that the Department allow, under the Development grant category, funding for small-scale projects that focus on the needs of relatively small populations of high-need students.

Discussion: An applicant would not be prohibited from proposing under the Development grant category a project that focuses on small populations of high-need students, provided that the project addresses one of the absolute priorities of the program.

Changes: None.

PRIORITIES

Priorities -- General

Comment: One commenter suggested that the Department draw explicit connections between the final priorities in the Investing in Innovation Fund and the final priorities in the Race to the Top Fund program so that projects can be successfully scaled at the State level. Another commenter recommended adding a competitive preference priority for projects that are aligned with activities supported by other programs administered by the Department (e.g., School Improvement Grants, Education Technology Grants, Teacher Quality Enhancement Grants) or by other Federal agencies (e.g., Community Development Block Grants).

Discussion: The absolute priorities under the Investing in Innovation Fund are aligned with the four education reform areas under the ARRA and complement the absolute priority of the Race to the Top Fund program, which requires States to submit applications that comprehensively address these same four reform areas. As noted elsewhere in this notice, we are revising the priorities, requirements, definitions, and selection criteria for this program, as appropriate, to ensure consistency between this program and other programs supported with ARRA funds, including the Race to the Top Fund program.

We encourage eligible applicants to align and coordinate activities under this program with activities supported with other ARRA funding, as well as activities funded through other Department and Federal programs. Because this program is designed to align with the ARRA’s four education reform areas and complement activities in other programs supported with ARRA funds, we do not believe it is necessary to add a competitive preference priority for eligible applicants that align and coordinate activities and funding from multiple sources.

Changes: None.

Comment: A number of commenters expressed support for the four absolute priorities as reflecting key areas where reform is needed in education. One commenter, however, expressed concern that requiring applicants to submit an application under one absolute priority contributes to a “silo effect” whereby individual projects are narrowly focused and implemented in isolation or in a manner that is disconnected from other key reform areas.

A few commenters requested clarification as to whether applicants could or should address more than one absolute priority. Some commenters recommended adding an absolute priority for projects that are based on comprehensive and multi-dimensional reform strategies that cut across the education reform areas. Other commenters recommended adding a competitive preference priority for projects that address more than one absolute priority or that address one absolute priority and demonstrate capacity and expertise in other absolute priority areas. Commenters also recommended that the Department require applicants to describe their work in each of the education reform areas, or how their proposed project would contribute to improvements across the spectrum of education reform. Some of these commenters asserted that lasting reform requires action in multiple or all of the ARRA reform areas.

Discussion: An applicant must identify one absolute priority under which it is submitting its application. Given the diversity of applications that we are likely to receive, we are requiring eligible applicants to write to one absolute priority to ensure that we can assess the quality of the applications within a given reform area. Although it must identify the absolute priority under which it is submitting its application, an eligible applicant is not prohibited from submitting an application that addresses multiple absolute priorities if that is necessary to describing the effort for which the applicant is seeking funds. However, such applications will not receive additional “credit” for doing so. All points will be assigned based on how well the eligible applicant addresses the selection criteria.

Changes: None.

Comment: A number of commenters recommended that the Department add a competitive preference priority for applications that serve high-need students. Several of these commenters stated that including a priority for projects that focus on high-need students would promote innovation and direct attention toward meeting the needs of these typically underserved students. Two of these commenters also recommended including a competitive preference priority for innovative programs in literacy instruction for students in secondary schools. Several commenters recommended adding a competitive preference priority for projects that propose to serve disconnected youth, particularly youth in secondary schools and youth who have dropped out of school. Other commenters recommended focusing on projects that propose to create or improve pathways to postsecondary education for high-need and disconnected students. One commenter suggested focusing priorities on projects that serve economically disadvantaged students, Native American students, and students from diverse ethnic and racial backgrounds.

Discussion: Under the requirements for this program, all projects funded under this program must focus on high-need students (as defined in this notice). It would, therefore, serve no purpose also to award competitive preference points for projects that propose to serve high-need students. We note that we define high-need student as a student at risk of educational failure or otherwise in need of special assistance and support. While we provide examples of students at risk of educational failure or otherwise in need of special assistance and support in the definition of high-need student, those examples are not intended to be an exhaustive or exclusive list. An eligible applicant has flexibility in determining the types of students that meet the definition.

Changes: None.

Comment: One commenter recommended that the Department clarify whether an applicant may propose to serve only certain student subgroups or students only in specific settings. The commenter requested that the Department clarify the relationship between the competitive preference priorities, which target specific groups of students (e.g., students with disabilities and limited English proficient students), and the absolute priorities, which do not appear to be so targeted. Another commenter suggested clarifying whether applications targeting multiple student subgroups would receive competitive preference points.

Discussion: An eligible applicant may propose a project that targets or serves only certain student subgroups or only students served in particular settings, provided that the project serves high-need students consistent with the definition of high-need student. However, an eligible applicant would not receive competitive preference points under this program simply for proposing a project to serve multiple student subgroups.

Changes: None.

Comment: We received numerous comments recommending that we add absolute priorities to address a wide array of other issues and concerns. Many commenters recommended that absolute priorities be added to focus on particular subject areas. For example, commenters suggested adding a priority for projects that improve vocabulary and increase the use of vocabulary assessments. One commenter recommended adding a priority for innovations in science education. Another commenter recommended adding a priority for eligible applicants that propose innovative ways to instruct students in the subjects of science, technology, engineering and mathematics (STEM). A number of other commenters suggested adding a priority for projects that propose to improve, reform, or increase access to art and music education. A few commenters recommended adding a priority for innovations in career and technical education and focusing on career-readiness outcomes, such as technical skill attainment and performance on work-readiness assessments.

A few commenters recommended adding an absolute priority for innovations that offer customized educational experiences for students based on individual learning needs and preferences. Two of these commenters asserted that such innovations provide a more flexible, student-centered approach to education and produce schools that are “student-based.”

Several commenters suggested adding an absolute priority for projects that propose to increase high school graduation rates, such as dropout recovery programs. Other commenters recommended adding an absolute priority for projects that focus on college readiness and transition to college. One commenter recommended that the absolute priorities explicitly reference middle schools because, according to the commenter, middle schools provide the foundation for high school graduation and college- and career-readiness.

In addition to recommendations to add absolute priorities, we received a number of comments recommending that we re-designate competitive preference priorities as absolute priorities. For example, a few commenters recommended changing the competitive preference priority on serving schools in rural LEAs to an absolute priority. Likewise, one commenter recommended that the competitive preference priority on supporting college access and success be changed to an absolute priority and several commenters recommended that the competitive preference priority on improving early learning outcomes be changed to an absolute priority.

Discussion: While we recognize the importance of the issues and topics mentioned by the commenters, we decline to include additional absolute priorities for this program. As stated elsewhere in this notice, the Department is using the Investing in Innovation Fund to support the overarching ARRA goal of improving student achievement and attainment by establishing four absolute priorities that are directly aligned with the four education reform areas under the ARRA. We believe that adding other absolute priorities would detract from this goal.

We note, in addition, that all applications for Investing in Innovation Fund grants will be assessed in part on the extent to which the proposed projects will have an impact on student achievement and attainment outcomes including the following: improving student achievement or growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, and increasing college enrollment and completion rates (see Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect) for each type of grant).

Changes: None.

Comment: One commenter recommended eliminating all the competitive preference priorities stating that they complicate the application process and constrain innovation.

Discussion: The Department routinely utilizes competitive preference priorities in grant competitions without any undue difficulty for either the agency or applicants. As noted elsewhere in this notice, we are including competitive preference priorities that are aligned with the Department’s reform goals. We believe that these competitive preference priorities complement, rather than detract from, the four ARRA reform areas. Furthermore, we do not believe that including competitive preference priorities constrains innovation. We have written the competitive preference priorities around broad general topics, within which eligible applicants are free to propose a range of innovative projects. We note that eligible applicants are not required to address the competitive preference priorities. For these reasons, we have concluded that no changes to the competitive preference priorities should be made in response to this comment.

Changes: None.

Comment: One commenter recommended that the Department change the absolute priorities to competitive preference priorities because, according to the commenter, the competitive preference priorities deserve equal status with the absolute priorities. One commenter recommended combining some of the absolute priorities with the competitive preference priorities.

Discussion: Changing the absolute priorities to competitive preference priorities or combining absolute priorities with competitive priorities would, in effect, diminish the focus of this program on the four ARRA education reform areas because it would allow projects that do not address any of the four reform areas to be funded. Therefore, we decline to make the changes recommended by commenters.

Changes: None.

Comment: One commenter recommended that the Department permit applicants to address more than one competitive preference priority. Several commenters recommended that the Department clarify whether applications receive additional points for addressing more than one competitive preference priority.

Discussion: The notice inviting applications for this program (NIA), published elsewhere in this issue of the Federal Register, states that competitive preference points will be awarded on an “all or nothing” basis (i.e., one point or zero points) for Competitive Preference Priorities 5, 6, and 7, depending on how well an application addresses the priority. For Competitive Preference Priority 8, we will award up to two points, depending on how well an application addresses this priority. Applications may address more than one competitive preference priority; however, the Department will not award additional points simply for addressing more than one competitive preference priority.

Changes: None.

Comment: A number of commenters recommended that the Department add a competitive preference priority for applicants that partner with specific entities. For example, some commenters recommended adding a competitive preference priority for applicants that partner with nonprofit organizations in order to help ensure that projects are innovative and can be scaled up successfully. One commenter stated that competitive preference points should be awarded to LEA applicants who propose projects that involve collaboration with other LEAs and charter schools. Another commenter recommended adding a competitive preference priority for applicants that partner with a State educational agency to ensure that funded activities can be implemented statewide. One commenter suggested that applicants who partner with institutions of higher education should be given competitive preference points in light of the focus of the ARRA on improving college- and career-readiness. One commenter recommended adding a competitive preference priority for applicants that partner with a community-based organization in order to be consistent with the Department’s general support for community-oriented schools and partnerships between communities and schools. A few commenters recommended adding a competitive preference priority for applicants that propose innovative partnerships to support program effectiveness and sustainability including interdisciplinary partnerships.

Discussion: We believe that eligible applicants should form partnerships with those entities that they believe will yield the best possible application and produce the best possible results. We do not believe it would be appropriate for the Department to judge who the best partners would be for a particular project and therefore decline to add a competitive preference priority for eligible applicants that partner with a specific entity.

We note that there appears to be some confusion about the roles and responsibilities of “eligible applicants,” “applicants,” “fiscal agents,” and “partners” under this program. Therefore, and as discussed in greater detail in the REQUIREMENTS section of this preamble, we are adding definitions for the terms applicant, official partner, and other partner and using these terms, as appropriate, throughout this notice.

Changes: As discussed elsewhere in this notice, we are adding definitions for the terms applicant, official partner, and other partner. We use these terms, as appropriate, throughout this notice.

Comment: One commenter recommended adding a competitive preference priority for eligible applicants that include charter schools in their proposed projects.

Discussion: As discussed elsewhere in this notice, depending on its legal status under State law, a charter school may be eligible to apply under this program in the following ways: as an LEA on its own if it is considered an LEA under State law; as a nonprofit organization, in a partnership with one or more LEAs or a consortium of schools (provided the charter school meets the definition of nonprofit organization under this program); or in a partnership with a nonprofit organization as an LEA or as part of a consortium of schools. Adding a competitive preference priority for charter schools would provide an unfair advantage to eligible applicants that include these schools. Therefore, we decline to make the change recommended by the commenter.

Changes: None.

Comment: One commenter recommended adding a competitive preference priority for eligible applicants that propose projects that encourage and support effective teacher professional development and collaboration. Another commenter recommended adding a competitive preference priority for projects that propose innovative approaches to attracting and developing school leaders.

Discussion: Absolute Priority 1 focuses on projects that increase the number or percentages of highly effective teachers or principals (or reduce the number or percentages of ineffective teachers or principals) by identifying, recruiting, developing, placing, rewarding, and retaining highly effective teachers or principals (or removing ineffective teachers or principals). It is unnecessary to include both an absolute priority and a competitive preference priority focused on improving teacher or principal effectiveness. Therefore, we decline to follow the commenters’ recommendations.

As explained in our responses to comments regarding Absolute Priority 1, we are changing the term, “school leader” to “principal” in order to clarify our intent to focus this priority on increasing the number and percentages of highly effective principals.

Changes: We are changing the term “school leader” to “principal” in the priorities, requirements, definitions, and selection criteria for this program and using the latter term in our response to comments. However, we are retaining references to “school leader” that commenters made in their statements.

Comment: A number of commenters recommended that the Department add a priority for projects that focus on improving outcomes related to school support services, school climate, school diversity, school safety, or parent or community involvement. Some commenters recommended adding a competitive preference priority for eligible applicants that propose initiatives to promote caring and culturally-responsive teachers as well as classrooms and schools that support positive social climates. One commenter recommended adding a competitive preference priority for projects that propose innovative approaches to reducing the use of alcohol, tobacco, and other addictive drugs. Another commenter recommended adding a competitive preference priority for projects proposing innovative approaches to reducing the incidence of crime, violence, and “uncivil behavior” (including bullying) in schools. One commenter recommended that the Department modify the proposed priorities to address student engagement, character education, and life skills; the commenter asserted that the proposed priorities ignore factors not directly associated with instruction that impact a student’s ability to achieve academically.

Some commenters recommended adding a priority for innovations that improve the social and other nonacademic supports that schools provide to students and families, such as assistance with child care, housing, transportation, and making college-related decisions. A number of commenters recommended adding a new priority or revising the proposed priorities to support innovative approaches to increase parental involvement. One commenter recommended focusing specifically on parent and community involvement in education in rural LEAs because, according to the commenter, rural LEAs face barriers such as limited transportation options, limited extracurricular programming, and limited community-based educational resources in promoting parent and community involvement in education.

One commenter recommended adding an absolute priority that would require all projects to promote diverse student populations in schools with respect to demographic factors such as race, ethnicity, and parent socioeconomic status and educational attainment. A few commenters recommended adding an absolute priority for innovative reforms to reduce racial and economic segregation and isolation and to assess the potential effects of a proposed project on the racial and economic segregation and isolation of students. One commenter suggested that the Department emphasize increasing the economic and racial diversity of institutions of higher education.

A number of commenters recommended that the Department add an absolute or competitive preference priority for innovative projects that engage communities in education reform, including increasing the representation of community stakeholders in reform-oriented policy- and decision-making.

Discussion: While we recognize the importance of the issues and topics mentioned by the commenters, we decline to include additional priorities or revise the proposed priorities for this program as the commenters recommend. As stated elsewhere in this notice, the Department is using the Investing in Innovation Fund to support the overarching ARRA goal of improving student achievement and attainment. All applications for Investing in Innovation Fund grants will be assessed in part on the extent to which the proposed projects will have an impact on student achievement and attainment outcomes including the following: improving student achievement or growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, and increasing college enrollment and completion rates. However, in providing evidence of the effects of their proposed projects, eligible applicants may also utilize intermediate variables that are strongly correlated with improving those outcomes (see Selection Criteria). These intermediate variables may include variables on the issues and topics mentioned by the commenters.

Changes: None.

Comment: A few commenters recommended that the Department include a competitive preference priority related to data collection and evaluation of project outcomes. One commenter recommended adding a competitive preference priority for eligible applicants that use systems for collecting project data that produce high-quality, reliable, and comparable data in order to ensure that funded projects can be properly evaluated. Another commenter recommended requiring systems for collecting project data to be created or utilized to support the innovation pursued under the priority. One commenter recommended adding a competitive preference priority for consortia applicants that demonstrate the capacity to collect and analyze consortium-level project data (as opposed to State-level data). Another commenter recommended that the Department add a competitive preference priority for applicants that propose innovative designs for evaluating project implementation and for disseminating project results and best practices.

Discussion: Under the requirements for this program, any eligible applicant receiving funds must conduct an independent evaluation of its proposed project and comply with the requirements of any evaluation of the program conducted by the Department (see Evaluation requirement). Further, all applications will be judged in part on the quality of the eligible applicant’s plan to evaluate its proposed project (see Selection Criterion D (Quality of the Project Evaluation)). Therefore, it is unnecessary to include a competitive preference priority focused on data collection and project evaluation as suggested by the commenters.

Changes: None.

Comment: One commenter recommended that the Department add a competitive preference priority for an applicant that provides confirmation in its application that it has secured matching funds from the private sector or philanthropic community.

Discussion: To be eligible for an award under this program, an eligible applicant must demonstrate that it has established one or more partnerships with an entity or organization in the private sector, which may include philanthropic organizations, and that the entity or organization in the private sector will provide matching funds in order to help bring project results to scale. Further, the Cost Sharing or Matching requirement for this program specifies that an eligible applicant must obtain matching funds or in-kind donations from the private sector equal to at least 20 percent of its grant award. Because these requirements apply to all applicants, it would serve no purpose to give competitive preference to eligible applicants that confirm receipt of matching funds in their applications. Therefore, we decline to make the change requested by the commenter.

Changes: None.

Comment: A few commenters recommended that the Department add a competitive preference priority for eligible applicants that propose projects that are based on well-conducted experimental studies or that have demonstrated records of success in implementing or scaling up research-based projects. Another commenter recommended that the Department investigate whether society believes it is morally imperative that educational practices be based on rigorous research.

Discussion: All applications will be judged in part on the strength of the research in support of the proposed project and on the experience of the applicant (see Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect) and Selection Criterion C (Experience of the Eligible Applicant)). Therefore, we do not believe it is necessary to add the competitive preference priority recommended by the commenter.

With regard to the recommendation that the Department investigate whether society believes it is morally imperative that educational practices be based on rigorous research, this is not the purpose of the Investing in Innovation Fund. Therefore, we decline to follow the commenter’s recommendation.

Changes: None.

Comment: One commenter recommended that the Department add a competitive preference priority for projects that would be implemented throughout a city or urban area.

Discussion: We decline to add the competitive preference priority suggested by the commenter because all applications will be evaluated in part based on the eligible applicant’s strategy and capacity to bring the proposed project to scale (see Selection Criterion E (Strategy and Capacity to Bring to Scale (in the case of Scale-up and Validation grants); Strategy and Capacity to Further Develop and Bring to Scale (in the case of Development grants)). As noted elsewhere in this notice, the extent to which an eligible applicant will bring its proposed project to scale will vary with the type of grant for which the eligible applicant applies (i.e., Development, Validation, or Scale-up grant).

Changes: None.

Comment: We received the following other recommendations for additional competitive priorities. A few commenters recommended that the Department add a competitive preference priority for projects that are designed to reduce resource inequities between LEAs. Other commenters recommended adding a competitive preference priority for technology-based projects or projects that increase the integration of technology into the classroom. Another commenter recommended that the Department add a competitive preference priority for projects that focus on performance-based systems that use competency-based instruction.

One commenter recommended adding a competitive preference priority for projects that propose to utilize effective education models from other countries including countries that excel on international assessments of educational achievement, such as the Program for International Student Assessment (PISA). Another commenter recommended revising the proposed priorities to emphasize the creation of “vertically-integrated systemic innovation zones.” One commenter recommended that the Department add an invitational priority for innovations in the development, use, and dissemination of open educational resources; the commenter asserted that using these resources is a cost-effective and sustainable strategy to scale up successful innovations.

Discussion: Similar to the approach we have taken with the absolute priorities, we decline to add more competitive preference priorities in order to maintain focus on the other major priorities of the Department that are reflected in the competitive preference priorities. Accordingly, we decline to include additional absolute or competitive preference priorities or an invitational priority, as recommended by the commenters.

Changes: None.

Absolute Priority 1--Innovations That Support Effective Teachers and Principals (Proposed Absolute Priority 1--Innovations That Support Effective Teachers and School Leaders)

Comment: One commenter recommended that the Department revise this absolute priority to include support for related services professionals, including school psychologists, school social workers, counselors, and speech-language pathologists, in order to reflect the contributions of these professionals to student learning.

Discussion: We decline to expand Absolute Priority 1 in this manner. While we appreciate the important role that such professionals play in supporting student achievement and attainment, we believe that the focus of the priority should be on increasing the number and percentages of highly effective teachers and principals (and reducing the number and percentages of ineffective teachers and principals) as teachers and principals are the individuals directly responsible for academic instruction. To clarify our intent, we are changing the term “school leader” to “principal” in this priority and elsewhere in the priorities, requirements, definitions, and selection criteria for this program.

We note that an applicant would not be prohibited from proposing under this priority an innovative strategy, practice, or program that includes support for related services professionals to the extent that this support is intended to increase the number or percentages of highly effective teachers and principals (or reduce the number or percentages of ineffective teachers and principals).

Changes: As noted earlier, we are changing the term “school leader” to “principal” in the final priorities, requirements, definitions, and selection criteria for this program and using the latter term in our response to comments. However, we are retaining references to “school leader” that commenters made in their statements.

Comment: A number of commenters recommended that the Department clarify whether projects under this priority must increase the number of both highly effective teachers and highly effective school leaders.

Discussion: It was not our intent to require projects under this priority to increase the number or percentages of highly effective teachers and highly effective principals (or reduce the number or percentages of ineffective teachers and ineffective principals). Therefore, we are changing the priority to make this clear.

Changes: We are changing Absolute Priority 1 to clarify that, under this priority, the Department provides funding to support practices, strategies, or programs that increase the number or percentages of highly effective teachers or principals (or reduce the number or percentages of ineffective teachers or principals) by identifying, recruiting, developing, placing, rewarding, and retaining highly effective teachers or principals (or removing ineffective teachers or principals).

Comment: A number of commenters expressed support for this absolute priority. One commenter, however, expressed concern that the priority does not address the need to ensure that low-income and minority children are not taught at higher rates than other children by inexperienced, unqualified, or out-of-field teachers, as provided in the ARRA. Another commenter stated that the Department’s citations to research on teacher effectiveness ignore a body of research that shows that some teacher “inputs” (such as teacher qualifications) have an impact on student achievement.

Discussion: Absolute Priority 1 focuses on practices, strategies, or programs that increase the number or percentages of highly effective teachers or principals (or reduce the number or percentages of ineffective teachers or principals), especially for high-need students. We chose to focus this priority on teacher and principal effectiveness rather than on teacher qualifications. Historically, in assessing the quality of our nation’s teachers, the Department has focused, through “highly qualified teacher” measures, on teacher qualifications (e.g., years of experience, types and numbers of certifications) to the exclusion of other factors. By including considerations of teacher effectiveness in the ARRA assurance in this reform area, we believe the Congress has signaled that this focus is unnecessarily narrow and that other measures of teacher quality are needed--and, in particular, measures that are associated more closely with the outcomes of teaching and learning than with “inputs” such as qualifications. We intend to promote those measures with this priority and believe that focusing the priority on increasing the number or percentages of highly effective teachers or principals (or reducing the number or percentages of ineffective teachers or principals) is consistent with the ARRA in this regard. Furthermore, we believe that this focus will help ensure that there is an equitable distribution of highly effective teachers and principals across LEAs and schools.

Changes: None.

Comment: One commenter recommended that the Department revise the priority to state that teacher and school leader evaluation systems should be objective, transparent, and fair, rather than rigorous, transparent, and fair.

Discussion: We believe that evaluation systems that are rigorous would necessarily be objective. Further, we believe that it is important that such systems be held to high standards of design, which is best captured by the term “rigorous.” In addition, we use “rigorous, transparent, and fair” to ensure consistency in the use of terms across programs supported with ARRA funds. Therefore, we decline to make the change suggested by the commenter.

Changes: None.

Comment: Several commenters recommended that the Department eliminate the requirement for teacher and school leader evaluation systems to include student growth as a significant factor. One of these commenters stated that there is nothing in the ARRA that refers to or encourages the use of student growth data in teacher and school leader evaluation systems. Several commenters also stated that there are limitations and methodological difficulties in accurately and fairly isolating individual teacher effects on student achievement. Another commenter stated that impacts on student performance or growth should be estimated only at the school level because schools are professional communities in which teachers and school leaders contribute collectively to student achievement.

Discussion: Under this priority, we encourage projects that propose methods of determining teacher and principal effectiveness that use an evaluation system that is rigorous, transparent, and fair; that differentiate performance using multiple rating categories of effectiveness and multiple measures of effectiveness, with data on student growth as a significant factor; and that are designed and developed with teacher and principal involvement. Although there is nothing in the ARRA that refers to using student growth data in teacher and principal evaluation systems, we believe this priority is consistent with the ARRA assurance in this reform area.

With regard to the commenters’ concerns about estimating individual teacher impact on student achievement, we recognize that the methods for providing these estimates may need further study or development. While this priority supports projects that determine teacher effectiveness using student growth as a significant factor, nothing in this priority requires that projects use estimates of individual teacher impact on student achievement to meet the priority or that impacts on student performance or growth be estimated only at the school level. We believe that such decisions are best left to applicants given the specific settings in which they plan to conduct their proposed projects. For these reasons, we have concluded that the changes suggested by the commenters should not be adopted.

Changes: None.

Comment: A number of commenters recommended that the teacher and school leader evaluation systems used by grantees under this priority incorporate multiple measures of effectiveness including measures related to the following: teacher practice; student outcomes such as results of written work, portfolios, and group and individual performances and presentations; other student factors such as engagement, socioeconomic status, and mobility; factors such as school safety, climate, and resources; and parent engagement in student learning. Some of these commenters stated that data on student growth should not be the sole criterion used to evaluate teacher and school leader performance.

Discussion: We did not intend for student growth to be the sole factor in determining teacher or principal effectiveness; rather, the intent was for student growth to be a significant, but not the only, factor. As reflected in the statement of the priority, an eligible applicant should use multiple measures in evaluating teacher and school leader performance, and may use measures such as those recommended by the commenters, provided that student growth data are used as a significant factor.

Changes: None.

Comment: Two commenters agreed that the measures for determining effectiveness in teacher and school leader evaluation systems should be designed and developed with teacher involvement. One commenter, however, recommended revising the priority to require measures used in these systems to be designed and developed with the involvement of school leaders and unions, in addition to teachers. Several other commenters recommended that we revise the priority to require that parents and other members of the community be involved in the design and development of these measures.

Discussion: We agree that teachers and principals should be involved in designing and developing measures of teacher and principal effectiveness and are revising the priority accordingly. With regard to the involvement of other stakeholders mentioned by the commenters, we believe that this is a decision that is best left to local officials.

Changes: We are revising this priority to include a statement that, in addition to teachers, measures of effectiveness should be designed and developed with principal involvement.

Comment: One commenter recommended revising the priority to directly support projects that improve the systems used to evaluate the performance of teachers and school leaders.

Discussion: The Department intends for Absolute Priority 1 to focus on innovative practices, strategies, or programs that increase the number or percentages of highly effective teachers or principals (or reduce the number or percentages of ineffective teachers or principals), especially for high-need students and that will have an impact on improving student achievement and attainment. While the priority addresses aspects of the teacher and principal evaluation systems that projects should use in furtherance of these goals, the Department does not intend for this priority to support the development or improvement of these systems exclusive of those goals. Therefore, we decline to change the priority in the manner suggested by the commenter.

Changes: None.

Comment: One commenter recommended that the Department eliminate, as a goal under this priority, reductions in the number and percentage of ineffective teachers and school leaders. The commenter stated that the only goal that is necessary under this priority is increasing the number and percentage of effective teachers and school leaders.

Discussion: We believe that it is important to remove ineffective teachers from classrooms in addition to increasing the number of effective teachers in classrooms. We also have concluded that the same is true for principals. Therefore, we decline to make the change recommended by the commenter.

Changes: None.

Comment: One commenter recommended that the Department revise the priority to include support for innovations that improve conditions for teaching and learning, such as physical working conditions, administrative supports provided, and availability of resources, because these conditions influence a teacher’s ability to be effective. Two commenters suggested revising the priority to include support for programs that enable school leaders to provide more effective assistance to teachers by improving school organizational structures.

Discussion: Nothing would preclude an applicant from proposing the initiatives mentioned by the commenters under this priority so long as the proposed project increases the number or percentages of highly effective teachers or principals (or reduces the number or percentages of ineffective teachers or principals), especially for high-need students, by identifying, recruiting, developing, placing, rewarding, and retaining highly effective teachers or principals (or removing ineffective teachers or principals). However, we do not believe it is necessary or advisable to change the priority to refer specifically to innovations that improve conditions for teaching and learning. We cannot include in the priority all the possible practices, strategies, or programs that could potentially support effective teachers and principals, nor do we want to restrict or constrain the innovative practices, strategies, and programs that this priority would support. Therefore, we decline to change the priority in the manner suggested by the commenters.

Changes: None.

Comment: We received a number of comments recommending that the Department revise this absolute priority to focus on improving the effectiveness of specific groups of teachers and school leaders in specific settings. One commenter recommended changing the priority to focus on improving the effectiveness of teachers who teach high-need students in low-performing schools. Another commenter recommended that the priority focus on improving the effectiveness of teachers in schools serving Native American students. One commenter stated that the priority should be revised to increase the ability of teachers to effectively teach students in racially and economically diverse schools. Several commenters recommended focusing the priority on projects that improve the effectiveness of teachers and leaders in early childhood and pre-kindergarten programs and one commenter recommended revising the priority to include programs that assist school leaders in integrating pre-kindergarten programs into their schools and LEAs.

Discussion: Under the requirements for this program, projects must serve high-need students (as defined in this notice). Further, this priority supports projects that increase the number or percentages of highly effective teachers or principals (or reduce the number or percentages of ineffective teachers or principals), especially for teachers of high-need students. Provided that proposed projects serve high-need students, there is flexibility in determining the groups of teachers and principals to be served in projects under this priority. Accordingly, we do not believe it is necessary to change the priority to focus on specific groups of teachers and principals in specific settings.

Changes: None.

Comment: A few commenters stated that the Department should revise the priority to focus on instructional effectiveness rather than educator effectiveness and include alternative instructional programs such as online learning and personalized digital content. The commenters asserted that alternative instructional programs are needed to improve instruction in certain subjects, such as STEM subjects.

Discussion: Teachers and principals play a critical role in improving student achievement and attainment outcomes. As stated in the NPP, research indicates that teacher quality is a critical contributor to student learning. Further, studies show that school leadership is a major contributing factor to what students learn at school and that strong teachers are more likely to teach in schools with strong principals. In light of these findings, we do not believe that this absolute priority should be expanded to include a focus on improving instructional effectiveness exclusive of increasing the number or percentages of highly effective teachers or principals (or reducing the number or percentages of ineffective teachers or principals). Therefore, we decline to change the priority in the manner recommended by the commenters.

Changes: None.

Comment: We received a number of comments recommending that we revise this absolute priority to focus on teacher preparation and professional development programs. One commenter recommended that the Department revise the priority to include support for efforts by States to expand teacher preparation programs that produce effective teachers and to provide financial incentives such as loan forgiveness to recruit and retain effective teachers. Several commenters recommended that the priority support teacher residency programs, instructional coaching, and “communities of practice” for planning and sharing resources, practices, and expertise with other educators. One commenter recommended including a focus on initiatives that support teachers’ efforts to help students make connections between academic work and college and career goals. Another commenter recommended supporting projects to train school leaders on evaluating teacher effectiveness.

Discussion: The purpose of the Investing in Innovation Fund is not to support States to expand teacher preparation programs or to support specific types of teacher or principal training (e.g., teacher residency programs, instructional coaching). Rather, the purpose is to support projects at the local level that propose to expand the implementation of, and investment in, innovative practices, strategies, or programs that increase the number or percentages of highly effective teachers or principals (or reduce the number or percentages of ineffective teachers or principals) and that will have an impact on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment or completion rates for high-need students. We believe the absolute priority reflects this purpose and, therefore, decline to change the priority in the manner recommended by the commenters.

Changes: None.

Comment: One commenter recommended that the Department revise the priority to include support for research on teacher effectiveness and for disseminating the results of that research to LEAs and schools.

Discussion: One of the purposes of the Investing in Innovation Fund is to identify and document best practices that can be shared and taken to scale based on demonstrated success. Research unrelated to this purpose would not be supported under this priority.

We note that under this priority, projects that increase the number or percentages of highly effective teachers or principals (or reduce the number or percentages of ineffective teachers or principals) will be evaluated based on the strength of the existing research evidence and the significance of effect in support of the proposed project, as well as the magnitude of the effect on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates (see Selection Criterion B). In addition, proposed Scale-up and Validation projects will be evaluated based on the quality of their evaluation plan and the extent to which methods of evaluation include a well-designed experimental or quasi-experimental study (see Selection Criterion D). With regard to the recommendation that the priority include support for disseminating the results of research findings, we note that eligible applicants must conduct an independent evaluation of their project and make broadly available the results of such evaluations (see Evaluation requirement).

Changes: None.

Highly Effective School Leader

Comment: A few commenters recommended that the Department clarify the types of individuals who would be considered a school leader under the definition of the term highly effective school leader used in Absolute Priority 1. Four commenters recommended that the term “school leader” include parents and students in addition to principals. One commenter recommended that the term include professional staff, such as media and information specialists, instructional coaches, school counselors, school psychologists, school social workers, and others who may not be directly involved in classroom instruction but nonetheless are crucial to student academic success.

Another commenter expressed concern that the terms highly effective school leader and highly effective teacher imply that the two categories are mutually exclusive. The commenter recommended revising the definitions to clarify that these two terms are not mutually exclusive.

Discussion: As discussed earlier, the Department appreciates the important role that individuals other than principals play in providing leadership in our Nation’s schools. However, for purposes of this program, we intend to focus on the effectiveness of principals, in particular, because they have the ultimate responsibility for the academic achievement of the students in their schools. For this reason and to ensure consistency in the use of terms across programs supported with ARRA funds, we are changing the defined term highly effective school leader to highly effective principal and removing references to the term “school leader” from the definition.

With this change, the terms highly effective principal and highly effective teacher are mutually exclusive and we intend them to be so.

Changes: We are changing the defined term highly effective school leader to highly effective principal and removing the references to the term “school leader” from the definition of this term.

Comment: We received a number of comments on the proposed definition of the term highly effective school leader with respect to the measures used to determine whether a school leader is effective. Several commenters expressed concern that the proposed definition of highly effective school leader requires that, to be considered highly effective, a school leader must demonstrate that his or her students have achieved high rates of student growth (e.g., more than one grade level in an academic year). Two commenters expressed concern that this proposed definition appears to be based solely on the ability to demonstrate high annual rates of student growth and is thus too narrow and restrictive to properly identify effective school leaders. These commenters recommended that student growth should not be the sole criterion for determining school leader effectiveness, and that the definition of highly effective school leader should factor in other aspects of the teaching and learning environment, including broader measures such as the use of instructional methodologies and adaptive technologies.

One commenter expressed appreciation that the proposed definition permits the use of additional measures of school leader effectiveness, but was concerned that the definition fails to require the use of other measures of effectiveness not based on student assessments. This commenter asserted that there are limitations in measuring school leader effectiveness using current student assessment instruments and recommended that the Department revise the definition to include school leaders who have demonstrated superior ability to improve student learning (including but not limited to student growth based on assessment results) and who have excelled at all other essential aspects of their profession. Another commenter recommended that several additional measures be included in the definition of highly effective school leader, including measures related to leadership, vision, management, learners and learning, instruction, ethics, equity, and advocacy. Another commenter recommended changing the measures of effectiveness in the definition to include high rates of student growth, evidence of teacher improvement in knowledge and practice, and the use of research-supported ongoing long-term professional development; the commenter argued that this change is needed to ensure that the definition does not benefit wealthier LEAs to the detriment of poorer LEAs, which often have more difficulty in showing student growth.

Discussion: The Department believes that student growth must be a significant factor in identifying highly effective principals. (As noted in the previous discussion, the Department is changing the defined term highly effective school leader to highly effective principal.) We agree with the commenters that data on student growth should not be used as the sole means of identifying highly effective principals and that eligible applicants should supplement student growth data with other effectiveness measures. While we cannot include in the definition of highly effective principals all of the measures recommended by the commenters, we believe it is important to include several examples for illustrative purposes and are adding as examples the following measures: high school graduation rates and college enrollment rates, evidence of providing supportive teaching and learning conditions, support for ensuring effective instruction across subject areas for a well-rounded education, strong instructional leadership, and positive family and community engagement; or evidence of attracting, developing, and retaining high numbers of effective teachers. However, we do not believe it is necessary to require the use of supplemental measures in identifying highly effective principals in projects funded in this program.

We note that the definition of highly effective principal in this program is similar to the definition of this term in the Department’s Race to the Top Fund program. However, because in this program the definition does not require the use of multiple measures to identify highly effective principals, the definitions are not identical. We believe that the difference between these definitions is warranted because the eligible applicants for these programs differ. Given the diverse pool of eligible applicants and the variety of projects that may be supported under this program, we believe that an eligible applicant should have the flexibility necessary to present a model for identifying highly effective principals that is appropriate for its proposed project and not be required to use multiple measures that may not be related to its project. Although eligible applicants may use multiple measures and we encourage them to do so if appropriate for their proposed projects, under this program an eligible applicant is only required to use student growth data.

Changes: As noted in the previous discussion, we are changing the defined term highly effective school leader to highly effective principal. We are revising the definition to read as follows: Highly effective principal means a principal whose students, overall and for each subgroup as described in section 1111(b)(3)(C)(xiii) of the ESEA (i.e., economically disadvantaged students, students from major racial and ethnic groups, migrant students, students with disabilities, students with limited English proficiency, student gender), achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth. Eligible applicants may include multiple measures, provided that principal effectiveness is evaluated, in significant part, by student growth. Supplemental measures may include, for example, high school graduation rates; college enrollment rates; evidence of providing supportive teaching and learning conditions, support for ensuring effective instruction across subject areas for a well-rounded education, strong instructional leadership, and positive family and community engagement; or evidence of attracting, developing, and retaining high numbers of effective teachers.

Comment: One commenter expressed concern that the definition of highly effective school leader applies only to leaders of elementary schools and may be problematic for the secondary school level. One commenter recommended that the Department allow eligible applicants to use an increase in graduation rates as a measure of student growth for high schools in tandem with student growth on required State assessments. This commenter also recommended that the Department require eligible applicants to propose how they would measure student growth for untested grades and subjects, particularly in high schools.

Discussion: As noted in the previous discussion, we are revising the definition of highly effective principal (proposed as highly effective school leader) to clarify that eligible applicants may include multiple measures, provided that principal effectiveness is evaluated, in significant part, by student growth. Specifically, we believe the addition of high school graduation rates and college enrollment rates as examples of supplemental measures makes clear that this definition covers principals in high schools.

Regarding the recommendation that eligible applicants be required to propose how they would measure student growth for untested grades and subjects, we believe that eligible applicants should have the flexibility to determine the measure(s) of student achievement (on which determinations of student growth are based, consistent with the definition of student growth used in this program) that are most appropriate for their proposed projects. We do not believe it is necessary to require eligible applicants to propose the measures they would use for untested grades and subjects for review and approval by the Department.

Changes: In the list of examples of supplemental measures for determining principal effectiveness that we are adding to the definition of highly effective principal, we are including high school graduation rates and college enrollment rates.

Comment: One commenter recommended that the Department clarify whether, to meet the definition of highly effective school leader, each of the school leader’s students must individually demonstrate a high rate of student growth.

Discussion: The definition of highly effective principal (proposed as highly effective school leader) requires that, to be considered highly effective, the principal’s students must demonstrate high rates of student growth overall and for each subgroup described in section 1111(b)(3)(c)(xiii) of the ESEA. Thus, under this definition, effectiveness is determined (in significant part) using aggregate rates of student growth. There is no requirement that each student in the principal’s school demonstrate a high rate of student growth individually.

Changes: None.

Comment: One commenter recommended that the Department change the definition of highly effective school leader so that a school leader is considered to be highly effective if his or her students achieve high rates of student growth overall and for one or more of the subgroups described in section 1111(b)(3)(C)(xiii) of the ESEA (i.e., economically disadvantaged students, students from major racial and ethnic groups, migrant students, students with disabilities, students with limited English proficiency, and student gender), rather than for each of these subgroups. The commenter argued that this change is needed to ensure that the definition does not favor principals in schools in wealthier LEAs to the detriment of those in poorer LEAs, which typically have higher concentrations of students in these subgroups and often have more difficulty in showing student growth.

Discussion: We believe that in order for a principal to be considered highly effective, that principal’s students should achieve high rates of student growth for each student subgroup represented in the school. As this program is designed to support, in general, projects that improve student academic achievement and attainment and, under Absolute Priority 1 in particular, projects that increase the number or percentage of highly effective principals or reduce the number or percentage of ineffective principals, we believe that projects supported under this program will help address the issue raised by the commenter regarding student performance in poorer LEAs.

Changes: None.

Highly Effective Teacher

Comment: Two commenters suggested that, in Absolute Priority 1, the Department change the defined term highly effective teacher to “highly qualified teacher.”

Discussion: The term “highly qualified teacher” has a specific meaning under the ESEA and is focused primarily on the qualifications of teachers. In this program (as in other programs supported with ARRA funds), we intend to focus instead on outcomes of teaching and the impact of teachers on the academic achievement and growth of their students. The definition of highly effective teacher is consistent with that focus and, for that reason, we do not believe the change recommended by the commenter is warranted.

Changes: None.

Comment: We received a number of comments on the proposed definition of the term highly effective teacher with respect to the measures used to determine whether a teacher is highly effective. A number of commenters expressed concern about using student growth as the measure to determine whether a teacher is highly effective under this definition. Specifically, several commenters expressed concern about the definition’s reliance on student assessment results and recommended that growth in student achievement on assessments be only one factor in determining whether a teacher is highly effective. Other commenters recommended that the provision on students achieving high rates of growth be removed from the definition because it places too much emphasis on State assessments. These commenters recommended revising the definition to encourage or require eligible applicants to use multiple effectiveness measures. The measures mentioned by commenters include the following: student-based measures such as local assessments, classroom assessments, portfolio assessments, progress monitoring, and nonacademic forms of evaluation (such as evaluations of student engagement); and teacher-based measures such as assessments of teacher subject knowledge and skills (including standards-based teacher evaluations), assessments of teaching practice and performance (including assessments of teacher planning and preparation), assessments of teacher reflectiveness, participation in learning communities, and training in helping students make connections between their performance in school and their goals for college and careers.

Discussion: The Department believes that student growth must be a significant factor in identifying highly effective teachers. As noted in our discussion of commenters’ concerns that student growth data should not be used as the sole means to identify highly effective principals, we agree with the commenters that data on student growth should not be used as the sole means of identifying highly effective teachers and that eligible applicants should supplement student growth data with other effectiveness measures. While we cannot include in the definition of highly effective teacher all of the measures recommended by the commenters, we believe it is important to include several examples for illustrative purposes and are adding as examples the following measures: multiple observation-based assessments of teacher performance or evidence of leadership roles (which may include mentoring or leading professional learning communities) that increase the effectiveness of other teachers in the school or LEA. However, we do not believe it is necessary to require the use of supplemental measures in identifying highly effective teachers in projects funded under this program.

We note that the definition of highly effective teacher in this program is similar to the definition of this term in the Department’s Race to the Top Fund program. However, because in this program the definition does not require the use of multiple measures to identify highly effective teachers, the definitions are not identical. We believe that the difference between these definitions is warranted because the eligible applicants for these programs differ. Given the diverse pool of eligible applicants and the variety of projects that may be supported under this program, we believe that an eligible applicant should have the flexibility necessary to present a model for identifying highly effective teachers that is appropriate for its proposed project and not be required to use multiple measures that may not be related to its project. Although eligible applicants may use multiple measures and we encourage them to do so if appropriate for their proposed projects, under this program an eligible applicant is only required to use student growth data.

Changes: We are revising the definition of highly effective teacher to read as follows: Highly effective teacher means a teacher whose students achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth. Eligible applicants may include multiple measures, provided that teacher effectiveness is evaluated, in significant part, by student growth. Supplemental measures may include, for example, multiple observation-based assessments of teacher performance or evidence of leadership roles (which may include mentoring or leading professional learning communities) that increase the effectiveness of other teachers in the school or LEA.

Comment: Two commenters expressed concern that the measures used to identify highly effective teachers may be problematic for teachers at the secondary school level. The commenters recommended that the Department require eligible applicants to propose how they would measure student growth for untested grades and subjects, particularly in high schools.

Discussion: As noted in the previous discussion, we are revising the definition of highly effective teacher to clarify that eligible applicants may include multiple measures for determining teacher effectiveness, provided that teacher effectiveness is evaluated, in significant part, by student growth. Under this definition, an eligible applicant would be free to use supplemental measures that it determines to be appropriate for assessing effectiveness of teachers at the secondary school level. These supplemental measures may include measures such as high school graduation rates or college enrollment rates.

As noted in our discussion of the commenter’s recommendation that the Department require eligible applicants to propose how they would measure student growth for untested grades and subjects with respect to the definition of highly effective principal, we believe that eligible applicants should have the flexibility to determine the measure(s) of student achievement (on which determinations of student growth are based, consistent with the definition of student growth used in this program) that are most appropriate for their proposed projects. We do not believe it is necessary to require eligible applicants to propose the measures they would use for untested grades and subjects for review and approval by the Department.

Changes: None.

Comment: One commenter asked the Department to clarify whether, to meet the definition of highly effective teacher, each of a teacher’s students must individually demonstrate a high rate of student growth.

Discussion: To meet the definition of highly effective teacher, a teacher’s students must achieve a high rate of student growth in the aggregate; a teacher’s students need not achieve high rates of growth individually.

Changes: None.

Comment: One commenter expressed concern that teachers of students with disabilities will face disproportionate difficulty in meeting the definition of highly effective teacher because students with disabilities are less likely to achieve high rates (e.g., more than one grade level in an academic year) of student growth.

Discussion: We appreciate the commenter’s concern. We believe that evaluation systems should support the equitable evaluation of teachers who are providing instruction to students with disabilities in regular education settings consistent with the requirements of the Individuals with Disabilities Education Act (IDEA) to educate students with disabilities in the least restrictive environment. However, while the definition of highly effective teacher provides an example of a high rate of student growth (e.g., one and one-half grade levels in an academic year), the definition does not specify the rate of student growth that eligible applicants must use. Further, the definition does not require that the same rate of growth must be used for all types of teachers. Thus, an eligible applicant would not be prohibited from using a rate of student growth that differs from the example provided and may determine that different rates of student growth are appropriate for teachers of different types of students included in its proposed project. However, we urge eligible applicants to ensure that any rate used enables the eligible applicant to distinguish teachers who are highly effective from those who are not.

Changes: None.

Comment: Three commenters sought clarification of the term “teacher” and requested that the Department add a definition of this term. In particular, two commenters sought clarification on whether “teacher” referred only to teachers in tested grades and subjects or to any teacher who meets the definition of “teacher” used in the State.

Discussion: We do not believe that a definition of “teacher” is necessary under this program. In determining which teachers meet the definition of highly effective teacher an eligible applicant may consider any educational personnel that meet the definition of “teacher” used in a State in which the project is being implemented, provided that data on student growth are available for those personnel. The term highly effective teacher is not restricted to teachers in the tested grades and subjects.

Changes: None.

Absolute Priority 2--Innovations That Improve the Use of Data

Comment: Several commenters expressed support for this proposed absolute priority. One commenter expressed appreciation for the priority’s support for local use of data, as opposed to an exclusive focus on the development and use of data systems at the State level. One commenter, however, expressed concern that the priority did not reflect the ARRA assurance in this reform area. The commenter asserted that the ARRA assurance pertaining to data relates to the development and implementation of statewide longitudinal data systems and not the use of data to inform local decision making as described in the priority.

Discussion: As noted earlier, we have designed the absolute priorities for this program to be consistent with the four education reform areas under the ARRA. Given that data from statewide longitudinal data systems could be used to inform decisions at the LEA and school levels, we believe that the proposed priority’s support for improvements in the local use of data is reasonable and consistent with the education reform area in the ARRA.

Changes: None.

Comment: One commenter suggested that the Department clarify whether the data to be used under this priority are data from a statewide longitudinal data system or data that is separately maintained at the local level.

Discussion: We do not intend to limit the source of data only to a statewide longitudinal data system or to local data systems. An eligible applicant may propose projects under this priority that utilize data from either or both of these sources, or any other available data sources.

Changes: None.

Comment: One commenter objected to the Department’s inclusion, under this priority, of estimates of individual teacher impact on student achievement as an example of the kinds of data on student achievement or growth that can drive education reform. The commenter cited research pertaining to limitations and difficulties in producing teacher “value-added” estimates. The commenter also asserted that estimates of individual teacher impact on student achievement are not sufficiently stable to determine teacher effectiveness and should not be used in decisions to recruit, retain or remove teachers.

Discussion: We recognize that currently available data-driven methods of evaluating teacher or principal impact on student achievement and student growth data may need further study or development. However, student achievement or growth data is one of many measures that can drive education reform in general, and facilitate improvement in the classroom, in particular. For this reason, we believe that student data can drive instructional improvement decisions at both the individual teacher level and the district level. That is why we have included innovations under this priority that encourage projects that increase the availability of data for teachers, principals, families, and other stakeholders, and projects that develop strategies to use data effectively to improve school and classroom instructional practices. With respect to the commenter’s concern about student achievement data being an “unstable” measure to evaluating teacher and principal effectiveness, we note that as previously discussed under absolute priority 1, we believe student achievement or growth data should be used as a significant factor, but need not serve as a single measure of effectiveness. Further, we believe this measure should be a component of teacher or principal evaluation systems that are rigorous, transparent, and fair; differentiate performance using multiple rating categories of effectiveness and multiple measures of effectiveness; and are designed and developed with teacher and principal involvement. For these reasons, we have concluded that the changes suggested by the commenters should not be adopted.

Changes: None.

Comment: One commenter recommended that the Department revise the priority to state that, where applicable, data should be disaggregated for Native American students.

Discussion: The priority requires disaggregation of data, where applicable, to be consistent with section 1111(b)(3)(C)(xiii) of the ESEA. Section 1111(b)(3)(C)(xiii) requires the disaggregation of data by major racial and ethnic groups which may include, among others, American Indians, Alaska Natives, and Native Hawaiians. Under this priority, an eligible applicant proposing a project would be expected to disaggregate data for these groups of students, where applicable.

Changes: None.

Comment: Several commenters expressed concern about protecting the privacy of students whose data are used under this priority. Two of these commenters noted that the requirements of the Family Educational Rights and Privacy Act (FERPA) are not discussed in the NPP and recommended that the Department provide guidance on how grant recipients can implement projects under this priority in a manner consistent with FERPA requirements. These commenters also expressed concerns about protecting the privacy of teachers and school leaders.

Discussion: Eligible applicants must consider how to protect student privacy as data are shared. Educational agencies and institutions, including LEAs, schools, and IHEs, that receive awards under this program or any other Department of Education program, must comply with FERPA, 20 U.S.C. 1232g, and its implementing regulations in 34 CFR Part 99, as well as any applicable State and local requirements. 34 CFR 99.31 specifies the conditions under which an educational agency or institution may non-consensually disclose personally identifiable information from an education record of a student to a third party (i.e., a nonprofit organization in partnership with an educational institution).  Consistent with 34 CFR 99.33, FERPA also applies to the non-consensual redisclosure of personally identifiable information from an education record by a third party. Because compliance with FERPA is a requirement that must be met by all educational agencies and institutions that are recipients of Department funds, we do not believe it is necessary to amend the priority as suggested by the commenter. In response to the commenter’s concern about ensuring teacher and school leader privacy, the Department agrees that teacher and principal privacy also must be protected. However, teacher and principal privacy is governed by State law. Eligible applicants that receive awards under this program must comply with any applicable State and local privacy requirements.

Changes: None.

Comment: Several commenters recommended that the Department revise the priority to support projects pertaining to specific uses of data. For instance, some commenters suggested that we revise the priority to support projects that focus on professional development, training, or other technical and expert assistance for teachers and school leaders on the analysis and use of data, as well as on the communication of data to parents and the community. Another commenter recommended that we revise the priority to include a focus on projects that use real-time data and related rapid response supports for teachers (with respect to professional development) and for students (with respect to academic content). One commenter recommended that we revise the priority to include a focus on the development of data-driven instructional improvement systems. One commenter recommended that we revise the priority to include support for the collection of data in addition to the aggregation, analysis, and use of data. Two commenters recommended that we revise the priority to include support for projects that align local data systems with other data systems, including statewide longitudinal data systems, and ensure interoperability between these systems. Similarly, another commenter recommended that we revise the priority to include a focus on projects that link local data systems with data systems of other agencies and institutions such as workforce agencies and institutions of higher education. One commenter recommended that the Department revise the priority to include a focus on projects that disaggregate data through cross-referencing of multiple subgroups and demographic categories, rather than disaggregating the data only by the discrete subgroups listed in the priority. One commenter suggested that we revise the priority to include support for projects that use student achievement or student growth data to identify and support students who are “off track” – presumably in reference to students that would qualify as “high need students” (as defined in this notice). Similarly, one commenter recommended that we revise the priority to include a focus on projects that use data specifically to inform student dropout prevention and recovery programs. One commenter recommended that we revise the priority to include a focus on projects that use student achievement or student growth data to improve the performance of persistently low-performing schools. One commenter recommended that the Department revise the priority to include a focus on projects that link the achievement or growth data for students of individual teachers to those teachers’ preparation programs so that the data can be used to improve those programs and ensure that they produce effective teachers. Two commenters suggested that we revise the priority to support projects that include plans for communicating the results of data analyses effectively in the community.

Discussion: There is nothing in this priority that precludes an eligible applicant from proposing any of these projects under this priority, provided that the proposed project (1) encourages and facilitates the evaluation, analysis and use of student achievement or student growth data by educators, families or other stakeholders to inform decision-making, (2) improves student achievement or student growth, or teacher, principal, school, or LEA performance and productivity, or (3) enables data aggregation, analysis, and research, as specified in the priority. We made this priority broad to provide eligible applicants with flexibility to propose a variety of projects. We believe we have achieved this goal, as evidenced by the array of projects proposed by the commenters. For this reason, we conclude that it is not necessary to revise the priority to include an express focus on such activities.

Changes: None.

Comment: One commenter recommended that the Department revise the priority to support projects that use data to improve student attendance or behavior in addition to student achievement or growth. Another commenter recommended supporting projects that use data to improve school culture or climate. Another commenter recommended that the Department revise the priority to include support for projects that use data to improve families’ ability to support student achievement at home.

Discussion: As noted elsewhere in this notice, the Department believes that, consistent with the ARRA, we must preserve improving student academic achievement and attainment as the primary goals of this program. Accordingly, we do not believe it is appropriate to revise this priority to include reference to improvements with respect to other outcome measures. We note, however, that in discussing the effects of a project proposed under this priority an eligible applicant may include discussion of the effects of the project on intermediate variables that are strongly correlated with improving student achievement and attainment outcomes. These intermediate variables could include variables related to the topics suggested by the commenters.

Changes: None.

Comment: One commenter recommended that the Department revise the priority to include community members among the list of stakeholders receiving and using student achievement and growth data.

Discussion: We have intentionally not provided a definition of the term “other stakeholders” to provide eligible applicants with flexibility to determine which stakeholders should be targeted under this priority. Accordingly, it is at the eligible applicant’s discretion to determine what other stakeholders should have a role in their proposed projects. Further, we believe that community members are reasonably included amongst the other stakeholders to whom projects would provide data under this priority. Therefore, we decline to make the changes requested by the commenters.

Changes: None.

Comment: One commenter recommended that States be required to create an “Opportunity to Learn Index,” to track data about the quality of State and local education systems.

Discussion: The commenter appears to misunderstand the purpose of the program, which is to expand the implementation of, and investment in, innovative practices that are demonstrated to have an impact on improving student achievement or student growth for high-need students. Because State educational agencies cannot apply for funding under this program, it would not be appropriate to establish such requirements for States.

Changes: None.

Absolute Priority 3--Innovations That Complement the Implementation of High Standards and High Quality Assessments

Comment: Several commenters expressed support for focusing this priority on high standards. One of these commenters expressed support for implementing common high standards across LEAs and States. One commenter expressed support for the priority with respect to the promotion of contextual learning opportunities. One commenter recommended that we specify which entity should be responsible for implementing initiatives that are responsive to the priority, because, according to the commenter, the priority appears to refer to State activities rather than matters for eligible applicants. Similarly, two commenters implied that States are the only entities that could be assisted under this priority. One commenter requested that the Department clarify whether the priority requires an LEA to work with its State to improve the State’s systems of standards and assessments or develop and implement new systems. Another commenter requested that the Department clarify how the initiatives included under this priority will support States’ efforts to transition to college- and career-ready standards and assessments; the commenter asserted that the initiatives do not seem related to the adoption of college- and career-ready standards and assessments.

Discussion: We appreciate the commenters’ support for the proposed priority. This priority is designed to support local efforts that complement States’ development and implementation of college- and career-ready standards and high-quality assessments aligned with those standards. This priority is not intended to support States’ efforts in this area directly or to require LEAs or other entities to provide direct assistance to States in the development and implementation of standards and assessments. Instead, this priority encourages projects at the LEA level that support and complement States’ transition to college and career ready standards and assessments, such as LEA activities of developing, acquiring, disseminating and implementing high-quality curricular instructional materials and assessments, or delivering high-quality professional development pertaining to such standards or assessments. We believe this priority in the context of this program is sufficiently clear.

Changes: None.

Comment: Another commenter requested that the Department clarify whether this priority requires LEAs to propose projects that are based on the college- and career-ready standards and assessments to which States are transitioning. The commenter also asserted that the priority appears to give an undue advantage to LEAs in States that have made more progress than other States in making this transition.

Discussion: Under this priority, an eligible applicant must propose projects that support States’ efforts to transition to standards and assessments that measure students’ progress toward college- and career-readiness. We recognize that States’ progress in developing and transitioning to standards that measure college- and career-readiness varies. However, this variable will not impact the competitiveness of an eligible applicant’s proposed project. Under this priority, eligible applicants may propose projects that are based on standards other than those of their home State, so long as the standards they select are aligned with and at least as rigorous as their home State’s standards. For this reason, LEAs in States that have made less progress toward standards that measure college- and career-readiness are not disadvantaged by this priority. We note that eligible applicants who propose projects under this priority that are not based on the applicant’s State standards must explain how their proposed standards are aligned with, and are at least as rigorous as, their home State’s standards, as well as how these standards differ.

Changes: We are revising the priority to clarify that an eligible applicant must propose a project that is based on standards that are at least as rigorous as its State’s standards. Further, we are revising the priority to clarify that if the proposed project is based on standards other than those adopted by the eligible applicant’s State, the applicant must explain how the standards are aligned with and at least as rigorous as the eligible applicant’s State’s standards as well as how the standards differ.

Comment: One commenter recommended that the Department revise this priority to support initiatives that increase students’ college and career readiness.

Discussion: We believe that the priority’s support for initiatives that complement States’ implementation of college- and career-ready standards and assessments aligned with those standards supports initiatives that increase students’ college- and career-readiness. For this reason, we do not believe revisions to this priority are necessary.

Changes: None.

Comment: One commenter expressed support for the priority with respect to promoting the use of formative and interim assessments. Another commenter recommended that the Department restrict the assessments under this priority to formative assessments; the commenter asserted that interim assessments typically are repetitions of larger-scale summative assessments and do not provide useful diagnostic information to educators or students.

Discussion: We made this priority broad to provide eligible applicants with flexibility to propose a variety of projects; we do not wish to constrain innovation by prohibiting specific activities under this priority such as utilizing interim assessments. We believe eligible applicants are in the best position to determine whether interim assessments are an appropriate tool under a proposed project. For this reason, we decline to amend the priority as suggested by the commenter.

Changes: None.

Comment: Several commenters recommended that the Department revise the priority to support projects pertaining to specific standards- and assessments-based activities. For instance, some commenters suggested that we revise the priority to specify the types of activities that would translate standards and information from assessments into classroom practices.

Another commenter recommended that we revise the priority to further emphasize initiatives that improve student engagement through real-world applications of learning to fully prepare students to compete and succeed in a global economy. One commenter suggested that we revise the priority to include initiatives that provide professional development to teachers regarding the use of results from formative assessments supported under the priority. Two other commenters recommended that we revise the priority to include initiatives that promote family understanding, and engagement in the implementation and monitoring, of education standards in order to ensure that such standards are of high quality. A few commenters recommended that the Department revise the priority to ensure that the initiatives pursued under this priority are consistent with the principles of universal design for learning (we presume this to be a reference to the principles of universal design for learning as that term is defined in section 103(24) of the Higher Education Act of 1965, as amended (HEA)). One commenter recommended that the Department revise the priority to encourage increased access to and use of open-content and web-based curricular materials. One commenter recommended that the Department include, among the curricular and instructional initiatives supported under this priority, initiatives regarding non-traditional instruction and relationship building in order to reengage disconnected students.

Discussion: There is nothing in this priority that would preclude an eligible applicant from proposing any of the projects recommended by commenters, provided that the proposed project meets the requirements specified in the priority. We made this priority broad to provide eligible applicants with flexibility to propose a variety of projects. We believe we have achieved this goal, as evidenced by the array of projects proposed by the commenters. For this reason, we conclude that it is not necessary to revise the priority to include an express focus on such activities.

Changes: None.

Comment: Two commenters recommended that we revise the priority to include support for early learning programs.

Discussion: Although, to meet this priority, an eligible applicant must propose a project that is designed to benefit students in elementary and secondary schools (by implementing activities that support States’ efforts to transition to college- and career-ready standards and assessments), an eligible applicant would not be prohibited from proposing a project that additionally serves students in early learning programs. Indeed, this notice specifically contains competitive preference priority 5 pertaining to Innovations for Improving Early Learning Outcomes. For these reasons, we do not believe it is appropriate to revise the priority as the commenters suggest.

Changes: None.

Comment: Two commenters expressed support for the priority’s focus on academically rigorous courses and programs; another commenter recommended that the Department maintain the list of academically rigorous courses and programs in the priority. Another commenter expressed support for including STEM courses in the priority. Two commenters, however, recommended that the Department provide an example, other than STEM subjects, of the core academic subjects for which curricular and instructional initiatives could be pursued under this priority. Another commenter recommended that we revise the priority to allow applicants to pursue activities in subjects that may not be included in common core standards initiatives, such as computer science; this commenter also recommended that the Department include references to computer science courses along with courses in STEM subjects.

Discussion: We appreciate the commenters’ support, but do not believe it is necessary to include the commenters’ recommended revisions in the priority; however, we are revising the priority to provide further clarity pertaining to the definition of “core academic subjects.” This priority is designed to support initiatives in any or all core academic subjects, consistent with section 9101(11) of the ESEA, including English, reading or language arts, mathematics, science, foreign language, civics and government, economics, arts, history, and geography. Consistent with the Race to the Top Fund program, the Department interprets the core academic subject of “science” under section 9101(11) to include STEM education (science, technology, engineering and mathematics) which encompasses a wide-range of disciplines, including computer science.

Changes: To clarify that “core academic subjects” refers to those under section 9101(11) of the ESEA, we are changing the priority to include the statutory reference. We are also including a footnote regarding the Department’s interpretation with respect to “science” under section 9101(11) of the ESEA.

Comment: Several commenters suggested that we revise the priority to support specific curricular and instructional initiatives. For instance, one commenter recommended that we revise the priority to support initiatives only in literacy and problem solving skills, arguing that these two areas are key to improving student achievement. A few commenters recommended that we revise the priority to specifically support initiatives in career and technical education. Another commenter recommended that we revise the priority to include initiatives that provide experiences in diversity in the classroom and school that prepare students for racially and economically diverse college and work settings. Two commenters recommended that we revise the priority to include initiatives that support student achievement at home and in other learning settings in order to promote family and community engagement in education. One commenter recommended that the priority be revised to include initiatives that use technology in ways that encourage self-directed learning.

Discussion: An eligible applicant would not be precluded from proposing under this priority a project that focuses on the subjects and areas recommended by the commenter so long as the project supports States’ efforts to transition to college- and career-ready standards and assessments, as specified in the priority. We do not believe it is appropriate or consistent with the purpose of this program to revise the priority to limit or narrow the priority to these specified initiatives.

Changes: None.

Absolute Priority 4--Innovations That Turn Around Persistently Low-Performing Schools

Comment: A number of commenters expressed general support for this absolute priority. However, several commenters recommended that the Department clarify the initiatives the priority would support. One of these commenters requested clarification as to whether projects under this priority may serve certain groups of students within schools rather than engage in whole-school reform.

Discussion: The Department appreciates the commenters’ support for this proposed absolute priority. Under the priority, an eligible applicant may propose a project that serves only certain groups of students (provided those students meet the definition of high-need student used in this program) as a targeted approach to reform.

Changes: None.

Comment: One commenter requested that we define the term “comprehensive intervention” as used with respect to whole-school reform supported under this priority.

Discussion: We agree with the commenter that further specificity regarding the comprehensive intervention approaches to whole-school reform under this priority is warranted and are revising the priority to include additional examples of those approaches. In addition to providing further specificity, the revisions we are making are intended to ensure that projects supported under this priority can be consistent with efforts to reform low-performing schools under other programs supported with ARRA funds.

As discussed later in this section, we are removing the definition of persistently low-performing schools and revising the priority to specify the schools for which the priority supports reform projects. Consistent with those changes, we refer to these schools as Investing in Innovation Fund Absolute Priority 4 Schools in the discussion of comments that follows.

Changes: We are revising paragraph (a) of this priority as follows: (a) Whole-school reform, including, but not limited to, comprehensive interventions to assist, augment, or replace Investing in Innovation Fund Absolute Priority 4 schools, including the school turnaround, restart, closure, and transformation models of intervention supported under the Department’s School Improvement Grants program.

Comment: Several commenters expressed support for the priority’s encouragement of expanded learning time as a targeted approach to reform (paragraph (b)(1) of the priority). However, a number of commenters recommended that the Department clarify whether out-of-school programs are included as targeted approaches under paragraph (b)(1). The commenters also recommended that out-of-school programs be required to include collaboration with community-based partners, institutions of higher education, and museums, and that these programs include project-based learning.

Discussion: To the extent that an “out-of-school” program includes programs to provide extended learning time either after school, over the weekend, or during the summer, these activities would be permissible under this priority as targeted approaches to reform, so long as the proposed project also meets the requirements specified in this priority. We made this priority broad to provide eligible applicants with flexibility to propose a variety of projects, and to collaborate with a wide range of entities that can support their specific projects, which could include those mentioned by the commenters. For these reasons, we conclude that it is not necessary to revise the priority to include an express focus on specific activities or entities.

Changes: None.

Comment: One commenter requested that the Department provide a definition of “core academic subjects” under this priority.

Discussion: As noted previously, for purposes of this program, we are using the definition of “core academic subject” as set forth in section 9101(11) of the ESEA, and are including a reference to the statutory definition in paragraph (b) of the priority.

Changes: We are revising the priority to reference section 9101(11) of the ESEA.

Comment: Two commenters requested that the Department clarify the non-academic barriers to student achievement that an applicant may propose to address under a targeted approach to reform under this priority.

Discussion: Although we do not intend to unduly restrict the projects this priority would support by identifying specific barriers in the priority, we note that such barriers may relate to issues such as the following: truancy, unsafe school environment, poor school climate, lack of student engagement, and lack of parent and community involvement.

Changes: None.

Comment: A few commenters recommended that the Department clarify the term “transfer school” that is used in the priority as an example of a pathway for students to earn a regular high school diploma.

Discussion: We agree with the commenters that the term “transfer schools” may not be commonly understood. Therefore, we are replacing the term “transfer schools” in the priority with “schools that serve the needs of over-aged, under-credited, or other students with an exceptional need for flexibility pertaining to when they attend school and what additional supports they require.”

Changes: We are revising paragraph (b)(3) in this priority as follows: (3) creating multiple pathways for students to earn regular high school diplomas (e.g., using schools that serve the needs of over-aged, under-credited, or other students with an exceptional need for flexibility pertaining to when they attend school and what additional supports they require; awarding credit based on demonstrated evidence of student competency; and offering dual enrollment options).

Comment: One commenter recommended that the Department revise the priority to ensure consistency with the priorities and requirements for turning around persistently low-performing schools under the Department’s Race to the Top Fund and School Improvement Grants programs.

Discussion: As discussed previously, we are revising the priority to include, as examples of whole-school reform, school turnaround, restart, closure, and transformation models of intervention supported under the Department’s School Improvement Grants program. We believe this will help ensure that projects supported under this priority are consistent with efforts to reform low-performing schools under other programs supported with ARRA funds.

Changes: As discussed previously, we are revising paragraph (a) of this priority as follows: (a) Whole-school reform, including, but not limited to, comprehensive interventions to assist, augment, or replace Investing in Innovation Fund Absolute Priority 4 schools, including the school turnaround, restart, closure, and transformation models of intervention supported under the Department’s School Improvement Grants program.

Comment: Many commenters recommended that the Department revise the priority to provide greater flexibility in the initiatives the priority would support. Several of these commenters cautioned, in particular, that the priority places an excessive focus on extended learning time and, without increased flexibility, may undercut the competency-based programs supported under the Department’s Race to the Top Fund program. Another commenter requested that the Department revise the priority to allow applicants to propose projects along a continuum of interventions ranging from targeted to comprehensive, rather than proposing projects using either whole-school or targeted approaches to reform. The commenter asserted that whole-school reform approaches typically involve multiple targeted interventions; thus, the commenter claimed, the distinction between these two approaches in the priority is artificial.

Discussion: We believe that maintaining a distinction between whole-school and targeted approaches to reform is useful to eligible applicants for the purposes of preparing applications to turn around Investing in Innovation Fund Absolute Priority 4 schools. We note that the priority provides a significant amount of flexibility and does not specify the types of activities that would fall under either reform approach. As such, we do not believe the priority undercuts priorities articulated in other Department programs.

Changes: None.

Comment: Several commenters recommended that the Department revise the priority to support projects pertaining to specific activities. For instance, many commenters encouraged the Department to revise the priority to include the creation and replication of high quality new schools, including charter and magnet schools, as an acceptable approach to reform. Another commenter suggested that the Department revise the priority to support projects that increase school choice options for parents and students. One commenter recommended that the Department revise the priority to include the development of “community schools,” in reference to schools that implement comprehensive, integrated strategies for providing academic instruction, offer student services and supports, and engage families and the community in the education of their children. Two commenters suggested that the Department revise the priority to include, in addition to initiatives that would expand learning time as a targeted approach to reform, initiatives for restructuring the current school day to make better use of existing in-class time. One of these commenters suggested that the priority support restructuring the current school day with a greater use of technology and other means of differentiated instruction. Several commenters recommended revising the priority to include support for new or alternative instructional practices in persistently low-performing schools. Several commenters recommended that the Department revise the priority to include initiatives that incorporate data-driven instruction and supports. Two other commenters recommended that the Department revise the priority to include support for alternative curricular approaches and instructional tools (e.g., curricular approaches that are based in research in cognitive science and neuroscience, curricular approaches that integrate the use of technological tools) as acceptable reform approaches. Another commenter suggested that the Department revise the priority to include initiatives that incorporate instructional improvement systems as an acceptable reform approach; this commenter referred to the inclusion of these systems in the Department’s Race to the Top Fund program. One commenter recommended that the Department revise the priority to include individual and small group instruction as a targeted approach to reform. Several commenters recommended that the Department revise the priority to include afterschool programs that provide older students with academic supports as an example of a targeted reform approach and, more specifically, as a graduation pathway for students. Two commenters recommended that the Department revise the priority to include as acceptable reform approaches initiatives that reduce racial and economic isolation such as reduction of resource gaps between schools and opportunities for intra- or inter-LEA transfers for students and educators. Several commenters recommended that the Department revise the priority to support initiatives that include strategies for improving teacher professional development and other support such as high-quality job-embedded professional development, common planning time, additional compensation, and peer involvement in staffing selections and resource allocation. One commenter requested that the Department revise the priority to include instruction in subjects beyond the core academic subjects in extended learning time initiatives implemented as targeted reform approaches. Two commenters suggested that the Department revise the priority to include, as a targeted approach to reform, strategies for increasing student engagement in order to address truancy, discipline, and social acceptance issues. A number of commenters recommended that the Department revise the priority to include building community and family links and increasing community and family engagement as acceptable school reform strategies, including ongoing parental involvement, wraparound services, increased parent-teacher interaction, and parent education programs regarding instructional programs and supports. Several commenters recommended that the Department revise the priority to include additional outcome measures, including measures regarding improvements in school climate, long-term student outcomes, and engagement in learning tied to real-world applications; and elimination of bullying and student harassment.

Discussion: There is nothing in this priority that precludes any of the projects recommended by the commenters, provided that the proposed project addresses the whole-school or targeted approaches to reform, as specified in this priority. This priority is intentionally broad to provide eligible applicants with flexibility to propose a variety of projects that best reflect the variety of resources applicants bring to bear and the students they intend to serve. For this reason, we conclude that it is not necessary to revise the priority to include a specific list of permissible activities.

Changes: None.

Comment: One commenter expressed concern about projects under this priority that would expand learning time by adding hours to the school day or extending the school year because these projects would be costly and constrained by teacher contracts; the commenter recommended that the Department focus on projects that would reform the existing school day using existing resources and that are not constrained by teacher contracts.

Discussion: We agree that applicants should be mindful of cost and contractual obligations as they develop their proposed projects. However, organizations and LEAs operate in a range of environments and therefore are best positioned to determine which approaches to extending learning time are most effective for their projects.

Changes: None.

Comment: A number of commenters recommended that the Department include early learning programs as an acceptable strategy for turning around low-performing schools under this priority in light of the impact of these programs on student achievement in later years. One of these commenters suggested that the priority include initiatives that integrate high quality pre-kindergarten programs with early language and literacy instruction in the elementary grades.

Discussion: We believe that any approach to reform under this priority (whether whole-school or targeted) must be designed expressly for the purpose of turning around Investing in Innovation Fund Absolute Priority 4 schools, which may only be public elementary and secondary schools. Accordingly, an initiative focused solely on improving early learning programs would not, by itself, meet this absolute priority. However, nothing would prevent an eligible applicant from proposing a project that includes such an initiative alongside efforts to directly reform Investing in Innovation Fund Absolute Priority 4 schools in accordance with the requirements of this priority.

Changes: None.

Comment: A number of commenters expressed support for the provision in this priority that included multiple pathways for students to obtain a regular high school diploma as a targeted approach to reform. Several commenters recommended, however, that the Department revise the priority to also support programs that provide alternative diplomas as viable graduation pathways. One of these commenters recommended that the Department recognize, in particular, General Education Development (GED) programs that connect GED students to postsecondary education.

Discussion: We appreciate the commenters’ support for the priority. However, we do not believe it is appropriate to include support for programs that provide alternative graduation credentials (such as GED programs) because such credentials, unlike regular high school diplomas, are not necessarily aligned with State academic content and achievement standards.

Changes: None.

Comment: One commenter recommended that the Department ensure that there is continued funding for schools that have successfully implemented reform approaches under this priority so that these schools do not hit a funding cliff that jeopardizes their performance gains.

Discussion: We appreciate the commenter’s concern and believe that Selection Criterion F (Sustainability) will help ensure that projects that receive funding under this priority will not be subject to sudden losses of or decreases in funds at the end of the grant period.

Changes: None.

Persistently Low-Performing Schools

Comment: One commenter noted differences between the proposed definition of persistently low-performing schools used in this priority and the definition of similar terms used in other programs supported with ARRA funds. The commenter recommended that the Department use consistent terminology and definitions of terms across programs.

Discussion: We appreciate the commenter’s concern. As the commenter notes, other programs supported with ARRA funds (including the School Improvement Grants, State Fiscal Stabilization Fund, and Race to the Top Fund programs) use and define the term “persistently lowest-achieving schools”. ¹Under this priority, we intend to support reform projects for schools that include, but are not limited to, the schools that meet the definition of “persistently lowest-achieving schools” used in those programs because we believe that focusing only on schools that meet the definition of “persistently lowest-achieving schools” would create a pool of schools for this priority that is overly narrow. However, we recognize that defining the term persistently low-performing schools as including, but not limited to, the schools that meet the definition of the similar term “persistently lowest-achieving schools” may be confusing to stakeholders (including prospective applicants for the different ARRA programs). Therefore, we are removing the definition of persistently low-performing schools and revising the priority to specify the schools for which the priority supports reform projects. To further prevent confusion with terms used in other programs supported with ARRA funds, we refer to these schools as Investing in Innovation Fund Absolute Priority 4 schools.

Changes: We are removing the definition of persistently low-performing schools and are revising the priority to specify that Investing in Innovation Fund Absolute Priority 4 schools are schools in any of the following categories: (a) persistently lowest-achieving schools (as defined in the final requirements for the School Improvement Grants program); (b) Title I schools that are in corrective action or restructuring under section 1116 of the ESEA (a); or (c) secondary schools (both middle and high schools) eligible for but not receiving Title I funds that, if receiving Title I funds, would be in corrective action or restructuring under section 1116 of the ESEA. These schools are referred to as Investing in Innovation Fund Absolute Priority 4 schools.

Comment: A number of commenters recommended that the Department revise the definition of persistently low-performing schools used in this priority to include additional types of schools. Two commenters recommended that the Department expand the definition of persistently low-performing schools to include low-performing non-Title I elementary schools and schools that without support would be at risk of becoming low-performing because they serve high-poverty communities. One commenter recommended that the Department revise the definition to include high schools, regardless of their AYP status, that are eligible for Title I and are “drop-out factories” (where a typical freshman class shrinks by 40 percent or more by the time the students reach their senior year) and middle schools, regardless of AYP status, that are feeder schools for these high schools. Two commenters recommended expanding the definition to include high schools with graduation rates below 60 percent. Another commenter recommended including schools in feeder patterns of high schools with low high school graduation rates compared to national or statewide averages, whether or not these schools are in improvement, corrective action, or restructuring. Another commenter recommended that the Department expand the definition to include schools, regardless of their AYP status, that are eligible for Title I funds and where persistent low performance has led to a decline in enrollment of 30 percent or greater over the last three years. One commenter recommended that the Department expand the definition of persistently low-performing schools used in this priority to include alternative schools and school programs serving incarcerated students and students held in juvenile detention facilities. Another commenter recommended including tribal and BIE schools in this definition.

Discussion: In general, the schools cited by commenters may be Investing in Innovation Fund Absolute Priority 4 schools if they are included in one of the three categories of schools listed in the priority. We do not believe it is appropriate to identify every type of school that may be included in these categories since there is variation in performance within common school types. For example, not all schools that serve incarcerated youth may necessarily be included in these categories.

With respect to low-performing non-Title I elementary schools, we do not believe it is necessary to revise the priority to include these schools because elementary schools are much more likely to receive Title I funds than secondary schools. If an elementary school is low-performing, it will thus in all likelihood be included in category (a) or (b) identified in the priority.

With respect to schools that without support would be at risk of becoming low-performing because they serve high-poverty communities, we believe that this priority should be used to focus attention on improving schools that have a record of low performance and do not believe it is appropriate to revise the priority to include support for reform efforts for schools that may become but are not currently low-performing.

Changes: None.

Comment: One commenter recommended that the Department require applicants to use data to assess the level of need in persistently low-performing schools. The commenter recommended this option to avoid what the commenter referred to as the “one size fits all challenge” under the ESEA whereby, according to the commenter, some schools fail to meet AYP because they miss their targets in one student subgroup, whereas other schools perform poorly across all subgroups and fail to meet AYP.

Discussion: An eligible applicant would not be prohibited from identifying, from among the Investing in Innovation Fund Absolute Priority 4 schools, specific schools that the eligible applicant intends to serve based on level of need or other factors. We do not believe that it is necessary to require eligible applicants to consider such factors, and that the priority, as written, will focus resources on schools with critical needs.

Changes: None.

Competitive Preference Priority 5--Innovations for Improving Early Learning Outcomes

Comment: A number of commenters expressed support for the inclusion of this proposed competitive preference priority and emphasized the importance of early learning for success later in life. Another commenter noted that this priority presents an opportunity to build on early learning’s research base. Several commenters recommended that the Department designate this priority as a fifth absolute priority in light of evidence that high-quality early learning programs can significantly close achievement gaps.

Discussion: We appreciate the commenters’ support for this priority. However, as stated elsewhere, we believe it is important to limit the absolute priorities under this program to the four education reform areas of the ARRA. Therefore, we decline to add innovations for improving early learning outcomes as an absolute priority.

Change: None.

Comment: One commenter recommended that the Department revise the priority to allow applicants to serve children from birth through fifth grade, rather than through third grade, in order to maintain students’ initial academic gains.

Discussion: We appreciate the commenter’s concern for sustaining early learning gains of students into later grades. While the Department is committed to ensuring that supports for all children are emphasized throughout our programs, we recognize that there are specific needs of early learners that can be addressed through targeted reforms. Further, inclusion of children birth to 3^rd grade is a widely-accepted range amongst the education community. For these reasons, we decline to make the changes suggested by the commenter.

Changes: None.

Comment: Two commenters asserted that pre-kindergarten or early childhood programs are often privately managed. These commenters suggested that the Department clarify whether projects under this priority can serve children enrolled in privately-managed programs.

Discussion: The primary goal of programs supported with ARRA funds is to improve the academic achievement and attainment of students in public elementary and secondary schools. However, to the extent that private early learning programs support students’ future achievement and growth in elementary and secondary education, an eligible applicant would not be prohibited under this priority from serving children enrolled in private early learning programs, provided the applicant’s proposed project met all requirements of the priority. An early learning provider would be eligible to apply for funding under this program if it is (1) an LEA or (2) a nonprofit organization applying in partnership with one or more LEAs or a consortium of schools.

Changes: None.

Comment: Several commenters recommended that the Department revise the priority to support projects pertaining to specific innovations for improved early learning outcomes. For instance, one commenter recommended that the Department revise the priority to include support for practices, strategies, or programs that improve, within an LEA’s geographic area, the collaboration among community-based early childhood providers and schools. Two commenters recommended that the Department revise the priority to include support for partnerships with community-based organizations and families in order to improve alignment between early learning programs and instruction in the early elementary grades. One commenter recommended that we revise the priority to include support for practices, strategies, or programs that serve children with disabilities in early learning environments. One commenter recommended that the Department revise the priority to emphasize the importance of socio-economically and racially diverse educational settings during students’ formative years because attitudes about race are still forming at this time. One commenter recommended that we revise the priority to support projects to improve and align early learning curricula with developmentally, culturally, and linguistically appropriate standards and assessments. A few commenters recommended that the Department revise the priority to support practices, strategies, or programs that emphasize teaching strategies that illustrate real-world applications of early learning subjects; we presume the commenter is referring to contextual learning opportunities. A few commenters recommended that we revise the priority to support practices, strategies, or programs that improve the skills of teachers in early learning programs.

One commenter suggested that we revise the priority to include support for projects that provide safe and enriching early learning physical settings and linkages to related health and human services. Several commenters recommended that the Department revise the priority to include support for parent engagement or assistance in the early learning of children. Another commenter recommended that the Department revise the priority to include strategies for conducting local outreach about early learning opportunities that target parents of high-need students in non-academic settings.

Discussion: There is nothing in this priority that precludes an eligible applicant from proposing any of the projects mentioned by the commenter, provided that the projects address paragraphs (a), (b), and (c) of the priority and also meet the eligibility and other requirements specified in this notice. We made this priority broad to provide eligible applicants with flexibility to propose a variety of projects. For this reason, we conclude that it is not necessary to revise the priority to include an express focus on the activities identified by the commenters.

Changes: None.

Comment: One commenter recommended that the Department revise the priority to allow applicants to address only one, rather than all three of the areas of focus, in order to meet the competitive preference priority. Specifically, the commenter recommended that the areas of focus be separated by “or” rather than “and.”

Discussion: We appreciate the commenter’s concern for applicant flexibility under this priority; however, we note that this is a competitive preference priority and applicants are under no obligation to address the priority in their applications. Moreover, we believe that, in order to meaningfully improve early learning outcomes for children, projects under this priority should address each of the focus areas and that these components are equally essential to early learning outcomes. For these reasons, we decline to make the changes recommended by the commenter.

Changes: None.

Comment: One commenter suggested that the Department ensure that the priority is aligned with the President’s Zero to Five Plan.

Discussion: This priority is consistent with the President’s Zero to Five Plan. For example, the Zero to Five Plan supports strategies that, among others, align early learning and development standards that lead to school readiness and are integrated with program quality to guide curriculum and program development. The Zero to Five Plan also encourages the development of evidence-based quality rating systems structured with progressive levels of quality--which may be used across early learning settings and programs. Accordingly, we believe that this priority is consistent with the President’s Zero to Five Plan and supports early learning initiatives under that program. For more information about this plan (as well as the Department’s Early Learning Challenge Fund), please see http://www.ed.gov/about/inits/ed/earlylearning/elcf-factsheet.html.

Changes: None.

Comment: None.

Discussion: This priority includes a reference to “core academic subjects.” Consistent with the revisions we are making to the other priorities that use this term, we are revising the priority to add a reference to section 9101(11) of the ESEA, which includes the definition of “core academic subjects”.

Changes: We are revising this priority to include the statutory reference to the definition of “core academic subjects” in section 9101(11) of the ESEA.

Competitive Preference Priority 6--Innovations That Support College Access and Success

Comment: Two commenters recommended that the Department revise the priority to include career and technical education systems that prepare students simultaneously for postsecondary education and careers. Similarly, two commenters recommended expanding the priority to include, in addition to programs that promote success in two- and four-year colleges, programs that promote success in career certificate programs and entry into the workforce.

Discussion: This priority supports projects that enable students to successfully prepare for, enter, and graduate from a two- or four-year college. As noted in the NPP, this priority is designed to help meet the national goal of restoring the United States to first in the world in the percentage of citizens holding college degrees. We believe we must maintain this focus and, therefore, decline to expand this priority to include applications that focus on practices, strategies, and programs that do not lead to success in two-and four-year colleges. A project that focuses on a career certificate program or a career-readiness program that is part of a career and technical education system would be eligible for competitive preference points under this priority only to the extent the project promotes success in two- and four-year colleges.

Changes: None.

Comment: One commenter stated that the priority focuses too heavily on non-academic issues such as helping students obtain financial aid and complete college applications. The commenter recommended that the Department revise the priority to support applications addressing both academic and non-academic issues associated with college access and success.

Discussion: In order to meet this competitive preference priority, applications must include practices, strategies, or programs for K-12 students that address students’ preparedness related to college, which may include ensuring that students are academically prepared for college. Therefore, it is unnecessary to revise the priority in the manner recommended by the commenter.

Changes: None.

Comment: Two commenters recommended that the Department revise the priority to support approaches that focus on decreasing dropout rates or increasing high school graduation rates.

Discussion: As stated elsewhere in this notice, the Department is using the Investing in Innovation Fund to support the overarching ARRA goal of improving student achievement and attainment. All applications for Investing in Innovation Fund grants will be assessed in part on the extent to which the proposed projects will have an impact on student achievement and attainment outcomes including the following: improving student achievement or growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates. Accordingly, peer reviewers will consider the magnitude of the effect of proposed projects on attaining these student outcomes (see, in particular, Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect)). Therefore, it is unnecessary to revise the priority in the manner suggested by the commenters.

Changes: None.

Comment: A few commenters recommended that the Department revise the priority to recognize GED programs as a viable graduation pathway for students and support projects that focus on the development of college-ready GED programs.

Discussion: As noted elsewhere in this notice, we do not believe it is appropriate to support projects that provide alternative graduation credentials (such as GED programs) because such credentials, unlike regular high school diplomas, are not necessarily aligned with State academic content and achievement standards.

Changes: None.

Comment: One commenter recommended that the Department revise the priority to include programs that provide services to and monitoring of students after enrolling in college.

Discussion: The Investing in Innovation Fund program does not provide funding for projects that are designed to serve students who are enrolled in college. Therefore, we decline to revise this priority in the manner suggested by the commenter.

Changes: None.

Comment: Several commenters recommended that the Department revise the priority to include support for middle school students as well as high school students. Other commenters recommended that the priority be revised to include a focus on supporting students in the early high school grades, including strategies that aim to assess the college readiness of students and close skill gaps before students graduate.

Discussion: This priority specifically states that competitive preference will be given to applications for practices, strategies, or programs that enable K-12 students to successfully prepare for, enter, and graduate from a two- or four-year college. Thus, this priority would include support for middle school students and students in the early high school grades.

Changes: None.

Competitive Preference Priority 7--Innovations To Address the Unique Learning Needs of Students With Disabilities and Limited English Proficient Students

Comment: A number of commenters expressed support for this priority and the Department’s efforts to support programs focused on improving outcomes for students with disabilities and limited English proficient students. Several commenters recommended that the Department clarify whether applications must address the needs of both students with disabilities and limited English proficient students in order to meet this competitive priority. Two commenters recommended that the Department separate the priority into two competitive preference priorities given the different needs of these students.

Discussion: We appreciate the commenters’ support for this priority and believe that the priority is clear that an applicant may propose a project under the priority that addresses the needs of either students with disabilities or limited English proficient students. Therefore, we do not believe it is necessary to provide separate competitive preference priorities for projects that propose to serve these student subgroups individually.

Changes: None.
Comment: One commenter suggested that the Department clarify whether projects under this priority may focus on improving academic outcomes or increasing high school graduation rates of the students served, rather than addressing both of these measures.

Discussion: Our intent under this priority is to give a competitive preference to projects that propose practices, strategies, and programs for students with disabilities or limited English proficient students that both increase academic outcomes and increase college- and career-readiness (including increasing high school graduation rates) for these groups of students. However, in light of the achievement gaps for these students, we are revising the priority to state that, to meet the priority, projects must also be designed to close achievement gaps for these students.

Changes: We are changing this competitive preference priority to state that, in order to meet the priority, applications must provide for the implementation of particular practices, strategies, or programs that are designed to improve academic outcomes, close achievement gaps, and increase college- and career-readiness, including increasing high school graduation rates (as defined in this notice), for students with disabilities or limited English proficient students.

Comment: One commenter recommended that, in this priority, the Department use “English language learners” in place of “students with limited English proficiency” because the former term helps educators focus on a student’s capacity as a learner.

Discussion: The Department recognizes that stakeholders often use terms such as “English language learners” rather than “limited English proficient students” when referring to students who are acquiring basic English proficiency and developing academic English skills. However, because the ESEA defines the term “limited English proficient,” and both the statute and the implementing regulations use this term, as well as the phrase “students with limited English proficiency,” we will continue to use the latter terms in this program.

Changes: None.

Comment: We received a number of recommendations to revise the priority to focus on specific groups of limited English proficient students including students from linguistically isolated homes and underrepresented limited English proficient subpopulations, and high-school students who are recent arrivals to the United States. Another commenter recommended that the Department revise the priority to include a focus on “standard English learners” (i.e., students who were born in the United States and whose native language is English but who speak a nonstandard English dialect).

Discussion: Section 9101(25) of the ESEA specifies that a limited English proficient student is a student who (1) was not born in the United States or whose native language is a language other than English; (2) who is a Native American, Alaska Native, or resident of the outlying areas who comes from an environment where a language other than English has had a significant impact on the student’s level of English language proficiency; or (3) is migratory, whose native language is a language other than English, and who comes from an environment where a language other than English is dominant. Under this competitive preference priority, there is nothing that would prevent an eligible applicant from proposing an innovative practice, strategy, or program that addresses the needs of specific subpopulations of limited English proficient students or limited English proficient students from specific backgrounds, provided these students meet the requirements of the ESEA definition. We do not believe it is necessary to refer to specific groups of limited English proficient students in this priority.

Regarding “standard English learners,” these students do not meet the ESEA definition referenced above because they speak English as their native language. Because we are maintaining the focus of this priority on students who meet the definition of limited English proficiency under the ESEA, projects that focus only on these students would not meet this priority.

Changes: None.

Comment: Two commenters recommended including examples of the practices, strategies, and programs that would be supported under this priority. One of these commenters recommended providing examples of instructional models that have proven to be effective for limited English proficient students. The other commenter recommended revising the priority to include innovations referenced in the Individuals with Disabilities Education Act, as amended (IDEA), such as response-to-intervention models and the use of assistive technologies.

Discussion: In order to meet this competitive preference priority, eligible applicants must propose innovative practices, strategies, or programs that address the unique learning needs of students with disabilities or limited English proficient students and that are designed to improve academic outcomes, close achievement gaps, and increase college- and career-readiness, including increasing graduation rates, for these students. It is up to eligible applicants to identify those practices, strategies, or programs that they believe, based on available evidence, should be included in their proposed projects. We do not want to restrict or constrain the projects that this priority would support by identifying specific initiatives in the priority statement. Therefore, we decline to make the changes recommended by the commenters.

Changes: None.

Comment: Two commenters recommended that the Department revise the priority to ensure that projects funded under the priority are consistent with the principles of universal design for learning.

Discussion: An applicant would not be precluded from proposing under this priority projects that are consistent with the principles of universal design for learning, as defined in the Higher Education Act of 1965, as amended (HEA), provided that the proposed project meets the requirements in the priority. We decline to include this level of specificity in this competitive preference priority, as we do not want to restrict or constrain the innovative practices, strategies, and programs that this priority would support.

Changes: None.

Comment: Several commenters recommended that the Department include gifted and talented students among the students with unique learning needs to be served under this priority. A few of these commenters stated that the needs of gifted and talented students are typically underserved. Another commenter recommended including students with low literacy levels among the students with unique learning needs to be served under this priority.

Discussion: We recognize that gifted and talented students have unique learning needs and may be underserved in some areas of the country. In addition, we recognize that students with low literacy levels who are not students with disabilities or limited English proficient students may also have unique learning needs. However, we believe that it is important to maintain this competitive preference for projects that serve students with disabilities and limited English proficient students in light of the achievement gaps between these students and their peers. Therefore, we are not changing the priority in the manner suggested by the commenter.

Changes: None.

Proposed Competitive Preference Priority 8--Innovations That Serve Schools in Rural LEAs

Comment: Several commenters expressed support for this priority. However, other commenters recommended that the Department eliminate this competitive preference priority; these commenters asserted that the priority is unnecessary, and gives an unfair advantage to rural areas over urban LEAs that are equally in need of financial support. Other commenters stated that rural grant recipients may reach only small numbers of students and could not easily be brought to scale at the State or regional level. One commenter recommended that the Department revise the priority to support applications that include a focus on students in rural LEAs, rather than applications that serve students in rural LEAs exclusively. Another commenter recommended that the Department revise the priority to include projects that are proposed by non-rural LEAs that would serve or benefit students in rural LEAs.

Discussion: This competitive preference priority acknowledges that solutions to educational challenges may be different in rural areas than in urban and suburban communities and that there is a need for solutions to unique rural challenges. To meet this priority, an eligible applicant need not be a rural LEA. Any eligible applicant may propose a project to serve students in rural LEAs under this priority. With regard to the concern that projects meeting this competitive preference priority will reach small numbers of students or could not easily be brought to scale at State or regional levels, we note that all applications for Investing in Innovation Fund grants will be assessed in part on the number of students to be reached by the proposed project and the eligible applicant’s capacity to reach the proposed number of students during the course of the grant period (see Selection Criteria E (Strategy and Capacity to Bring to Scale (in the case of Scale-up and Validation grants); Strategy and Capacity to Further Develop and Bring to Scale (in the case of Development grants)). For these reasons, we decline to remove this priority or to change this priority in the manner recommended by commenters.

Changes: None.

Comment: We received a number of comments recommending that we revise this competitive preference priority to focus on specific types of projects in rural areas such as projects that improve college- and career-readiness of students in rural LEAs, projects that serve students across county and State lines, early learning projects, projects that increase the use of educational technology in rural LEAs, and projects that promote innovative strategies for educator recruitment in rural LEAs.

Discussion: An applicant would not be precluded from proposing under this priority any of the projects mentioned by the commenters provided that the proposed project meets the requirements in this priority (i.e., the proposed project focuses on the unique challenges of high-need students in schools within a rural LEA and addresses the particular challenges faced by students in these schools; and improves student achievement or student growth, closes achievement gaps, decreases dropout rates, increases high school graduation rates, or improves teacher and principal effectiveness in one or more rural LEAs). We cannot include in the priority all the possible programs that could address this competitive priority, nor do we want to restrict or constrain the innovative practices, strategies, and programs that this priority would support. Therefore, we decline to follow the commenters’ recommendations.

Changes: None.

Rural LEA

Comment: Several commenters recommended that the Department expand the definition of rural LEA used in this priority. One commenter recommended expanding the definition beyond the Small Rural School Achievement and Rural Low-Income School programs under Title IV, Part B of the ESEA to include small and medium-sized, low-performing, high-need LEAs in rural areas. One commenter recommended revising the definition to include LEAs designated as rural by the Locale Code in the National Center for Education Statistics Common Core of Data. Another commenter recommended revising the definition to be more expansive and inclusive of rural LEAs that used to be urban LEAs. Finally, several commenters recommended that the Department revise the priority to include practices, strategies, or programs that would serve students in one or more rural schools (irrespective of the designation of the LEA of those schools) rather than only students in LEAs that meet the definition of rural LEA.

Discussion: This competitive preference priority is intended to encourage applications that focus on the particular challenges faced by students in rural LEAs. In determining the definition of rural LEA for use in this program, we chose to use a definition that is used in many Department grant programs. In addition, we note that the definition of rural LEA for use in this program includes schools served by LEAs that are designated with a school locale code of 6, 7, or 8. Therefore, we do not believe the definition of rural LEA should be expanded in the ways suggested by commenters.

With regard to the recommendation that we include support under this priority for practices, strategies, or programs that serve students in one or more rural schools (irrespective of the designation of the LEA of those schools), we believe that most LEAs that have schools in rural areas would qualify as a rural LEA under the definition of rural LEA, and that accordingly no change to the priority is necessary.

Changes: None.

REQUIREMENTS

Providing Innovations that Improve Achievement for High-Need Students

Comment: A number of commenters expressed support for the Department’s requirement that applicants implement practices, strategies, or programs for high-need students. Two commenters, however, argued that eligible applicants should not be required to serve only high-need students.

Discussion: In this program, we define high-need student as a student at risk of educational failure or otherwise in need of special assistance and support. While eligible applicants are required to implement practices, strategies, or programs for high-need students, eligible applicants have discretion in determining which types of students meet this definition. Moreover, nothing in the authorizing statute or the priorities, requirements, definitions, or selection criteria for this program prohibits eligible applicants from using program funds to help other students as well. Indeed, the Department expects that robust proposed projects would benefit all students, but with disproportionate benefit to high-need students. We believe that this program’s focus on funding projects that serve high-need students--students at risk of educational failure or otherwise in need of special assistance and support--is consistent with the goal of this program, which is to improve student academic achievement and attainment.

Consistent with other clarifying changes we are making with respect to the use of the term “applicant” and “eligible applicant” throughout this notice, we are making a minor technical change to the Providing Innovations that Improve Achievement for High-Need Students requirement.

Changes: We are replacing the word “applicant” in this requirement with the words “eligible applicant” to clarify that it is the eligible applicant (i.e., the LEA or the partnership) that must implement practices, strategies, or programs for high-need students (as defined in this notice).

Eligible Applicants

Comment: As discussed in more detail in the following paragraphs, a number of commenters asked about the roles and responsibilities of “eligible applicants,” “applicants,” “fiscal agents,” and “partners” under this program.

Discussion: In analyzing this group of comments, the Department determined that there appears to be some confusion about how these important terms are used in the context of this program. For this reason, we are adding definitions for the terms applicant, official partner, and other partner.

Section 14007(a)(1) of the ARRA describes the types of entities that are eligible to apply for funding under this program. These eligible entities, referred to in this notice as “eligible applicants,” must be either (a) an LEA, or (b) a partnership between a nonprofit organization and (1) one or more LEAs or (2) a consortium of schools. An “eligible applicant,” therefore, is either an LEA or a partnership.

For applications that are submitted on behalf of partnerships, consortia, or groups--as is necessarily the case under section 14007(a)(1)(B) of the ARRA, the Department makes an award to a single entity only. The entity designated by the partnership, consortia or group to apply on behalf of it to the Department in accordance with 34 CFR 75.127 to 75.129 (the Department’s regulations governing group applications) is referred to as the applicant. If the group application is awarded a grant, the applicant then becomes the “grantee.” Under this program, an applicant (or grantee) may, therefore, be--

(a) An LEA² under section 14007(a)(1)(A) of the ARRA; or

(b) A nonprofit organization, an LEA, or a school in a consortium of schools applying on behalf of a partnership provided that the partnership is between a nonprofit organization and (1) one or more LEAs or (2) a consortium of schools (pursuant to section 14007(a)(1)(B) of the ARRA).

For applications submitted under section 14007(a)(1)(B) of the ARRA, a single applicant, which could be the nonprofit organization, an LEA, or a school in the consortium of schools that is part of the partnership, must submit a group application on behalf of the eligible applicant (i.e., the partnership). This partnership must include the partners referenced in section 14007(a)(1)(B) of the ARRA. For the sake of clarity, we refer to each of the partners referenced in section 14007(a)(1)(B) of the ARRA as an official partner (i.e., the nonprofit organization and, depending on the make-up of the partnership, each LEA or consortium of schools in the partnership).

The Department anticipates that LEAs and section 14007(a)(1)(B) partnerships may wish to propose projects that involve working with additional entities as well. For purposes of this program, we define any of these other entities as an other partner. Therefore, an LEA applying under section 14007(a)(1)(A) of the ARRA may apply with a proposed project that involves working with other partners. Likewise, an applicant applying on behalf of a partnership in accordance with section 14007(a)(1)(B) of the ARRA may propose a project that involves working with additional official partners, other partners, or both, provided that the partnership includes the minimally required official partners.

We believe that the distinction between official partners and other partners is necessary, especially in light of the addition of the subgrant authority to section 14007 of the ARRA as a result of section 307 of Division D of the Consolidated Appropriations Act, 2010 (P.L. 111-117). New section 14007(d) of the ARRA provides that, in the case of an eligible applicant that is awarded a grant and is in a partnership described in section 14007(a)(1)(B) of the ARRA, the partner serving as the fiscal agent³ may make subgrants to one or more of the other entities in the partnership. We interpret this subgrant authority to permit the grantee to make subgrants to only those partners identified in the statute (i.e., official partners), but not to other entities that are proposed to be involved in a project (i.e., other partners). A grantee can make subgrants to any official partner, including those that are in addition to the minimally required official partners.

Changes: In the DEFINITIONS section, we define the term applicant to mean the entity that applies for a grant under this program on behalf of an eligible applicant (i.e., an LEA or partnership in accordance with section 14007(a)(1)(B) of the ARRA). We also define the term official partner as any of the entities required to be part of a partnership under section 14007(a)(1)(B) of the ARRA. Finally, we define the term other partner to mean any entity, other than the applicant and any official partner that may be involved in a proposed project. We use these terms, as appropriate, throughout this notice. We also have revised other sections of the notice to use these terms, where appropriate.

Comment: A number of commenters recommended that the Department broaden the Eligible Applicants requirement to include additional types of applicants. The entities suggested by the commenters to be made eligible include the following: State educational agencies, municipalities and other units of local government, and other public agencies and institutions; Native American Tribes and the Bureau of Indian Education; institutions of higher education, including community colleges and accredited four-year baccalaureate degree-granting institutions; local and regional early intervention and preschool programs under part B or C of the IDEA; private schools including religious schools; community-based organizations; youth councils; teacher unions in partnership with LEAs; workforce investment boards; for-profit charter management organizations; nonprofit organizations applying independently of an LEA or consortium partnership; and nonprofit organizations partnering with individual schools rather than with consortia of schools or LEAs.

Discussion: Section 14007(a)(1) of the ARRA describes the types of entities that are eligible to apply for funding under this program. The Department has no authority to expand this statutorily-prescribed requirement.⁴

With respect to most of the entities mentioned by the commenters, the critical questions for determining whether the entity is an eligible applicant are (1) whether it includes an entity that qualifies as a nonprofit organization (as defined in this notice) and (2) whether the nonprofit organization has partnered with one or more LEAs or a consortium of schools. In this program, we define nonprofit organization as an entity that meets the definition of “nonprofit” under 34 CFR 77.1(c) or is an institution of higher education under section 101(a) of the HEA. Section 77.1(c) defines the term “nonprofit”, as applied to an agency, organization, or institution, as meaning that it is owned and operated by one or more corporations or associations whose net earnings do not benefit, and cannot lawfully benefit, any private shareholder or entity. The definition of “institution of higher education” in section 101(a) of the HEA includes both public and private two- and four-year institutions of higher education. Partnerships that include an entity that meets this definition of nonprofit organization and that partner with one or more LEAs or a consortium of schools are eligible to apply for funding under this program; those that do not include an entity that meets the definition or that do not partner with one or more LEAs or a consortium of schools are not eligible.

However, nothing in the authorizing statute or the priorities, requirements, definitions, and selection criteria for this program prevents an eligible nonprofit organization that partners with one or more LEAs or a consortium of schools in accordance with section 14007(a)(1)(B) of the ARRA from applying with a proposed project that involves the eligible applicant working with other entities, including those mentioned by the commenters. These other entities would be considered other partners, as that term is defined in this notice.

Further, as noted in the preceding discussion, the Congress amended the authorizing statute for this program with respect to a grantee’s ability to make subgrants. Under new section 14007(d) of the ARRA, in the case of an eligible entity that is a partnership under section 14007(a)(1)(B) of the ARRA, the partner serving as the fiscal agent may make subgrants to one or more of the other entities in the partnership. We are revising the requirements for this program to incorporate this statutory change. In doing so, we interpret the fiscal agent’s (i.e., the applicant’s) ability to make subgrants as extending only to the official partners.⁵ Thus, while an eligible applicant can include other partners in its section 14007(a)(1)(B) partnerships, the applicant may not make subgrants to those other partners.

Changes: As discussed elsewhere in this notice, we are revising the requirements for this program, consistent with the amendments to section 14007 of the ARRA, to specify that, in the case of an eligible applicant that is a partnership between a nonprofit organization and (1) one or more LEAs or (2) a consortium of schools, the partner serving as the applicant may make subgrants to one or more official partners.

Comment: One commenter recommended that the Department clarify the circumstances under which an applicant may submit multiple applications for different projects.

Discussion: Under this program, an eligible applicant may apply alone, if it is an LEA, or on behalf of a partnership pursuant to section 14007(a)(1)(B) of the ARRA. Applications submitted on behalf of partnerships, consortia, or groups are subject to the Department’s regulations in 34 CFR 75.127 and 75.129. Any applicant, whether it is an LEA or the entity within the partnership designated as the applicant, may submit multiple applications for substantially different projects.

However, to ensure that this program provides funding for the widest possible array of innovative projects, we are adding to the requirements for this program limits on the awards made to any individual grantee (see Limits on Grant Awards). Under this requirement, the Department will not award more than two grants to any grantee. Additionally, no grantee may receive more than $55 million in grant awards under this program. Because we estimate that the maximum awards will be $50 million, $30 million, and $5 million for Scale-up, Validation, and Development grants, respectively, this requirement effectively means that a grantee awarded a Scale-up grant may also receive a Development grant, but may not receive a Validation grant or a second Scale-up grant.

We note, in addition, that the Funding Categories requirement for this program prohibits an applicant from submitting an application for the same proposed project under more than one type of grant.

Changes: We are adding to the requirements limits on the awards made to an individual grantee under this program. Under this requirement, the Department will not award more than two grants to any grantee under this program. Additionally, no grantee may receive more than $55 million in grant awards under this program.

Comment: Several commenters asked that the Department include definitions of the terms “LEA” and “educational service agency” from the ESEA in order to clarify that educational service agencies are eligible applicants under this program.

Discussion: Consistent with section 14013(6) of the ARRA, any term used in this program that is not defined in the ARRA but is defined in section 9101 of the ESEA shall have the meaning given the term in that section. The term “local educational agency” is defined in section 9101(26) of the ESEA. Accordingly, we are using the definition of “local educational agency” in section 9101(26) of the ESEA for this program. This definition specifically includes educational service agencies (defined in section 9101(17) of the ESEA) and consortia of those agencies; thus, an educational service agency may be an eligible applicant under this program. We believe it is unnecessary to include these definitions in this notice as they are readily available to interested parties.

While we do not include these definitions in this notice, we do include a note about eligibility for LEAs under this program. The note clarifies that, to be eligible for this program, an LEA (whether it is the applicant or an official partner) must be within one of the 50 States, the District of Columbia, or the Commonwealth of Puerto Rico.

Changes: Consistent with other minor changes related to the use of the terms “applicant” and “eligible applicant,” we are making a minor change to the Note about LEA Eligibility.

Comment: Two commenters recommended that the Department clarify whether a partnership of multiple LEAs may apply for funding under this program (as opposed to a single LEA applying on its own).

Discussion: The Department only makes grant awards to single entities; the single entity can apply on behalf of itself or on behalf of a group, consortium, or partnership in accordance with the Department’s group application regulations in 34 CFR 75.127 through 75.129.

Under this program, a single LEA may apply for a grant pursuant to section 14007(a)(1)(A) of the ARRA. However, as a single applicant, it could propose a project that involves working with other partners (as that term is defined in this notice); these other partners could include other LEAs that do not meet the eligibility requirements for this program.⁶ Finally, a single LEA may serve as the applicant for a partnership applying under section 14007(a)(1)(B) of the ARRA. This partnership must include the official partners, which could include one or more LEAs, and may also include other LEAs as other partners.

Changes: We are adding language to the Eligible Applicants section of the requirements to clarify that an eligible applicant that is a partnership applying under section 14007(a)(1)(B) of the ARRA must designate one of its official partners (as defined in this notice) to serve as the applicant in accordance with the Department’s regulations governing group applications in 34 CFR 75.127 through 75.129.

Comment: Several commenters recommended that the Department clarify whether an eligible applicant that is a partnership may include multiple nonprofit organizations.

Discussion: An eligible partnership must include at least one nonprofit organization as an official partner. An eligible partnership may include additional nonprofit organizations as additional official partners (as defined in this notice) or as other partners. If a nonprofit organization is an other partner (i.e., not an official partner), that nonprofit organization would not be eligible to receive a subgrant from the applicant.

Changes: None.

Comment: A few commenters recommended that the Department clarify whether charter schools are eligible applicants. Specifically, one commenter recommended clarifying whether charter schools that are not identified in State law as having LEA status, but are otherwise eligible applicants, may apply without the review or approval of an LEA.

Discussion: As discussed earlier in this notice, depending on its legal status under State law, a charter school may be eligible to apply under this program in the following ways: as an LEA on its own (if it is considered an LEA under State law); as a nonprofit organization, in partnership with one or more LEAs or a consortium of schools (if it meets the definition of nonprofit organization under this program); or in partnership with a nonprofit organization as an LEA (if it is considered an LEA under State law) or as part of a consortium of schools (if it not considered an LEA under State law). Because charter school laws vary from State to State, we encourage any charter school interested in applying for funds under this program to verify its status and authority to receive funds before applying.

A charter school that does not qualify as an LEA, a nonprofit organization, or a school in a consortium of schools may still be able to be involved with a project funded under this program. It could do so as an other partner (as defined in this notice) provided that the eligible applicant for the project met all of the eligibility requirements.

Changes: None.

Comment: One commenter expressed concern that the legal framework of the commenter’s State would prevent entities from that State from being eligible to apply for funding under this program.

Discussion: In general, the requirements for eligible applicants under this program do not relate to State statutes or regulations. Applicants are required to certify, as part of their application, that they have the legal authority to receive program funds.

Changes: None.

Nonprofit Organization

Comment: Several commenters requested that the Department explicitly state in the notice that nonprofit organizations may be the fiscal agent in an application. Some of these commenters expressed concern that if only LEAs or consortia of schools can be the fiscal agent for a grant, this might lead them to minimize the roles and responsibilities of their nonprofit partners.

Discussion: A nonprofit organization may serve as the fiscal agent (i.e., the applicant) applying on behalf of a partnership under section 14007(a)(1)(B) of the ARRA.

Changes: None.

Comment: A few commenters recommended that the Department clarify whether nonprofit organizations that do not directly work in schools or with LEAs may still partner with LEAs or consortia of schools as eligible applicants.

Discussion: There is no requirement that a nonprofit organization applying in partnership with one or more LEAs or a consortium of schools under section 14007(a)(1)(B) of the ARRA have a history of working directly in schools or with LEAs. However, consistent with the amendments to the eligibility requirements for this program made by the Consolidated Appropriations Act, 2010 (as discussed elsewhere in this notice), for partnerships that include a nonprofit organization and one or more LEAs or a consortium of schools, the nonprofit organization must have a record of significantly improving student achievement, attainment, or retention in order to be eligible for an award under this program. In accordance with the requirements established in this notice, an eligible applicant that includes a nonprofit organization must demonstrate that the nonprofit organization has such a record through its record of work with an LEA or schools in the past.

Changes: None.

Comment: Two commenters recommended that the Department clarify whether nonprofit organizations may submit applications that include multi-city or multi-State partners (i.e., LEAs or schools in different cities or States).

Discussion: Nothing in the authorizing statute or the priorities, requirements, definitions, or selection criteria for this program prohibits nonprofit organizations from partnering with LEAs or schools in different geographic locations.

Changes: None.

Comment: One commenter expressed concern that the proposed definition of nonprofit organization includes institutions of higher education. The commenter asserted that the Congress did not intend to include these institutions as eligible nonprofit organizations in section 14007(a)(1)(B) partnerships.

Discussion: Nothing in the authorizing statute for this program prohibits the inclusion of institutions of higher education in partnerships eligible to apply under section 14007(a)(1)(B) of the ARRA. Further, we believe that institutions of higher education possess unique expertise--particularly regarding methods of evaluation--that will positively benefit the types of projects that the Department seeks to fund under this program. We have concluded, based on our review of sections 14007 and 14013 of the ARRA and section 101(a) of the HEA, that all entities that meet the definition of institution of higher education under section 101(a) of the HEA--whether they are public or private--may be considered nonprofit organizations for purposes of this program.

Changes: None.

Consortium of Schools

Comment: Several commenters expressed concern that the definition of the term consortium of schools limits the schools that may be included in a consortium only to public schools. These commenters requested that the Department expand the definition of consortium of schools to include private schools, as well.

Discussion: We believe that it is consistent with the goals of the ARRA, which include improving the academic achievement and attainment of students in public elementary and secondary schools, to define consortium of schools to include only public schools. However, as discussed earlier in this notice, a private school may be a partner within an eligible applicant if it qualifies as a nonprofit organization and if it partners with one or more LEAs or a consortium of public schools. In addition, we note that private schools may be included as other partners and students in those schools could be served by projects that receive funding under this program.⁷

Changes: None.

Comment: One commenter recommended that the Department expand the definition of consortium of schools to include Bureau of Indian Education (BIE) schools.

Discussion: The definition of consortium of schools includes BIE schools because BIE schools are public schools. We note also that a BIE school may be eligible to apply as an LEA on its own, or in partnership with a nonprofit organization, as an LEA, because the definition of local educational agency in section 9101(26) of the ESEA (which we are using in this program) includes a provision under which a BIE school may be considered an LEA. If a BIE school is considered an LEA, the BIE school would be able to apply as an eligible LEA on its own, or in partnership with a nonprofit organization, consistent with the requirements for eligible applicants under this program. In addition, a BIE school could also be involved with a project as an other partner.

Changes: None.

Comment: One commenter suggested that the Department broaden the definition of consortium of schools to include university schools of education.

Discussion: The proposed definition of consortium of schools is limited to public elementary and secondary schools. As discussed earlier in this notice, we regard this definition as consistent with the authorizing statute’s goal of improving the academic achievement and attainment of students in public elementary and secondary schools. However, as discussed elsewhere in this notice, an institution of higher education (as defined in section 101(a) of the HEA) may apply for funding under this program as a nonprofit organization in partnership with one or more LEAs or a consortium of schools. In addition, an institution of higher education could also be involved with a project as an other partner.

Changes: None.

Comment: One commenter recommended that the Department expand the definition of consortium of schools to include public or private early learning providers.

Discussion: In the NPP, we proposed to define consortium of schools as two or more public elementary or secondary schools. As discussed earlier in this notice, we determined that including only public elementary and secondary schools in this definition is consistent with the ARRA’s goal of improving the academic achievement and attainment of students in public elementary and secondary schools. Thus, we decline to include early learning providers in the definition of consortium of schools, unless they are considered to be part of a public elementary school under State law. However, any early learning provider (whether public or private) would be eligible to apply for funding under this program if it is (1) an LEA or (2) a nonprofit organization applying in partnership with one or more LEAs or a consortium of schools. In addition, an eligible applicant (whether an LEA or partnership applicant) would not be prohibited from including early learning providers as other partners to the proposed project provided that the eligible applicant otherwise met the eligibility requirements. We believe that these provisions are sufficient to allow for the participation of early learning providers in projects under this program.

Changes: None.

Comment: One commenter expressed concern that many public schools may be unaware that it is illegal for them under State law to accept Federal funding that is not distributed through their LEA. The commenter recommended including a note in the requirements providing that any applicant that applies on behalf of a partnership that includes a consortium of schools must include as part of the application a signed authorization from the legal authority for each of the schools in the consortium (generally the LEA).

Discussion: Eligible applicants should act consistent with State law when applying for, receiving, or using funds under this program. Applicants are required to certify, as part of their application, that they have the legal authority to receive program funds. We do not believe it is necessary also to require that an applicant include as part of its application a signed authorization from the legal authority for each of the schools in the consortium.

Changes: None.

Eligibility Requirements

Eligibility Requirements in General

Note: As noted in the NPP, proposed paragraphs (1) through (4) of the eligibility requirements for this program repeated requirements prescribed by section 14007 of the ARRA. We included these requirements in the NPP for clarity. As we do not have authority to alter or eliminate statutorily-prescribed requirements, we do not discuss comments recommending changes to, or deletions of, these requirements. However, we also received a number of comments requesting further clarification of the proposed requirements or recommending inclusion of additional eligibility requirements. We discuss those comments in the paragraphs that follow.

In addition, we note that, since publication of the NPP, the Congress amended the ARRA with respect to the eligibility requirements for this program. We are revising the eligibility requirements for this program to incorporate those statutory changes. We discuss these revisions in the immediately following paragraphs and elsewhere in this section, as appropriate.

Comment: None.

Discussion: As stated in the NPP, paragraphs (1) through (4) of the proposed eligibility requirements for this program repeated requirements prescribed by section 14007 of the ARRA. Section 307 of Division D of the Consolidated Appropriations Act, 2010 (P.L. 111-117), which was signed into law on December 16, 2009, made several amendments to these statutory requirements. The major substantive changes to section 14007 are discussed in the following paragraphs.

Section 14007(b)(1) has been amended to require that, to be eligible for an award under this program, an eligible applicant must (A) have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or (B) have demonstrated success in significantly increasing student academic achievement for all groups of students described in such section. In addition, section 14007(b)(2) of the ARRA has been eliminated; this section would have required that an eligible applicant have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years or have demonstrated success in significantly increasing student achievement for all groups of students described in that section through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress). As a result of this amendment, to be eligible for an award, eligible applicants are no longer required to have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years. In addition, the statutory changes make clear that eligible applicants do not have to show that they have both significantly closed achievement gaps and significantly increased student achievement for all groups described in section 1111(b)(2) of the ESEA. Rather, eligible applicants must show either (A) that they have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA or (B) that they have demonstrated success in significantly increasing student academic achievement for all groups described in such section.

Section 14007(c) of the ARRA has been amended to specify that an eligible applicant that includes a nonprofit organization is considered to have met the requirements of new paragraphs (1) and (2) of section 14007(b) if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. Under the amendments to section 14007(c), an eligible applicant that includes a nonprofit organization is thus no longer required to demonstrate that the nonprofit organization has a record of each of the following: (1) having significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA; (2) having exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years or having demonstrated success in significantly increasing student achievement for all groups of students described in that section through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress); and (3) having made significant improvement in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and school leaders, as demonstrated with meaningful data. Instead, an eligible applicant is required to demonstrate that the nonprofit organization in the partnership has a record of significantly improving student achievement, attainment, or retention.

In addition, section 14007(c) of the ARRA has been amended to specify that an eligible applicant that includes a nonprofit organization is considered to have met the requirements of new paragraph (3) of section 14007(b) if it demonstrates that it will meet the requirement relating to private-sector matching. This statutory change makes clear that the requirement in section 14007(b)(3) of the ARRA relating to establishing partnerships with the private sector does not apply to such an eligible applicant, as the eligible applicant by its very nature consists of such a partnership, and thus does not require an eligible applicant that includes a nonprofit organization to establish additional partnerships with the private sector.

Changes: We are making several changes to the eligibility requirements for this program to reflect these statutory changes. Consistent with the amendments to section 14007(b) of the ARRA, we are revising proposed paragraph (1) of the eligibility requirements to require that, to be eligible for an award under this program, an eligible applicant must--except as specifically set forth in the requirements: (A) have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or (B) have demonstrated success in significantly increasing student academic achievement for all groups of students described in such section. We are also removing proposed paragraph (2) of the eligibility requirements, which would have required that an eligible applicant have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years or have demonstrated success in significantly increasing student achievement for all groups of students described in that section through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress). We are redesignating the subsequent paragraphs of the eligibility requirements accordingly.

Consistent with the amendments to section 14007(c) of the ARRA, we are revising the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization to specify that an eligible applicant that includes a nonprofit organization is considered to have met paragraph (1) and paragraph (2) (proposed paragraph (3)) of the eligibility requirements for this program if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. In addition, we are revising the Note to specify that an eligible applicant that includes a nonprofit organization is considered to have met paragraph (3) (proposed paragraph (4)) of the eligibility requirements for this program if it demonstrates that it will meet the requirement relating to private-sector matching.

Comment: One commenter recommended that the Department clarify whether low-performing LEAs may partner with high-performing LEAs that meet all the eligibility requirements. This commenter argued that this approach would allow low-performing LEAs that do not meet the requirements to still benefit from funds under this program. The same commenter also suggested that if the lead LEA meets all the requirements, it should not have to select LEA partners that also meet those requirements.

Discussion: High-performing LEAs are permitted to partner with low-performing LEAs in projects under this program.

While an LEA that applies for funds under section 14007(a)(1)(A) of the ARRA must meet the requirements in new section 14007(b)(1) through (3) of the ARRA (which are now reflected in paragraphs (1) through (3) of the eligibility requirements, as discussed elsewhere in this notice), nothing in the statute or the priorities, requirements, definitions, or selection criteria for this program prohibits such an eligible LEA from proposing a project that involves the LEA partnering with other partners, including other LEAs.

In addition, a section 14007(a)(1)(B) partnership could include one or more LEAs, either as an official partner or as an other partner that does not meet the eligibility requirements. This is because the partnership is deemed to have met the eligibility requirements in new section 14007(b)(1) through (3) of the ARRA if the nonprofit organization in the partnership satisfies the requirements in new section 14007(c) of the ARRA.

Changes: None.

Proposed Paragraph (1) of Eligibility Requirements: Significantly Closed Achievement Gaps

Comment: Several commenters recommended that the Department clarify what the phrase “significantly closed the achievement gaps” means in proposed paragraph (1) of the eligibility requirements. Many commenters were particularly interested in clarification of the term “significantly;” many asked for guidance as to how to measure whether an achievement gap was significantly closed. For example, one commenter requested that the Department provide the requisite time period that should be used to measure whether an achievement gap has been closed. Another commenter suggested having flexible indicators for judging whether or not eligible applicants have significantly closed the achievement gaps, such as increases in grade point average, gains in standardized test scores, as well as qualitative measures. One commenter argued that the Department should not interpret the phrase “significantly closed” to mean full achievement gap closure across all grade levels and subject areas, while another commenter argued that eligible applicants who can show success in raising achievement system-wide and moving all students toward proficiency should satisfy this requirement. One commenter recommended that the Department allow an eligible applicant to meet this eligibility requirement through an intermediate variable directly correlated with significantly closing the achievement gaps. The commenter expressed concern that without including such language, the program might exclude eligible applicants with innovative programs for which it has been difficult to directly measure progress in student achievement.

Discussion: Proposed paragraph (1) of the eligibility requirements, which repeats the eligibility requirement in old section 14007(b)(1) of the ARRA (new section 14007(b)(1)(A) of the ARRA), states that to be eligible for an award, an eligible applicant must have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA. The Department declines to define the term “significantly” for purposes of this eligibility requirement. Given the diversity of potential eligible applicants under this program, the Department wishes to encourage eligible applicants to present their arguments for how they have significantly closed the achievement gaps. Similarly, the Department understands that eligible applicants will bring to bear different areas of expertise and that they likely will focus on improving various aspects of student achievement. Eligible applicants are best suited to present information on how they have significantly closed those achievement gaps and to determine the metrics by which they measure those achievements. Because the Department is not identifying the specific measures or variables that an eligible applicant may use to meet this requirement, eligible applicants would not be prohibited from using an intermediate variable strongly correlated with significantly closing the achievement gaps.

Changes: None.

Comment: A number of commenters recommended that the Department clarify whether, to meet this eligibility requirement, an eligible applicant must have significantly closed achievement gaps between all groups described in section 1111(b)(2) of the ESEA, or whether eligible applicants that have significantly closed the achievement gaps between some groups, but not all, would be eligible for an award. One commenter pointed to success in narrowing the achievement gaps between African American and white students, but not across all groups.

Discussion: The Department interprets the eligibility requirement reflected in old section 14007(b)(1) of the ARRA (new section 14007(b)(1)(A) of the ARRA) as concerning the achievement of students in the groups of students in section 1111(b)(2) of the ESEA (i.e., economically disadvantaged students, students from major racial and ethnic groups, students with limited English proficiency, students with disabilities) relative to the achievement of the “all students” category under section 1111(b)(2)(C)(v)(I) of the ESEA. To meet this requirement, therefore, an eligible applicant must have significantly closed the gap in achievement between at least one of those groups and the “all students” category. An eligible applicant is not required to have significantly closed achievement gaps between all of those student groups and the “all students” category, or to have significantly closed achievement gaps between each of the student groups themselves.

Changes: None.

Proposed Paragraph (2) of Eligibility Requirements: Exceeded the State’s Annual Measurable Objectives for Two Years in a Row, or Demonstrated Success in Significantly Increasing Student Achievement for All Groups of Students

Comment: One commenter requested clarification as to how the Department interprets proposed paragraph (2) of the eligibility requirements. The commenter asked the Department to confirm that an eligible applicant would meet this requirement if it satisfied either the “AMO” clause of this requirement (i.e., have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years) or the “another measure” clause (i.e., have demonstrated success in significantly increasing student achievement for all groups of students described in section 1111(b)(2) of the ESEA through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress)).

Discussion: Proposed paragraph (2) of the eligibility requirements, which repeated the eligibility requirement in old section 14007(b)(2) of the ARRA, stated that an eligible applicant must have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years (the “AMO” clause) or have demonstrated success in significantly increasing student achievement for all groups of students described in such section through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress) (the “another measure” clause). As discussed earlier in this notice, section 307 of Division D of the Consolidated Appropriations Act, 2010 amended the ARRA by removing this requirement. As amended, the ARRA now requires that an eligible applicant either (A) have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA (20 U.S.C. 6311(b)(2)), or (B) have demonstrated success in significantly increasing student academic achievement for all groups of students described in such section. We are revising the eligibility requirements to incorporate these statutory changes. Therefore, an eligible applicant can meet this eligibility requirement by showing either (A) or (B) above; it is not required to show that it has done both.

Changes: Consistent with the amendments to section 14007(b) of the ARRA, we are revising proposed paragraph (1) of the eligibility requirements to require that, to be eligible for an award under this program, an eligible applicant must--except as specifically set forth in the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization: (A) have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of ESEA, or (B) have demonstrated success in significantly increasing student academic achievement for all groups of students described in such section. We are also removing proposed paragraph (2) of the eligibility requirements, which would have required that an eligible applicant have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years or have demonstrated success in significantly increasing student achievement for all groups of students described in that section through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress).

Comment: A number of commenters recommended that the Department clarify the meaning of the phrase “success in significantly increasing student achievement” in the “another measure” clause of proposed paragraph (2) of the eligibility requirements. Commenters asked what standard the Department will use to determine whether eligible applicants have met this requirement.

Discussion: As discussed earlier in this notice, the Department declines to define the term “significantly” as it is used in paragraph (1)(A) of the eligibility requirements. Similarly here, the Department declines to define the term “significantly” as it is used in the requirement mentioned by the commenters (which is now incorporated, consistent with the amendments to section 14007(b) of the ARRA, in paragraph (1)(B) of the eligibility requirements). Given the diversity of potential eligible applicants, the Department wishes to encourage eligible applicants to present their arguments for how they have significantly increased student academic achievement. The Department also understands that eligible applicants will bring to bear different areas of expertise and will focus on improving various aspects of student achievement. Eligible applicants are best suited to present information on how they have significantly increased student achievement and to determine the metrics by which they measure those achievements.

Changes: None.

Comment: Several commenters argued that although the “another measure” clause of proposed paragraph (2) of the eligibility requirements mentions NAEP as an example of an appropriate alternative measure for demonstrating success in significantly increasing student achievement, NAEP does not provide information at the LEA level. These commenters requested that the Department provide other examples of acceptable achievement measures that eligible applicants can use under the “another measure” clause to demonstrate success in significantly increasing student achievement, such as graduation rates, Advanced Placement and International Baccalaureate course completion, SAT or PSAT scores, and college enrollment rates.

Two other commenters argued that although NAEP is referenced in section 1111(c)(2) of the ESEA, that section refers to “Other Provisions to Support Teaching and Learning,” not student achievement, which is addressed in section 1111(b) of the ESEA. Those commenters argued that it is, therefore, not appropriate to cite section 1111(c)(2) of the ESEA (i.e., NAEP) as an appropriate measure of student achievement.

Discussion: As discussed earlier in this notice, section 307 of Division D of the Consolidated Appropriations Act, 2010, amended the ARRA by eliminating the requirement set forth in proposed paragraph (2) of the eligibility requirements. As amended, the ARRA now requires that an eligible entity either (A) have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA (20 U.S.C. 6311(b)(2)), or (B) have demonstrated success in significantly increasing student academic achievement for all groups of students described in such section. Under the amendments, the eligibility requirements thus no longer mention NAEP as an example of an appropriate alternative measure for demonstrating significant student achievement. We believe that these statutory changes respond to the commenters’ concerns regarding NAEP.

With respect to the comments requesting other examples of acceptable achievement measures, we decline to incorporate these examples in the eligibility requirements. As discussed earlier in this notice, we believe that eligible applicants are best suited to identify and present information on how they have significantly increased student achievement and do not wish to limit the metrics by which they measure those achievements.

Changes: Consistent with the amendments to section 14007(b) of the ARRA, we are revising proposed paragraph (1) of the eligibility requirements to require that, to be eligible for an award under this program, an eligible applicant must--except as specifically set forth in the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization: (A) have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or (B) have demonstrated success in significantly increasing student academic achievement for all groups of students described in such section. We are also removing proposed paragraph (2) of the eligibility requirements and renumbering the remaining requirements accordingly.

Proposed Paragraph (3) (Newly Redesignated Paragraph (2)) of Eligibility Requirements: Made Significant Improvements in Other Areas

Comment: One commenter recommended that the Department clarify the term “significant improvement” in proposed paragraph (3) of the eligibility requirements.

Discussion: Proposed paragraph (3) (newly redesignated paragraph (2)) of the eligibility requirements, which repeated the eligibility requirement in old section 14007(b)(3) of the ARRA (new section 14007(b)(2)), states that an eligible applicant must have made significant improvement in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and school leaders, as demonstrated with meaningful data. The Department declines to provide a definition of the term “significant improvement” as that term is used in this requirement. The Department wishes to encourage a diverse set of eligible applicants, and believes that eligible applicants are best suited to provide arguments for whether or not their improvements are significant. Eligible applicants are encouraged to present their arguments for how they have made significant improvements in other areas and are not limited in the metrics by which they measure those improvements.

Changes: None.

Proposed Paragraph (4) (Newly Redesignated Paragraph (3)) of Eligibility Requirements: Established Private-sector Partnerships

Comment: Several commenters suggested that the Department clarify proposed paragraph (4) of the eligibility requirements with respect to how many and what types of partnerships are permitted. Specifically, these commenters suggested that the Department clarify whether one or more private-sector partners could provide matching funds or in-kind donations. One commenter suggested that the Department also clarify whether eligible applicants may include private-sector partners that do not provide matching funds or in-kind donations. Another commenter suggested that the Department clarify whether private-sector partners may provide products or services that are used as core components in a project.

Discussion: Proposed paragraph (4) of the eligibility requirements, which repeated the eligibility requirement in old section 14007(b)(4) of the ARRA, stated that eligible applicants must demonstrate they that have established partnerships with the private sector, which may include philanthropic organizations, and that the private sector will provide matching funds in order to help bring results to scale. Section 307 of Division D of the Consolidated Appropriations Act, 2010, amended section 14007(b)(4) of the ARRA to clarify that, to be eligible for an award, an eligible applicant must demonstrate that it has established one or more partnerships with the private sector. We are revising proposed paragraph (4) (newly redesignated paragraph (3)) to incorporate this statutory change. Thus, the revised eligibility requirement makes clear that there are no limits on the number of private-sector partnerships that an eligible applicant may establish.

The statutory requirement likewise does not set any limits on the types of private-sector partnerships that an eligible applicant may establish, except that they must be non-governmental and that, through one or more of these partnerships, the eligible applicant must obtain matching funds from the private sector in order to help bring results to scale. An eligible applicant would not be prohibited under this requirement from establishing partnerships with the private sector for additional purposes.

Changes: Consistent with the amendments to section 14007(b) of the ARRA, we are revising proposed paragraph (4) (newly redesignated paragraph (3)) to clarify that, to be eligible for an award, an eligible applicant must demonstrate that it has established one or more partnerships with the private sector.

Comment: Several commenters asked the Department to clarify the phrase “established partnerships” in proposed paragraph (4) of the eligibility requirements with respect to whether partnerships with the private sector must have previously existed or be ongoing.

Discussion: Proposed paragraph (4) (newly redesignated paragraph (3)) of the eligibility requirements does not require that an eligible applicant utilize preexisting or ongoing partnerships with the private sector. To meet this requirement, an eligible applicant may establish new partnerships or use existing ones.

Changes: None.

Comment: One commenter recommended that the Secretary establish authority to waive the requirement that eligible applicants have established partnerships with the private sector if it can be determined that the lack of such partnerships will not adversely affect the implementation of a project under this program. Other commenters recommended that the Secretary waive this requirement for eligible applicants from rural areas because it will be difficult for these eligible applicants to find private-sector partners to provide matching funds.

Discussion: As noted earlier in this notice, proposed paragraph (4) (newly redesignated paragraph (3)) of the eligibility requirements repeats statutory requirements from the ARRA. The Secretary does not intend to waive these requirements and believes strongly that innovative projects to improve student achievement and attainment should include partnerships with the private sector. However, as discussed in the Cost Sharing or Matching requirement of this program, the Secretary may consider decreasing, in the most exceptional circumstances, on a case by case basis, the amount of matching funds that an eligible applicant must obtain from the private sector to less than the required amount (i.e., 20 percent of its grant award). An eligible applicant that anticipates being unable to meet the 20 percent matching requirement may request that the Secretary reduce the matching level requirement. The request, along with a statement of the basis for the request, must be included in the application.

Changes: None.

Proposed Paragraph (5) (Newly Redesignated Paragraph (4)) of Eligibility Requirements: Providing LEA and School Names

Comment: One commenter offered strong support for proposed paragraph (5) of the eligibility requirements regarding the LEA and school information that a nonprofit organization applicant must include in its application. The commenter asserted that providing nonprofit organizations the option to describe the demographics of the additional LEAs or schools with which they will partner will give eligible applicants that include nonprofit organizations useful flexibility before and after applying for funds under this program. Another commenter suggested that the Department allow an eligible applicant that includes a nonprofit organization not to name any LEA or school partners in its application, but rather only describe the demographics and other characteristics of the LEAs or schools with which the nonprofit organization intends to partner. The commenter argued that this will improve project outcomes by providing eligible applicants that include nonprofit organizations with greater flexibility in the timeline for forging partnerships.

Discussion: Under section 14007(a)(1)(B) of the ARRA, an eligible applicant must be a partnership of the nonprofit organization with (1) one or more LEAs or (2) a consortium of schools. To meet this requirement, an eligible applicant that includes a nonprofit organization must submit an application that identifies each of the official partners in the partnership (i.e., the nonprofit organization and at least one LEA or a consortium of schools). We will not consider an application submitted on behalf of an eligible applicant that includes a nonprofit organization that does not do so. If the eligible applicant intends to involve additional LEAs or schools as additional official partners at a later date or as other partners, it is not required to identify those LEAs or schools in the application.

Changes: None.

Comment: One commenter recommended that the Department clarify the point at which an eligible applicant that includes a nonprofit organization must name any additional LEAs or schools as partners that were not identified in its application. The commenter asked specifically whether or not additional LEAs or schools must be named before a grant award is made. Another commenter recommended that the Department not allow eligible applicants that include a nonprofit organization to identify additional LEAs or schools as partners after a grant has been awarded. The commenter argued that all partners in a grant should be involved from the outset of the grant, and that LEA eligible applicants are being held to a different standard than eligible applicants that include a nonprofit and are applying under section 14007(a)(1)(B) of the ARRA because they are not afforded this same flexibility with respect to naming partners.

Discussion: Under proposed paragraph (5) (newly designated paragraph (4)), we proposed to permit an eligible applicant that includes a nonprofit organization to describe the demographics and other characteristics of any additional LEAs or schools with which it intends to partner (apart from the official and other partners that it names in its application) and the process it will use to select them because we recognize that this type of eligible applicant may need additional time to make official arrangements with all of its partners beyond the date by which applications must be submitted under this program. However, as stated in the NPP, an eligible applicant that includes a nonprofit organization must identify all of its partners (including other partners) before a grant award is made; it may not identify additional partners after this date. We agree with the commenter that all partners in a grant should be involved from the outset of the grant. We do not believe that allowing nonprofit organization applicants to name additional partners prior to receiving a grant award holds LEA applicants to a more stringent standard than eligible applicants that include nonprofit organizations. As noted in the preceding discussion, an eligible applicant that includes a nonprofit organization must still demonstrate that it has met the requirements for eligible applicants under this program and this requires that the application identify at least one LEA or a consortium of schools as an official partner; we will not consider an application on behalf of an eligible applicant that includes a nonprofit organization that does not do so.

Changes: None.

Comment: One commenter suggested that the Department require eligible applicants that include a nonprofit organization to describe the demographics of all partner LEAs or schools in order to better determine and ensure equity among grant recipients in terms of students or populations served.

Discussion: Although an eligible applicant that includes a nonprofit organization would not be prohibited from describing the demographics of the LEAs or schools with which it partners and names in its application, we do not believe it is necessary to require them to do so because we do not intend to use equity as a selection criterion in making grant awards under this program. We also note that if an eligible applicant that includes a nonprofit organization intends to partner with additional LEAs or schools that are not named in the application, it must describe in the application the demographics and other characteristics of these LEAs and schools and the process it will use to select them as either official or other partners.

Changes: None.

Comment: One commenter recommended that the Department require that eligible applicants that include a nonprofit organization specify the proposed conditions of the partnership agreement, including the roles and responsibilities that each partner will have, in the grant application. The commenter noted that the agreement should include conditions for autonomy for the nonprofit organization and specify the degree to which each partner will have control over the budget and program generally.

Discussion: Consistent with the Department’s regulations governing group applications in 34 CFR 75.128, a partnership applicant under this program must enter into an agreement that details the activities that each member of the partnership plans to perform. We do not believe it is necessary, however, to require that these agreements be included as part of the applications. Further, we do not believe it is appropriate for the Department to specify the level of autonomy or control over projects under this program that partners may have; rather, we believe that eligible applicants should have the flexibility to determine the conditions of their partnerships on an individual basis provided that those conditions comply with these requirements.

We do note, however, that under Selection Criterion G (Quality of the Management Plan and Personnel), the Secretary will consider the adequacy of the eligible applicant’s management plan, including clearly defined responsibilities, timelines and milestones for accomplishing project tasks. In responding to this selection criterion, the eligible applicant is encouraged to describe the roles and responsibilities of its partners so that the Secretary can appropriately evaluate the eligible applicant’s management plan.

Changes: None.

Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization

Comment: One commenter recommended that the Department clarify what is meant by the sentence in the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization stating that the eligible entity shall be considered to have met the requirements of proposed paragraphs (1), (2), and (3) of the eligibility requirements if the nonprofit organization has a record of meeting those requirements. The commenter argued that this sentence might exempt certain eligible applicants from complying with some of the eligibility requirements.

Discussion: As discussed earlier in this notice, the eligibility requirements that were reflected in proposed paragraphs (1), (2), and (3) of the eligibility requirements tracked the statutory requirements from old section 14007(b)(1) through (b)(3) of the ARRA. Those requirements have been amended and consolidated into section 14007(b)(1) and (b)(2) of the ARRA. In addition, section 14007(c) of the ARRA has been amended to specify that an eligible entity that includes a nonprofit organization is considered to have met the requirements of sections 14007(b)(1) and (b)(2) of the ARRA (as amended) if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. Under the amendments to section 14007(c) of the ARRA, an eligible entity that includes a nonprofit organization is thus no longer required to demonstrate that the nonprofit organization has a record of meeting proposed paragraphs (1), (2), and (3) of the eligibility requirements. Instead, the eligible applicant is required to demonstrate that the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. We are revising the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization to incorporate these statutory changes regarding the eligibility of an eligible applicant that includes a nonprofit organization.

Changes: As discussed earlier in this notice, consistent with the amendments to section 14007(c) of the ARRA, we are revising the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization to specify that an eligible applicant that includes a nonprofit organization is considered to have met paragraph (1) and paragraph (2) of the eligibility requirements for this program if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention.

Comment: One commenter recommended that the Department clarify whether a nonprofit organization that cannot meet the eligibility requirements discussed in the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization may partner with an LEA or a consortium of schools that meets those requirements. A number of commenters requested that, if a nonprofit organization may partner with a consortium of schools that meets these requirements, the Department clarify whether all schools in the consortium must meet the requirements.

Discussion: As discussed earlier in this notice, section 14007(c) of the ARRA has been amended to specify that an eligible applicant that includes a nonprofit organization is considered to have met the requirements of paragraphs (1) and (2) of section 14007(b) of the ARRA (as amended) if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. We are revising the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization to incorporate these statutory changes. Thus, any eligible applicant that includes a nonprofit organization must demonstrate that the nonprofit organization in the partnership has a record of significantly improving student achievement, attainment, or retention. Accordingly, an eligible applicant that includes a nonprofit organization that cannot demonstrate that the nonprofit organization in the partnership has a record of significantly improving student achievement, attainment, or retention is not eligible for an award under this program (regardless of whether the LEA(s) or schools with which the nonprofit organization partners meet the requirements of paragraph (1) and paragraph (2) (proposed paragraph (3)) of the eligibility requirements for this program). However, under this program, an LEA may apply on its own as an eligible applicant consistent with section 14007(a)(1)(A) of the ARRA, and may partner with other entities, including nonprofit organizations, as other partners. In that respect, an LEA applying under section 14007(a)(1)(A) of the ARRA that meets the requirements of paragraph (1) and paragraph (2) (proposed paragraph (3)) of the eligibility requirements for this program may involve entities (including nonprofit organizations) that do not meet the applicable eligibility requirements for this program without limitation, except as otherwise proscribed by law.

Changes: None.

Comment: One commenter recommended that the Department consider an eligible applicant that includes a nonprofit organization to have met the requirements of proposed paragraphs (1), (2), and (3) of the eligibility requirements based on the nonprofit organization’s record of work with one LEA, instead of more than one LEA.

Discussion: We originally proposed in the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization that the eligible applicant must demonstrate that the nonprofit organization has a record of meeting the requirements of proposed paragraphs (1), (2), and (3) of the eligibility requirements through its work with an LEA. We are replacing this provision with a requirement that the nonprofit organization serving as an official partner have a record of significantly improving student achievement, attainment, or retention, consistent with the amendments to the authorizing statute for this program through its work with an LEA or schools. Thus, there is no requirement that an eligible applicant that includes a nonprofit organization demonstrate that the nonprofit organization serving as an official partner has a record of significantly improving student achievement, attainment, or retention through its work with more than one LEA.

Changes: We are revising the Note to specify that, to meet this requirement, an eligible applicant that includes a nonprofit organization must demonstrate that it has a record of significantly improving student academic achievement, attainment, or retention through the assistance it has provided to an LEA or schools in the past; we are making conforming changes to Selection Criterion C (Experience of the Eligible Applicant) for all three types of grants.

Comment: Two commenters requested that the Department amend the Note About Eligibility for an Eligible Applicant that Includes a Nonprofit Organization to provide that an eligible entity that includes a nonprofit organization may demonstrate that the nonprofit organization serving as an official partner has a record of meeting the requirements in proposed paragraphs (1), (2), and (3) of the eligibility requirements through its record of work with an LEA or a consortium of schools--rather than only though its record of work with an LEA. The commenters argued that this change would ensure that nonprofit organizations that have not worked with an entire LEA would be eligible if they can meet the requirements in proposed paragraphs (1), (2), and (3) based on their previous work with schools.

Discussion: We agree with the commenters that, to meet the requirement that it have a record of significantly improving student achievement, attainment, or retention (which replaces, for eligible applicants that include a nonprofit organization, the requirements of proposed paragraphs (1), (2), and (3) of the eligibility requirements), a nonprofit organization should not be limited only to its record of work with an LEA. We are revising the Note to specify that, to meet this requirement, an eligible applicant that includes a nonprofit organization must provide the nonprofit organization’s record of work with an LEA or schools; we are making conforming changes to Selection Criterion C (Experience of the Applicant) for all three types of grants. Thus, an eligible applicant that includes a nonprofit organization may provide the nonprofit organization’s record of work with schools. However, because we believe that the nature of this program and the scope of its goals require that nonprofit organizations serving as an official partner have broad experience, such a nonprofit organization may not provide its record of work with only a single school in order to meet this requirement.

Changes: We are revising the Note to specify that, to meet this requirement, an eligible applicant that includes a nonprofit organization must demonstrate that it has a record of significantly improving student academic achievement, attainment, or retention through the assistance it has provided to an LEA or schools in the past; we are making conforming changes to Selection Criterion C (Experience of the Eligible Applicant) for all three types of grant.

Additional Eligibility Requirements

Comment: One commenter recommended that the Department add an eligibility requirement that would require eligible applicants to have significantly closed achievement gaps between genders.

Discussion: We decline to require eligible applicants to have significantly closed the achievement gap between genders in order to be eligible for funding under this program. While gender equity in education is a laudable goal that the Department supports, we do not believe it is necessary to add such a requirement because the authorizing statute requires eligible applicants only to have significantly closed achievement gaps specifically between the groups of students described in section 1111(b)(2) of the ESEA, which do not include student gender.

Changes: None.

Comment: One commenter suggested that the Department add an eligibility requirement that would require eligible applicants to have significantly closed graduation rate gaps between the designated groups of students described in section 1111(b)(2) of the ESEA.

Discussion: The eligibility requirement reflected in old section 14007(b)(3) of the ARRA (now section 14007(b)(2) of the ARRA) requires eligible applicants to make significant improvements in other areas, and specifically mentions improving graduation rates as an area of improvement that would meet the requirement. We believe that this requirement, which is now reflected in paragraph (2) of the eligibility requirements, provides an appropriate amount of focus on the need to improve high school graduation rates. We, therefore, decline to make the change recommended by the commenter.

Changes: None.

Comment: One commenter suggested that the Department add an eligibility requirement that would require eligible applicants to provide documentation that relevant student achievement data will be readily available and accessible for progress monitoring purposes.

Discussion: We do not believe it is appropriate to include the eligibility requirement suggested by the commenter because it could unnecessarily constrain the types of projects eligible applicants may submit for the different types of grants under this program. We note, however, that under Selection Criterion D (Quality of the Project Evaluation) for each type of grant, the Secretary will consider the extent to which the methods of project evaluation will provide high-quality implementation data and performance feedback, and permit periodic assessment of progress toward achieving intended outcomes.

Changes: None.

Comment: One commenter recommended that the Department add an eligibility requirement that would require eligible applicants to ensure that their project, by design or outcome, does not exacerbate the concentration of poverty or the racial or linguistic concentration of students.

Discussion: As discussed earlier in this notice, the Department believes that the promotion of diverse student populations is a laudable goal. We do not, however, believe that an eligibility requirement of the type recommended by the commenter is appropriate for this program. Consistent with the ARRA, we seek to ensure that the primary focus of this program is improving student academic achievement and attainment. That said, in discussing the effects of its proposed project an eligible applicant may include discussion of the effects of the project on intermediate variables that are strongly correlated with improving student achievement and attainment outcomes. These intermediate variables may include variables on topics such as those the commenter mentions.

We also note that the Department has for many years administered the Magnet Schools Assistance Program. This program provides grants to LEAs to fund magnet schools that--in addition to strengthening students’ academic knowledge and their attainment of tangible and marketable skills--will further the elimination, reduction or prevention of minority group isolation in elementary and secondary schools. 20 U.S.C. 7231(b).

Changes: None.

Funding Categories

Comment: Although one commenter supported the requirement that an applicant be considered for an award only for the type of grant for which it applies, a few commenters noted that an applicant may have difficulty determining the grant type under which its proposed project falls and recommended that the Department allow applicants to submit the same proposed project under more than one grant category. A few other commenters recommended that the Department allow reviewers to move an application between grant categories or allow an application that does not meet the level of evidence for one category of grant to be considered in another category.

Discussion: We decline to accept the commenters’ recommendations because we do not believe it is appropriate for the Department or its reviewers to determine the grant category for a proposed project; rather, eligible applicants should bear the responsibility for determining which grant type most closely matches their capabilities and needs. Applicants may submit as many applications as they deem appropriate--bearing in mind that the grant categories are different and, therefore, a project proposed under one category would not meet the requirements of another category. This is the reason the Department does not believe it makes sense to permit applicants to submit the same project under multiple categories.

Changes: None.

Cost Sharing or Matching

Comment: While several commenters supported the eligibility requirement that eligible applicants demonstrate that they have established one or more partnerships with an entity or organization in the private sector (proposed paragraph (4) of the eligibility requirements), many commenters disagreed with the proposed requirement that an eligible applicant obtain private-sector matching funds or in-kind donations equal to at least 20 percent of its grant award. These commenters recommended that the 20 percent private-sector matching funds requirement be eliminated or reduced. Commenters cited several reasons for eliminating or reducing the required match, including: the possible lack of available resources from the private sector, due to current economic conditions or other reasons; the possibility that the size of the match will discourage many small LEAs and nonprofit organizations from applying; and the possible unintended consequence of giving unfair advantage to entities that already have access to or relationships with private-sector organizations. Two commenters suggested that the Department use a sliding scale in which the amount of matching funds would be higher for the Development and Validation grant categories and lower for Scale-up grants.

Several other commenters encouraged the Department to allow an eligible applicant’s current financial commitments, including existing philanthropic donations, to be reallocated and used to meet the Cost Sharing or Matching requirement and not require eligible applicants to raise new funds. A few commenters recommended that the Department allow private-sector funds that support the entirety of an eligible applicant’s organizational efforts, not solely or specifically the eligible applicant’s proposed project, to be counted toward the 20 percent private-sector match. Similarly, one commenter recommended giving a grantee flexibility to use matching funds for more general programmatic costs that are not necessarily tied to its project.

Discussion: As discussed in the context of the eligibility requirements for this program, old section 14007(b)(4) of the ARRA (new section 14007(b)(3) of the ARRA) requires an eligible applicant to demonstrate that it has established one or more partnerships with the private sector and that the private sector will provide matching funds in order to help bring results to scale. The purpose of the Cost Sharing or Matching requirement is to help ensure that the results of the funded projects will be brought to scale and sustained. The Department’s decision that eligible applicants for all three grant types--Scale-up, Validation, and Development grants--demonstrate a private-sector match of at least 20 percent of the total amount of Federal funds requested for each proposed project is based on the belief that this amount of private support is a strong indicator of the potential for sustainability of the proposed project over time. However, the Department understands the concerns raised by these commenters and, in response, provides the following information and clarifications.

First, in-kind contributions may be counted towards the 20 percent private sector matching requirement.

Second, the Secretary will consider granting waivers of the matching requirement in the most exceptional circumstances.

Third, the Department has reviewed data on private giving in K-12 education over the past several years and has concluded that the private sector has the capacity and resources to fulfill this matching requirement. Data from the Foundation Center (2007, the most recent year for which data are available) indicate that asking the private sector to provide $130 million (i.e., 20 percent of the $650 million appropriated for this program in fiscal year 2009) over five years will amount to less than five percent of total K-12 giving from the private sector over that period of time. We believe that this reasonably demonstrates availability of private sector resources to fulfill the matching requirement.

Fourth, eligible applicants may count existing private sector support towards the required match so long as these funds are reallocated in support of the project for which the eligible applicant seeks funding and the eligible applicant can provide appropriate evidence of this commitment.

And lastly, as discussed later in this notice, the Department is changing the time by which eligible applicants must demonstrate that they have fulfilled their matching requirement. Specifically, rather than secure this match at the time of application, an eligible applicant is not required to demonstrate that it has secured the match until so requested by the Department after its application has been reviewed and scored at the top of the rank-order list for the respective types of grants. This means that not all eligible applicants will be required to secure a match, and that those required to do so will not have to secure that match until after the peer review of applications.

Based upon this information and considerations, we do not believe it is necessary to reduce, eliminate, or further modify the 20 percent matching requirement.

Changes: None.

Comment: A few commenters expressed concern about the time period in which eligible applicants would need to secure and provide evidence of the commitment of the 20 percent private-sector matching funds. Commenters noted the hesitancy of the private sector to commit matching funds for multiple applications before knowing how many applications will be funded. One commenter suggested allowing eligible applicants 150 days after being approved for funding to secure the 20 percent private-sector match. Another commenter suggested allowing up to 10 percent of the required match to be obtained within one year of the award. Another commenter suggested that the Department work with private foundations to include a tiered review process to minimize the number of requests private foundations may have to review before a Federal grant is awarded.

Discussion: As noted earlier, the Department is committed to requiring eligible applicants to obtain a 20 percent private-sector match to be eligible to receive funds under this program but is making some modifications to this requirement that address the concerns raised by these commenters. We are revising the Cost Sharing or Matching requirement with respect to the timing of submission of the evidence of the private-sector match. Selected eligible applicants are now required to submit evidence of the full 20 percent private-sector matching funds following the peer review of applications--not at the time of application as was initially proposed by the Department. An award will not be made unless the applicant provides adequate evidence that the full 20 percent private-sector match has been committed or the Secretary approves the eligible applicant’s request to reduce the matching-level requirement.

Eligible applicants that score at the top of the rank-order list for the respective types of grant and thus are being most seriously considered for funding will be contacted and given a limited period of time, approximately four to six weeks, to provide evidence of the private-sector match. Given that applications will be submitted in the spring, we expect that there will be adequate time between the completion of the peer review process and the final deadline for awarding funds under this program to allow for this additional step in the grant process.

Changes: We are revising the Cost Sharing or Matching requirement with respect to the timing of submission of the evidence of the private-sector match. Selected eligible applicants are now required to submit evidence of the full 20 percent private-sector matching funds following the peer review of applications. An award will not be made unless the applicant provides adequate evidence that the full 20 percent private-sector match has been committed or the Secretary approves the eligible applicant’s request to reduce the matching-level requirement.

Comment: Several commenters sought clarification of the types of funding sources that may be used to satisfy the Cost Sharing or Matching requirement, including clarification regarding in-kind donations. Some commenters suggested that the Department clarify whether in-kind donations may include discounts off products and services that are components of the innovation to be scaled up and that are provided by private-sector partners. One commenter recommended that resources from Federal programs be counted as part of the match. One commenter recommended that LEAs be allowed to reallocate their own funds to meet the matching requirement.

Discussion: Section 14007(b)(3) of the ARRA specifically requires a private-sector match for this program. Thus, an eligible applicant may not use funding from other Federal programs or other public sources (including the LEAs’ own funds) to satisfy the Cost Sharing or Matching requirement.

Discounts off products and services that are components of the innovation to be scaled up could be considered in-kind donations that count toward the Cost Sharing and Matching requirement. Eligible applicants should review the Department’s regulations on matching funds, including in-kind contributions, in 34 CFR 74.23 and 80.24 for further clarification on requirements pertaining to in-kind donations.

Changes: None.

Comment: Several commenters sought clarification about the conditions that would constitute the “most exceptional circumstances” under which the Secretary might consider reducing the 20 percent private-sector match under the Cost Sharing or Matching requirement.

Discussion: The Department understands that there may be extenuating circumstances that will create challenges for some eligible applicants in securing a commitment from the private sector for the full 20 percent private-sector match. For this reason, we included in the NPP and are retaining in this notice a provision in the Cost Sharing or Matching requirement that allows an eligible applicant that believes it will be unable to obtain the full 20 percent private-sector match to include in its application a request to the Secretary to decrease the private-sector match amount. The Secretary will grant waivers on a case-by-case basis. As the Secretary’s decision to decrease the private-sector match amount will depend on the individual facts presented in an eligible applicant’s request, we decline to describe what situations might or might not be considered “the most exceptional circumstances” warranting the grant of a waiver.

Changes: None.

Comment: Three commenters recommended that the Department clarify whether the Cost Sharing or Matching requirement applies only to eligible applicants for Scale-up grants, and not to eligible applicants for Validation or Development grants. The commenters noted that the purpose of the Cost Sharing or Matching requirement, as stated in the NPP, is to help bring results to scale.

Discussion: The Cost Sharing or Matching requirement applies to all eligible applicants under this program, not just to applicants for Scale-up grants.

Changes: None.

Comment: One commenter expressed concern that the Cost Sharing or Matching requirement does not support the goal of sustainability because a matching requirement that lasts only as long as the life of the grant does not sustain meaningful reform. The commenter recommended that the Department require applicants to describe the administrative and other efforts and activities the eligible applicant will pursue in order to raise additional funds to sustain the project.

Discussion: The Department believes that the requirement that matching funds be from the private sector increases the likelihood that projects will be able to be sustained beyond the grant period. Although the Department may not require eligible applicants to obtain matching funds from the private sector for activities after the grant period, peer reviewers will consider an eligible applicant’s plans to sustain its proposed project after the grant period, consistent with the selection criteria related to strategy and capacity to bring to scale and sustainability (Selection Criteria E (Strategy and Capacity to Bring to Scale (in the case of Scale-up and Validation grants); Strategy and Capacity to Further Develop and Bring to Scale (in the case of Development grants)) and F (Sustainability)).

Changes: None.

Comment: One commenter recommended that the Department require eligible applicants to notify their State educational agency if they submit an application under this program. The commenter argued that this would provide the State educational agency with the ability to leverage these grants by scaling them up with State or local funds.

Discussion: This program is subject to Executive Order 12372 and 34 CFR part 79, which allows States that have chosen to participate in Intergovernmental Review the opportunity to review and comment on applications submitted to the Department for funding.  We do not believe it is necessary to separately require an eligible applicant to notify its State educational agency that it has submitted an application for a grant under this program.

However, eligible applicants should consider including State educational agencies as other partners and leveraging available State and local funds to increase the reach and sustainability of proposed projects. As noted in the preceding discussion, peer reviewers will consider, in general, the reach and sustainability of a proposed project under this program consistent with the selection criteria related to strategy and capacity to bring to scale and sustainability (Selection Criteria E (Strategy and Capacity to Bring to Scale (in the case of Scale-up and Validation grants); Strategy and Capacity to Further Develop and Bring to Scale (in the case of Development grants)) and F (Sustainability)). Applicants may not include State and local funds in their cost sharing and cost matching calculation.

Changes: None.

Evaluation

Note: For an analysis of comments and changes relating to the proposed evaluation requirements, please see the Evidence and Evaluation section elsewhere in this notice.

Participation in Communities of Practice

Comment: One commenter supported the requirement that all grantees participate in communities of practice. Another commenter recommended that the Department expand the requirement to include participation in knowledge and innovation networks established by the Department. Under the commenter’s expanded model, grantees would be required to participate not only in communities of practice but also in the development and implementation of new networking opportunities. Finally, two commenters suggested that the Department use intermediary organizations to organize and facilitate the communities of practice among grantees.

Discussion: All grantees under this program are required to participate in communities of practice throughout the grant period. How those communities of practice will be organized, who will facilitate them, and the extent to which grantees will participate in networks such as those recommended by the commenter will be determined by the Department at a later date. The expectation is that grantees will have the opportunity to provide input on the structure and activities of the communities of practice and help shape them as a mechanism to serve grantees and inform the Department about what they have learned.

Changes: None.

Comment: One commenter recommended that the Department require grantees to make all outputs produced through grants under this program freely available in order to maximize the program’s reach.

Discussion: At this time, the Department is only requiring grantees to make the results of their evaluations transparent to the public. We are not specifying how grantees must disseminate these results because we believe that grantees are best positioned to determine the methods of dissemination that are most appropriate for their organizations.

It should be noted, however, that the Department has regulations related to products produced with grant funds. Specifically, under 34 CFR 75.621, grantees may copyright intellectual property produced with Department grant funds. However, under 34 CFR 74.36 and 80.34, the Department retains a non-exclusive and irrevocable license to reproduce, publish, or otherwise use those project materials for government purposes. This gives the Department the authority needed to ensure that materials produced in these grants can be made available to the public.

Changes: None.

DEFINITIONS

Definitions Related to Evidence

Note: For an analysis of comments and changes regarding the proposed Definitions Related to Evidence, please see the Evidence and Evaluation section elsewhere in this notice.

Other Definitions

Note: We provide analyses of comments and changes regarding the proposed definitions of highly effective school leader, highly effective teacher, persistently low-performing schools, and rural LEA in the PRIORITIES section earlier in this preamble. We discuss comments and changes regarding other definitions in the proposed Other Definitions in the paragraphs that follow.

Formative Assessment

Comment: None.

Discussion: As we indicated in footnote 9 of the NPP, we use for this program many of the same terms that are used and defined in the Race to the Top Fund and other programs supported with ARRA funds. We further stated in the NPP that we would align the definitions for those terms, as appropriate, with those used in the Race to the Top Fund program. Accordingly, we are making minor changes to the definition of the term formative assessment for consistency with the definition of this term in the Race to the Top Fund program (see 74 FR 59804).

Changes: We are revising the definition of formative assessment to mean assessment questions, tools, and processes that are embedded in instruction and are used by teachers and students to provide timely feedback for purposes of adjusting instruction to improve learning.

Interim Assessment

Comment: One commenter recommended that the Department include, in the definition of interim assessment, student report card scores provided that the scores are assigned relative to specified standards.

Discussion: We do not believe it is appropriate to include student report card scores in the definition of interim assessment because these scores are reporting tools, not assessments. Assessments that are used in producing such scores, however, may meet the definition of interim assessment to the extent they evaluate knowledge and skills relative to a specific set of academic standards.

Changes: None.

High-Need Student

Comment: While a number of commenters supported the proposed definition of the term high-need student, several commenters recommended that the Department modify the definition to include the following types of students: gifted and talented students, students who are pregnant or parenting, students who have been held in a juvenile detention facility; students meeting only minimum standards; students who are high-achieving but live in high-risk communities; American Indian, Alaska Native, and Native Hawaiian students; students whose parents have not graduated from college; students who are racially isolated; and students who demonstrate adverse patterns of behavior, attendance, discipline, or other non-academic outcomes that impede overall success.

Discussion: The Department understands the interest of the commenters in expanding the definition of high-need student to include other categories of students at risk of educational failure or otherwise in need of special assistance and support. While the proposed definition provided examples of these types of students, those examples are not intended to be an exclusive list. Eligible applicants may include other types of students they consider to be high-need as students to be served by their proposed projects.

As noted elsewhere in this notice, in cases where this program defines a term that is used and defined in other programs supported with ARRA funds, we intend to use the same definitions. For consistency with the definition of high-need student used in the Race to the Top Fund program, we are making a minor change in the definition of the term for this program by including students who attend high minority schools (as defined by the State in which the students attend school) as an additional example.

Changes: We are revising the definition of the term high-need student by adding to the list of students who are at risk of educational failure students who attend high-minority schools (as defined by the student’s State).

Regional Level

Comment: Three commenters asked for greater clarity and specificity regarding the definition of the term regional level. Specifically, the commenters sought clarification on the following issues: what constitutes a regional level project; whether a regional level project must be implemented in more than one LEA; and whether a project that serves multiple regions of a single, large, urban LEA would qualify as a regional level project.

Discussion: The proposed definition of regional level, as used in connection with Scale-up and Validation grants, describes projects that are able to serve a variety of communities and student populations within a State or multiple States, including rural and urban areas. We are revising the definition of regional level to clarify that, to meet the definition, a project must serve students in more than one LEA, excluding a project implemented in a State in which the State educational agency is the sole educational agency for all schools and thus may be considered an LEA under section 9101(26) of the ESEA. Thus, a project that is implemented in a single LEA (if not the sole educational agency for all schools in a State) would not be considered a regional level project consistent with the definition of regional level used in this program.

Changes: We are revising the definition of regional level to clarify that, to meet the definition, a project must serve students in more than one LEA, excluding a project implemented in a State in which the State educational agency is the sole educational agency for all schools and thus may be considered an LEA under section 9101(26) of the ESEA.

Student Achievement

Comment: Several commenters recommended that the Department revise the definition of student achievement to clarify that student achievement can be determined using multiple measures. These commenters recommended that we revise the definition to include additional measures such as the following: grades; end-of-course exams; rates at which students are on track to graduate from high school or meet learning objectives; Advanced Placement exams; college readiness measures or tests; career readiness measures such as technical skill attainment and work-place readiness assessments; formative assessments; interim assessments if aligned to end-of-course exams or LEA pacing guides; online reading comprehension measures; assessments of student writing, presentations, performances, projects, portfolios, and group work.

Discussion: The Department agrees with the commenters about the need for multiple ways in which to measure student achievement. We did not intend for the proposed definition of student achievement to preclude the use of multiple measures including those recommended by the commenters provided that, for the tested grades and subjects, the measures include student performance on State assessments. That said, to ensure consistency in definitions of terms across programs supported with ARRA funds, we are revising the definition of student achievement used in this program. The revised definition retains the flexibility for eligible applicants to use multiple measures of student achievement but also requires that the measures used be rigorous and comparable across classrooms.

Changes: We are revising the definition of student achievement to mean--

(a) For tested grades and subjects: (1) a student’s score on the State’s assessments under section 1111(b)(3) of the ESEA; and, as appropriate, (2) other measures of student learning, such as those described in paragraph (b) of this definition, provided they are rigorous and comparable across classrooms; and

(b) For non-tested grades and subjects: alternative measures of student learning and performance such as student scores on pre-tests and end-of-course tests; student performance on English language proficiency assessments; and other measures of student achievement that are rigorous and comparable across classrooms.

Comment: Several commenters recommended that the Department revise the definition of student achievement to include data on student achievement in non-tested grades and subjects including the arts.

Discussion: The definition of student achievement under this program would not preclude the use of data on student achievement in non-tested grades and subjects; in fact, paragraph (b) of the definition requires the use of such data.

Changes: None.

Comment: A few commenters recommended that the Department revise the definition of student achievement to include measures for early learning such as school readiness assessments. A few other commenters recommended that the Department include nonacademic measures such as measures of student attendance and engagement.

Discussion: Within the definition of student achievement, we intend to include only measures relating directly to student academic performance in the elementary and secondary grades and subjects. We note, however, that, consistent with the selection criterion regarding Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect), eligible applicants may also demonstrate the success of their proposed projects using intermediate variables that are strongly correlated with improving student achievement and attainment outcomes. These variables may include school readiness and nonacademic measures such as those recommended by the commenters.

Changes: None.

Comment: Two commenters recommended that the Department expand the definition of student achievement to include measures regarding postsecondary education, namely, rates at which students enroll in an institution of higher education (including two-and four year colleges and trade and vocational schools) and complete one year’s worth of college credit within two years.

Discussion: As outlined in the preceding discussion, within the definition of student achievement, we intend to include only measures relating directly to student academic performance in the elementary and secondary grades and subjects. However, we agree with the commenters that it is important to recognize and support projects under this program that improve college enrollment and completion rates. We are revising Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect) to include college enrollment and completion rates among the student achievement and attainment outcomes for which the Secretary will consider the effect of a proposed project.

Changes: We are revising Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect) to include college enrollment and completion rates among the student achievement and attainment outcomes for which the Secretary will consider the effect of a proposed project.

Comment: One commenter recommended that the Department clarify that paragraph (b) of the definition of student achievement refers to STEM-related academic subjects, thereby eliminating any confusion over the provision’s application to all academic subjects.

Discussion: The definition of student achievement does not limit the non-tested subjects to STEM-related subjects and includes any non-tested academic subject. We note also that science is a tested subject--States are required to administer assessments in science under the ESEA.

Changes: None.

Student Growth

Comment: One commenter suggested that the Department revise the definition of student growth to specify that student growth data must be based on criterion-referenced growth measures rather than norm-referenced measures.

Discussion: We do not believe it is appropriate to require that student growth data be based on a specific growth measure because to do so would effectively prevent eligible applicants in certain States from using data from the assessments their States administer pursuant to section 1111(b)(3) of the ESEA.

Changes: None.

Comment: Several commenters recommended that the Department revise the definition of student growth so that the term would cover change in other areas, not only student achievement. Some of the other areas mentioned in the comments include: student behavior, social and emotional skills, collaborative skills, ethical decision-making skills, problem solving skills, civic skills, physical skills, and technical skills.

Discussion: Within the definition of student growth, we intend to include only measures of change in student achievement (as that term is defined in this program). We note, however, that, consistent with Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect), eligible applicants may also demonstrate the success of their proposed projects using intermediate variables that are strongly correlated with improving student achievement and attainment outcomes. These variables may include measures on topics such as those discussed by the commenters.

Changes: None.

Comment: Several commenters recommended that the Department revise the definition of student growth to include growth with respect to improved performance on student portfolios and other performance measures.

Discussion: Under this program, an eligible applicant would be permitted to use student growth as measured by student portfolios and other performance measures to the extent these measures meet the requirements for measures of student achievement (in particular, the requirement that the measures are rigorous and comparable across classrooms) included in the definition of student achievement and to the extent that the approach used to determine growth on these measures is statistically rigorous.

Changes: None.

Additional Definitions

Comment: A few commenters suggested that the Department provide a definition of the term “innovation” as it is used in this program. The commenters expressed concern that, without such a definition, the program would not sufficiently promote innovation in the projects that are supported.

Discussion: Although we appreciate the commenters’ concerns, we do not believe that including a definition of “innovation” is necessary. Rather, we believe that the innovativeness of proposed projects should be determined through the review of applications using the selection criteria for this program. We have designed the selection criteria for the respective types of grants particularly Selection Criterion A (Need for the Project and Quality of Project Design) and Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect) in a way that identifies the aspects of a proposed project that would make it innovative. We believe these criteria are sufficient to ensure that only innovative projects receive funding under this program.

Changes: None.

Comment: Two commenters recommended that the Department provide a definition of the term “high school graduation rate” for purposes of this program. The commenters recommended that the Department require eligible applicants to use a uniform graduation rate and suggested using either the Averaged Freshman Graduation Rate or standards that meet or exceed those set forth in the Department’s regulations.

Discussion: We agree with the commenters that a definition of “high school graduation rate” is warranted for this program. Therefore, we are adding a definition of the term that is consistent with the Department’s regulations in 34 CFR 200.19. To satisfy this definition of high school graduation rate, an eligible applicant must use a four-year adjusted cohort graduation rate consistent with 34 CFR 200.19(b)(1) and may also use an extended-year adjusted cohort graduation rate consistent with 34 CFR 200.19(b)(1)(v) if the State in which the proposed project is implemented has been approved by the Secretary to implement such a rate.

Changes: We are adding a definition of high school graduation rate. As defined in this notice, the term means a four-year adjusted cohort graduation rate consistent with 34 CFR 200.19(b)(1) and may also include an extended-year adjusted cohort graduation rate consistent with 34 CFR 200.19(b)(1)(v) if the State in which the proposed project is implemented has been approved by the Secretary to implement such a rate.

Comment: Two commenters recommended the Department provide a definition of the term “regular high school diploma” for purposes of this program. The commenters recommended that the definition include diplomas awarded by accredited institutions operating within a State that enable students to progress to postsecondary education, but that may not be entirely aligned with State academic content standards.

Discussion: We agree with the commenters that a definition of “regular high school diploma” is warranted for this program. However, it is the intent of the Department to support projects under this program that enable students to obtain diplomas that are fully aligned with State academic content standards. We, therefore, are adding the definition of “regular high school diploma” established in the Department’s Title I regulations (at 34 CFR 200.19(b)(1)(iv)) to accomplish this. An alternative degree that is not fully aligned with the State's academic content standards, such as a GED credential, is excluded under this definition.

Changes: We are adding a definition of regular high school diploma. As defined in this notice, this term means, consistent with 34 CFR 200.19(b)(1)(iv), the standard high school diploma that is awarded to students in the State and that is fully aligned with the State’s academic content standards or a higher diploma and does not include a GED credential, certificate of attendance, or any alternative award.

Comment: One commenter recommended that the Department provide a definition for the term “dropout rate” for purposes of this program. In particular, the commenter requested that the Department clarify whether students who move from the area or transfer to another school, LEA, or State should be considered dropouts.

Discussion: Unlike for high school graduation rates, there are no Federal requirements for determining dropout rates. We recognize that there are a variety of ways to calculate dropout rates, and do not wish to limit eligible applicants in how they calculate those rates.

However, regarding whether students who move from the area or transfer to another school, LEA, or State should be considered dropouts, we note that the graduation rate that eligible applicants must use under this program (consistent with 34 CFR 200.19(b)(1)) is designed to adjust the cohort of students used in the rate for a given school to account for when a student transfers into that school or when a student transfers out of that school, emigrates to another country, or dies during the year covered by the rate. Thus, students who transfer out of a given school are not considered dropouts (because they become part of the cohort of students for the school into which they transfer). In calculating a dropout rate, an eligible applicant should not include students who transfer out of a school.

Changes: None.

SELECTION CRITERIA

Note: For an analysis of comments and changes on the proposed selection criteria as they relate to the evidence for and evaluation of a proposed project (Selection Criteria B and D), please see the EVIDENCE AND EVALUATION section below.

Selection Criteria in General

Comment: None.

Discussion: As discussed elsewhere in this notice, we are adding definitions of the terms applicant, official partner, and other partner in order to clarify the roles and responsibilities of entities included in applications and participating in projects under this program. Consistent with these definitions and the Eligible Applicants requirement, we are revising the selection criteria, where appropriate, to clarify the entities for which the criteria apply. We incorporate those changes in the responses to comments that follow.

In addition, we are renumbering, for each selection criterion, the factors in the criterion in order to clarify how the factors will be used.

Changes: Consistent with the Eligible Applicants requirement and the definitions of applicant, official partner, and other partner, we are revising the selection criteria for this program, where appropriate, to clarify the entities for which the criteria apply. In addition, we are renumbering, for each selection criterion, the factors in the criterion in order to clarify how the factors will be used.

Comment: One commenter recommended that the Department broaden the selection criteria used to assess Development grant pre-applications by including Selection Criterion C (Experience of the Eligible Applicant), Selection Criterion E (Strategy and Capacity to Further Develop and Bring to Scale), and Selection Criterion F (Sustainability).

Discussion: As discussed elsewhere in this notice, we no longer intend to use a two-tier process to review applications for Development grants. Thus, we will no longer include a pre-application process for Development grants. Accordingly, we are removing, from the selection criteria for Development grants, the discussion of a two-tier application process (including pre-applications) for those grants.

Changes: We are removing, from the selection criteria for Development grants, the discussion of a two-tier application process (including pre-applications) for those grants.

Comment: One commenter suggested that the Department provide a chart to show more clearly the differences in the selection criteria for the three types of grants.

Discussion: We agree with the commenter that a chart could help clarify the differences in selection criteria for the three types of grants. We will provide a chart of the selection criteria for each type of grant on the Department’s Web site for this program (see www.ed.gov/programs/innovation/index.html).

Changes: None.

Comment: Two commenters suggested that the selection criteria emphasize the importance of the effects of proposed projects on education reform and the importance of applicants’ plans to scale up projects. The commenters suggested that these changes would communicate the importance of innovation, not as an end in itself, but as a means to effect significant education reform, raise student achievement, and close achievement gaps at State, regional, and national levels.

Discussion: We agree with the commenters that innovation alone should not be the end result sought under this program. The purpose of the Investing in Innovation Fund is to support the implementation of and investment in innovative practices that are demonstrated to have an impact on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, and increasing college enrollment and completion rates. We believe that the selection criteria--particularly Selection Criteria B and E--strongly emphasize the need for eligible applicants to provide evidence that their proposed projects will lead to these outcomes and can be successfully scaled.

Changes: None.

Comment: One commenter suggested that the multiple provisions of the many selection criteria may stifle creativity and lead applicants to focus on checking off criteria rather than developing an innovative project.

Discussion: The selection criteria identify areas that the Department has determined are important for evaluating applications under this program. For Department discretionary grant programs, it is typical to have multiple selection criteria and factors that eligible applicants will address in their applications. In addition to helping ensure that only the strongest applications are selected for funding, the selection criteria provide eligible applicants flexibility and room for creativity, and we expect that each eligible applicant will address the various criteria in ways appropriate to the proposed project.

Changes: None.

Selection Criterion A--Need for the Project and Quality of the Project Design

Comment: One commenter suggested that the Department add a new factor to Selection Criterion A that focuses on the extent to which a proposed project includes and effectively leverages an established record of collaboration across multiple LEA partners. The commenter stated that this record of collaboration would enable the proposed project to address common needs and demonstrate outcomes at the regional or State level during the grant period, while providing a solid foundation to further scale the proposed project. The commenter suggested that the established record should be required to include evidence of shared plans, practices, research, and metrics to scale success beyond the students in a single LEA.

Discussion: Selection Criterion A focuses on the need for the project and the quality of the project design. The issues identified by the commenter are addressed under Selection Criteria C and E. Under Selection Criterion C (Experience of the Eligible Applicant), the Secretary considers the past performance of the eligible applicant in implementing large, complex, and rapidly growing projects (in the case of Scale-up grants); in implementing complex projects (in the case of Validation grants); or in implementing projects of the size and scope proposed by the eligible applicant (in the case of Development grants). In responding to this criterion, an eligible applicant could provide information about past collaboration across multiple LEA partners. Under Selection Criterion E (Strategy and Capacity to Bring to Scale (in the case of Scale-up and Validation grants); Strategy and Capacity to Further Develop and Bring to Scale (in the case of Development grants)), the Secretary considers the eligible applicant’s capacity to bring the proposed project to scale (in the case of Scale-up and Validation grants) or to further develop and bring to scale the proposed project (in the case of Development grants). In light of these criteria, we do not believe that it is necessary to add the recommended factor to Selection Criterion A.

Changes: None.

Comment: One commenter recommended that the Department add a new factor to Selection Criterion A for the pre-application for Development grants that focuses on the extent to which an applicant involves other entities--including local school boards, LEA and school administrators, teachers, parents, community leaders, small businesses, faith-based organizations, and other non-profit organizations--in designing the proposed project.

Discussion: As discussed elsewhere in this notice, we no longer intend to use a two-tier application process (including pre-applications) to review applications for Development grants and are removing, from the selection criteria for Development grants, the discussion of a two-tier application process for those grants.

Under Selection Criterion A, the Secretary considers the extent to which the proposed project has a clear set of goals and represents an exceptional approach to the priorities the eligible applicant is seeking to meet. In addressing this criterion, eligible applicants may wish to seek input from and partner with local organizations to determine the need that the proposed project would meet and a process for collaborating to implement the project. An eligible applicant may describe this collaborative process in addressing Selection Criterion A without the addition of a new factor. Thus, we do not believe it is necessary to adopt the commenter’s suggestion to ensure that eligible applicants include this information where appropriate.

Changes: As discussed elsewhere in this notice, we are removing, from the selection criteria for Development grants, the discussion of a two-tier application process (including pre-applications) for those grants.

Comment: One commenter recommended that the Department add a new factor to Selection Criterion A that focuses on the extent to which the applicant shows that its proposed project serves the needs of students, schools, and communities in rural areas or regions.

Discussion: Under Selection Criterion A(1) (proposed Selection Criterion A(2)(a)), the Secretary considers the extent to which the proposed project represents an exceptional approach to the priorities the eligible applicant seeks to meet. We believe that this criterion provides sufficient opportunity for eligible applicants to address the needs of students and schools in rural LEAs.

We note, in addition, that this program includes a competitive preference priority for projects that serve schools in rural areas (Competitive Preference Priority 8). Eligible applicants are eligible to receive additional points for addressing the competitive preference priorities.

Changes: None.

Comment: A few commenters recommended that the Department clarify Selection Criterion A(2)(a) regarding the extent to which a proposed project should represent an exceptional approach that has not already been widely adopted. Specifically, the commenters requested that the Department clarify whether “widely adopted” refers to scale or scope.

Discussion: In Selection Criterion A(1) (proposed Selection Criterion A(2)(a)), “widely adopted” refers to scale. If an eligible applicant’s proposed project represents an approach that is already in common usage and has achieved scale, then the project would not meet the purposes of this program.

Changes: None.

Comment: One commenter recommended that, for Scale-up grants, Selection Criterion A give greater weight to projects that fulfill needs that have already been widely documented as critical or of national significance, such as improving student performance in math and science, improving student performance in multiple grades and in multiple subjects, or improving college readiness and success for all students.

Discussion: We believe that Selection Criterion A for Scale-up grants provides adequate opportunity for eligible applicants to substantiate the critical need for the proposed project and to address issues of national significance. We do not want to limit the consideration of project need under this criterion only to critical or nationally significant issues because we believe Scale-up grants could support projects that do not necessarily rise to the level of critical or national significance.

Changes: None.

Comment: One commenter suggested that, for both Validation and Development grants, the Department award additional points under Selection Criterion A(2)(b) to an applicant that has a demonstrated record of implementing a system of continuous improvement, including the use of performance data to improve instructional practices.

Discussion: We agree with the commenter that continuous improvement systems are important to the success of projects under this program. However, we believe that an eligible applicant would be able to address the issue raised by the commenter in response to Selection Criterion C (Experience of the Eligible Applicant) and that no additional points need to be added to Selection Criterion A.

Changes: None.

Comment: None.

Discussion: The Department believes that the quality of the design of a proposed project for this program depends on the extent to which the proposed project is supported by existing research evidence. Because an eligible applicant for a Validation grant may use prior research on a strategy, practice, or program that is very similar to that of the proposed project in order to demonstrate that there is moderate evidence for the proposed project, we are revising Selection Criterion A, for Validation grants, to include consideration of whether the design of the proposed project is consistent with the existing research evidence, taking into consideration any differences in context.

Changes: For Validation grants, we are revising Selection Criterion A to include, among the factors for which the Secretary will consider the quality of the proposed project design, the extent to which the proposed project is consistent with the research evidence supporting the proposed project, taking into consideration any differences in context.

Selection Criterion C--Experience of the Eligible Applicant

Comment: One commenter suggested that the Department revise Selection Criterion C and Selection Criterion F (Sustainability) to include consideration of the extent to which an applicant has a record of support from mayors and other local government leaders.

Discussion: We agree that a record of support from mayors and other local government leaders can be one meaningful way for an eligible applicant to demonstrate both the strength of its past experience and the potential for sustainability of its proposed project. We believe that Selection Criteria C and F adequately allow for eligible applicants to provide evidence of that support.

Changes: None.

Comment: One commenter recommended that, under this criterion, the Department give full weight to applications from applicants that are successful at increasing achievement for all groups of students described in section 1111(b)(2) of the ESEA at any scale, and not consider whether applicants have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) for two or more consecutive years.

Discussion: As discussed elsewhere in this notice, section 14007(b)(1) has been amended to require that, to be eligible for an award under this program, an eligible applicant must (A) have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or (B) have demonstrated success in significantly increasing student academic achievement for all groups of students described in that section. In addition, section 14007(b)(2) has been eliminated; this section would have required that an eligible applicant have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years or have demonstrated success in significantly increasing student achievement for all groups of students described in that section through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress). Thus, to be eligible for an award, eligible applicants are no longer required by the statute to have exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years.

In addition, the statutory changes make clear that eligible applicants do not have to show that they have both significantly closed achievement gaps and significantly increased student achievement for all groups described in section 1111(b)(2) of the ESEA. Rather, eligible applicants must show either (A) that they have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA or (B) that they have demonstrated success in significantly increasing student academic achievement for all groups described in that section.

Further, section 14007(c) has been amended to specify that an eligible applicant that includes a nonprofit organization is considered to have met the requirements of new paragraphs (1) and (2) of section 14007(b) if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. Under the amendments to section 14007(c), an eligible applicant that includes a nonprofit organization is thus no longer required to demonstrate that the nonprofit organization has a record of each of the following: (1) having significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA; (2) having exceeded the State’s annual measurable objectives consistent with section 1111(b)(2) of the ESEA for two or more consecutive years or having demonstrated success in significantly increasing student achievement for all groups of students described in that section through another measure, such as measures described in section 1111(c)(2) of the ESEA (i.e., the National Assessment of Educational Progress); and (3) having made significant improvement in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and school leaders, as demonstrated with meaningful data.

We are revising Selection Criterion C to reflect these statutory changes. Under Selection Criterion C(2) (proposed Selection Criterion C(2)(b)), the Secretary now considers, in the case of an eligible applicant that is an LEA, the extent to which the eligible applicant provides information and data demonstrating that it has (A) significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or significantly increased student achievement for all groups of students described in such section; and (B) made significant improvements in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and principals, as demonstrated with meaningful data. In the case of an eligible applicant that includes a nonprofit organization, the Secretary now considers the extent to which the eligible applicant provides information and data demonstrating that the nonprofit organization has significantly improved student achievement, attainment, or retention through its record of work with an LEA or schools.

Changes: We are revising Selection Criterion C(2) (proposed Selection Criterion C(2)(b)) for all three types of grants to reflect the statutory changes. Under Selection Criterion C(2) (proposed Selection Criterion C(2)(b)), the Secretary now considers:

(2) The extent to which an eligible applicant provides information and data demonstrating that--

(a) In the case of an eligible applicant that is an LEA, the LEA has--

(i) Significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or significantly increased student achievement for all groups of students described in such section; and

(ii) Made significant improvements in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and principals, as demonstrated with meaningful data; or

(b) In the case of an eligible applicant that includes a nonprofit organization, the nonprofit organization has significantly improved student achievement, attainment, or retention through its record of work with an LEA or schools.

Comment: A few commenters recommended that the Department revise Selection Criterion C to consider evidence of applicants’ past successes. One commenter recommended that the Department consider the extent to which applicants have a record of handling operations and multi-year funding from private sources. Two other commenters recommended that the Department consider the extent to which applicants have had past success with scaling up projects. One commenter recommended that the Department consider applicants’ past success with implementing projects on a national level or in various geographic locations and academic environments.

Discussion: Under Selection Criterion C(1) (proposed Selection Criterion C(2)(a)), the Secretary considers the past performance of the eligible applicant in implementing large, complex, and rapidly growing projects (in the case of Scale-up grants); in implementing complex projects (in the case of Validation grants); or in implementing projects of the size and scope proposed by the eligible applicant (in the case of Development grants). Although this criterion does not specifically reference the types of past successes mentioned by the commenters, an eligible applicant could provide information on such successes in response to the criterion, as appropriate for the type of grant for which the eligible applicant is applying. Accordingly, we believe that the criterion is sufficient to address the commenters’ recommendations.

Changes: None.

Comment: One commenter recommended that the Department modify this criterion to include consideration of the experience of key partners who plan to work with the applicant.

Discussion: As noted in the preceding discussion, the Secretary considers under Selection Criterion C(1) (proposed Selection Criterion C(2)(a)) the past performance of the eligible applicant in implementing large, complex, and rapidly growing projects (in the case of Scale-up grants); in implementing complex projects (in the case of Validation grants); or in implementing projects of the size and scope proposed by the eligible applicant (in the case of Development grants). In response to Selection Criterion C(1) (proposed Selection Criterion C(2)(a)), an eligible applicant may discuss the experience of the applicant and official partners (as those terms are defined in this notice) in project implementation, as appropriate for the type of grant for which the applicant is applying. Because the purpose of this criterion is to assess the experience of the eligible applicant, we will not consider the experience of any other partners (as defined in this notice) that are proposed to be involved in a project.

In addition, consistent with the changes to Selection Criterion C(2)(proposed Selection Criterion C(2)(b)) discussed earlier, the eligible applicant may provide data and information in response to C(2)(b) only for the eligible applicant itself (if the eligible applicant is an LEA) or for the nonprofit organization (if the eligible applicant includes a nonprofit organization).

Changes: None.

Selection Criterion E--Strategy and Capacity to Bring to Scale (In the Case of Scale-up and Validation Grants); Strategy and Capacity to Further Develop and Bring to Scale (In the Case of Development Grants)

Comment: A number of commenters recommended that the Department remove the geographic limitation to scale a Validation grant to a State or regional level and instead allow scaling on a limited national level in noncontiguous areas. Another commenter recommended that the Department broaden the geographic areas for scaling under Validation grants to include two or more targeted urban locales in order to allow applicants the opportunity to reach several large cities and metropolitan areas. Another commenter sought clarification about whether scaling for Validation grants could occur within a single urban LEA or a large metropolitan area.

Discussion: Under Selection Criterion E, the Secretary considers, for Validation grants, an eligible applicant’s capacity to bring its proposed project to scale on a State or regional level. Through this criterion, the Department does not limit the geographic reach of proposed projects for Validation grants. If eligible applicants wish to propose a project for a Validation grant the scale of which extends beyond a State or regional level, they may do so.

As discussed earlier, we are revising the definition of regional level to clarify that, to meet this definition, a project must serve students in more than one LEA, excluding a project implemented in a State in which the State educational agency is the sole educational agency for all schools and thus may be considered an LEA under section 9101(26) of the ESEA. Thus, a project that is implemented in a single LEA (if not the sole educational agency for all schools in a State) would not be considered a regional level project consistent with the definition of regional level used in this program. Further, a project that is implemented in a single area would be considered a regional level project only if the area includes more than one LEA.

In addition, the definition of regional level does not require that regional level projects be implemented in contiguous areas.

Changes: As discussed earlier, we are revising the definition of regional level to clarify that, to meet this definition, a project must serve students in more than one LEA, excluding a project implemented in a State in which the State educational agency is the sole educational agency for all schools and thus may be considered an LEA under section 9101(26) of the ESEA. Comment: A number of commenters noted that the process of scaling a project may be hampered by internal capacity issues and recommended that the Department revise Selection Criterion E to provide for consideration of the following issues: stability of administrative leadership, teacher and staff capacity, consistency of LEA policy, external monitoring, data management, communications systems, and alignment of K-12 curricula.

Discussion: Under Selection Criterion E(2) (proposed Selection Criterion E(2)(b)), the Secretary considers an eligible applicant’s capacity, in the case of Scale-up and Validation grants, to bring its proposed project to scale, and in the case of Development grants, to develop and further scale the proposed project. The criterion provides examples of the types of capacity an eligible applicant may address: qualified personnel, financial resources, and management capacity. These examples are not intended to be an exhaustive or exclusive list. An eligible applicant may address other types of capacity not covered by the examples, including those mentioned by the commenters.

Changes: None.

Comment: One commenter asked the Department to clarify whether an applicant for a grant can meet Selection Criterion E if it has not identified in its application all of the partners with which it intends to work.

Discussion: So long as the eligible applicant meets the eligibility requirements for this program (which include, for eligible applicants that include a nonprofit organization, that the eligible applicant describe the demographics and other characteristics of any LEAs or schools with which it intends to partner that are not named in its application), an eligible applicant will be considered for funding. It will be up to reviewers to determine whether an eligible applicant that has not identified all of its partners has provided sufficient documentation demonstrating the quality of the eligible applicant’s strategy and capacity to bring its proposed project to scale consistent with this criterion.

Changes: None.

Comment: One commenter recommended that the Department require an applicant to describe its methodology for scaling up its proposed project, including how the methodology will minimize risks and how the applicant will use benchmarks.

Discussion: We believe that Selection Criterion E adequately addresses the commenter’s recommendation that an eligible applicant describe its scaling up methodology. We do not believe it is necessary therefore to include an additional requirement that eligible applicants provide the descriptions recommended by the commenter. In addition, we note that an eligible applicant could potentially discuss the specific methodological elements mentioned by the commenter in response to other selection criteria, including Selection Criteria F (Sustainability) and G (Quality of the Management Plan and Personnel).

Changes: None.

Comment: One commenter recommended that the Department emphasize the creation of platforms (i.e., systemic frameworks) for innovation rather than emphasizing project replication, which suggests a one-size-fits-all approach. The commenter recommended that Selection Criterion E(2)(c), under which the Secretary considers the feasibility of the proposed project to be replicated successfully, should instead provide for consideration of innovative platforms or frameworks that can be readily adapted and tailored to individual school settings.

Discussion: The Department recognizes that the ways in which organizations replicate and bring to scale their work may vary. We do not intend to suggest that a one-size-fits-all approach is preferred under this program. Selection Criterion E(3) (proposed Selection Criterion E(2)(c)) clearly states that the Secretary considers the feasibility of the proposed project to be replicated successfully in a variety of settings and with a variety of student populations. However, we believe that an eligible applicant is best positioned to determine the scaling strategy that is most appropriate for its proposed project. We do not believe that it is necessary to establish parameters for these strategies and therefore decline to modify this criterion as the commenter recommends.

Changes: None.

Comment: Two commenters recommended that the Department revise Selection Criterion E to include project outcomes, in addition to the geographic reach of projects and the number of students to be served, as indicators of applicants’ capacity to scale up projects effectively. One of these commenters suggested that the Department define the expected outcomes and determine the specific skills that projects should help students acquire.

Discussion: The Department agrees that geographic reach and numbers of students to be served are not by themselves sufficient to determine whether the scaling up of an eligible applicant’s project will be effective with respect to outcomes. However, an eligible applicant may address project effectiveness in response to other selection criteria. Under Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect), the Secretary considers the strength of the evidence for the potential effects of proposed projects on student achievement and attainment outcomes, including: improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates. In response to this criterion, eligible applicants may also address the effects of proposed projects on intermediate variables that are strongly correlated with improving these outcomes, such as (but not limited to) teacher or principal effectiveness. We believe that this criterion provides sufficient opportunity for eligible applicants to discuss the expected outcomes of proposed projects and for reviewers to assess an applicants’ capacity to scale up proposed projects in relation to those outcomes, and thus sufficiently addresses the recommendations of the commenters.

Changes: None.

Comment: A number of commenters requested that the Department clarify whether Selection Criterion E(2)(d) establishes specific numeric expectations for the scale of proposed projects. Some commenters recommended that the Department not require grantees to reach the numeric student targets proposed for each type of grant during the grant period. Many of these commenters were particularly concerned that applicants with limited resources or from rural areas would not be able to meet these scaling expectations; they requested that the requirements be reduced or that applicants have an opportunity to request a waiver from meeting Selection Criteria E(2)(d) and E(2)(b). Some commenters expressed concern that the numeric student targets were unrealistic and suggested that the Department allow alternatives for determining the size of the student targets (such as the size of the applying LEA) or allow other ways of demonstrating capacity to scale (such as evidence of collaborative partnerships).

Discussion: Selection Criterion E(4) (proposed Selection Criterion E(2)(d)) does not establish requirements for scaling proposed projects to specific numbers of students. Rather, the intent of the criterion is to gather information that can help judge project cost-effectiveness. Under Selection Criterion E(4) (proposed Selection Criterion E(2)(d)), the Secretary considers cost estimates both (a) for the total number of students to be served by the proposed project, which is determined by the eligible applicant, and (b) for the eligible applicant or others (including other partners) to reach the scaling targets for the respective grant types (i.e., 100,000, 250,000, and 500,000 students for Development and Validation grants; and 100,000, 500,000, and 1,000,000 students for Scale-up grants). The total number of students that the eligible applicant proposes to serve is expected to be reached by the end of the grant period. The scaling targets, in contrast, are theoretical and allow peer reviewers to assess the cost-effectiveness generally of proposed projects, whether implemented by the eligible applicant or any other entity; grantees are not required to reach these numbers during the grant period.

An eligible applicant is free to propose how many students it will serve under its project, consistent with its project goals, capacity, and resources. Because there is no minimum threshold established for the number of students to be served, an eligible applicant would under no circumstance need a waiver of Selection Criterion E(4) (proposed Selection Criterion E(2)(d)) or Selection Criterion E(2) (proposed Selection Criterion E(2)(b)) (the latter of which considers an eligible applicant’s capacity, in the case of Scale-up and Validation grants, to bring its proposed project to scale, and in the case of Development grants, to develop and further scale the proposed project). Neither is it necessary for the Department to consider alternative means of determining numerical student targets or to consider alternative means of showing capacity to scale in lieu of meeting student targets.

The Department recognizes, however, that the two types of estimates considered in Selection Criterion E(4) (proposed Selection Criterion E(2)(d)) could benefit from further distinction. Therefore, we are revising the criterion to explicitly distinguish between the eligible applicant’s estimate of the per-student cost of the proposed project, which includes the start-up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project, and the cost estimates for the eligible applicant or others (including other partners) to reach the scaling targets for the respective grant types (i.e., 100,000, 250,000, and 500,000 students for Development and Validation grants; and 100,000, 500,000, and 1,000,000 students for Scale-up grants).

We note, in addition, that this program establishes the expectation under Selection Criterion E that eligible applicants for Scale-up grants bring a project to scale on a national, regional, or State level and that eligible applicants for Validation Grants bring a project to scale on a State or regional level. Both regional level and national level are defined under this program. Neither of these definitions, however, references specific targets for the numbers of students to be served.

Changes: We are revising Selection Criterion E(4) (proposed Selection Criterion E(2)(d)) for each type of grant to clarify that the Secretary will consider the following cost estimates: the eligible applicant’s estimate of the cost of the proposed project, which includes the start-up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project; and an estimate of the costs for the eligible applicant or others (including other partners) to reach the scaling targets for the respective grant types (i.e., 100,000, 250,000, and 500,000 students for Development and Validation grants; and 100,000, 500,000, and 1,000,000 students for Scale-up grants).

Comment: Several commenters recommended that the Department revise Selection Criterion E(2)(d) regarding the manner in which project cost estimates are provided. A few commenters recommended that the Department consider total costs per student and total costs per student per year. One commenter recommended that the Department consider project costs along a timeline (i.e., at one year, five years, and ten years) and require grantees to evaluate project cost estimates in self-evaluations. Another commenter recommended that the Department consider costs per student per hour of programming to ensure a more accurate and fair measure of project cost.

Discussion: We agree that clarifying Selection Criterion E(4) (proposed Selection Criterion E(2)(d)) regarding the manner in which eligible applicants should provide project cost estimates is warranted. We are therefore revising Selection Criterion E(4) (proposed Selection Criterion E(2)(d)) to specify that the Secretary will consider the eligible applicant’s estimate of the cost of the proposed project, which includes start-up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project. Thus, the Secretary will consider estimates of total project cost per student per year. We believe that all eligible applicants will be able to provide these estimates and that this measure will enable useful analysis of project costs. We believe that this change sufficiently addresses the commenters’ recommendations that we consider costs over time. We decline to accept the commenter’s recommendation that we consider costs per student per hour of programming because we do not believe this measure will enable a similarly useful analysis of project costs.

Consistent with the Evaluation requirement for this program, eligible applicants that receive funding must comply with the requirements of any program evaluation conducted by the Department, are required to conduct an independent evaluation of their proposed projects, and must agree to cooperate with technical assistance provided by the Department to ensure that these evaluations are of the highest quality. We believe that these provisions are adequate to address concerns regarding evaluation of cost estimates.

Changes: As discussed earlier in this notice, we are revising Selection Criterion E(4) (proposed Selection Criterion E(2)(d)) for each type of grant to clarify that the Secretary will consider the eligible applicant’s estimate of the cost of the proposed project, which includes start-up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project.

Comment: Some commenters recommended that the Department clarify the specific types of costs that applicants should include when estimating costs in response to Selection Criterion E(2)(d). One commenter recommended that the Department require applicants to distinguish the costs associated with research and evaluation from the costs for project infrastructure, development, and operation. Two commenters recommended that the Department provide guidance on how applicants should calculate indirect start-up costs to ensure that only costs specific to the proposed project itself are included in cost estimates. Another commenter recommended that the Department consider estimated direct and indirect cost savings during the grant period.

Discussion: Cost estimates should include all costs for implementing the project, including but not limited to start-up costs, operating costs, indirect costs, evaluation costs, materials, and personnel training. The cost estimates may only include costs for activities designed to serve students directly through the project. The eligible applicant should discuss how it arrived at its cost estimates and what specific items and activities were included in the calculations used to arrive at those estimates. These calculations should show fixed and variable costs, incremental costs, and savings over time. The eligible applicant should provide the calculations used to arrive at the estimates for the cost of the proposed project (in terms of the number of students to be served) as well as the costs for the eligible applicant or others (including other partners) to reach the scaling targets for the respective grant types (i.e., 100,000, 250,000, and 500,000 students for Development and Validation grants; and 100,000, 500,000, and 1,000,000 students for Scale-up grants). We believe that this guidance provides sufficient clarification on the types of costs an eligible applicant should include and adequately addresses the commenters’ concerns.

Changes: None.

Comment: A few commenters recommended that the Department give greater consideration to the infrastructure costs associated with different types of projects. The commenters cautioned the Department not to rely heavily on estimates of costs for the initial stages of a proposed project, as these estimates may not accurately reflect infrastructure costs as projects are expanded to serve more students. Two commenters stated that applicants should describe the resources required to implement a project and indicate whether or not the project is a replication of existing activities. Two commenters noted that the Department should acknowledge that start-up and operating costs in sites that replicate a project may decrease significantly over time through economies of scale.

Discussion: Although we agree with the commenters that infrastructure costs may inflate start-up costs, we believe that estimates of the start-up and operational costs per student per year (as under revised Selection Criterion E(4) (proposed Selection Criterion E(2)(d))) will provide reviewers a sufficiently informative measure of costs. To the extent that eligible applicants can provide context for their estimates of start-up and operating costs (including for variable costs relating to project infrastructure), the Department encourages eligible applicants to provide this information.

Changes: None.

Comment: A number of commenters recommended that the Department consider cost estimates, including estimates of cost savings over time, in relation to the impact of proposed projects on student outcomes. Some commenters expressed concern that estimates of costs per student are not, by themselves, an adequate measure of cost-effectiveness and suggested that the Department consider measures of the benefits of proposed projects as well. One commenter expressed concern that Selection Criterion E might place applicants that propose technology-based projects at a competitive disadvantage relative to other applicants because of the potential high costs of developing and implementing such projects; the commenter stated that cost estimates would not address the benefits of these projects and similarly recommended that the Department consider costs relative to outcome gains. Two commenters suggested that the Department compare cost-effectiveness across projects that address the same outcome.

Discussion: The Department agrees that estimates of costs per student per year, which may include cost savings over time, are not, by themselves, an adequate measure of project cost-effectiveness. However, as discussed earlier, an eligible applicant may address project effectiveness with respect to outcomes in response to other selection criteria. Under Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect), the Secretary considers the evidence for the potential effects of proposed projects on outcomes including the following: improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates. In response to this criterion, eligible applicants may also address the effects of proposed projects on intermediate variables that are strongly correlated with improving these outcomes, such as (but not limited to) teacher or principal effectiveness. We believe that this criterion provides sufficient opportunity for eligible applicants to discuss the expected outcomes of proposed projects and for reviewers to assess project costs in relation to those outcomes. We note that peer reviewers evaluate applications against the selection criteria; reviewers do not evaluate applications by comparing them with each other.

Changes: None.

Comment: A few commenters expressed concern that providing the cost estimates in response to Selection Criterion E will be burdensome to applicants and that this burden may outweigh the value of the estimates. One of these commenters suggested that the Department instead consider other, less burdensome cost measures such as initial and targeted investments. One commenter recommended that the Department allow reviewers to assess cost through consideration of the budget for each year of a proposed project. Another commenter recommended that the Department rely on reviewers to make sensible judgments of project cost-effectiveness and not require applicants to provide the estimates discussed in the criterion.

Discussion: Under Selection Criterion E(4) (proposed Selection Criterion E(2)(d)), the Secretary considers an eligible applicant’s estimates both of the cost for reaching the total number of students to be served by the proposed project and for the eligible applicant or others (including other partners) to reach the scaling targets for the respective grant types (i.e., 100,000, 250,000, and 500,000 students for Development and Validation grants; and 100,000, 500,000, and 1,000,000 students for Scale-up grants). We appreciate the commenters’ concerns that providing these estimates may be burdensome to eligible applicants. However, as discussed earlier, we believe that these estimates will provide reviewers a useful and informative measure of costs of the projects that may be proposed under this program; and as a result, we believe that the benefits of these estimates outweigh the burden on eligible applicants in providing them. In addition, it is not clear to us that the alternative measures recommended by the commenters would be less burdensome to eligible applicants or more useful to reviewers. Therefore, we decline to add to this criterion an alternative or additional cost measure.

Changes: None.

Selection Criterion F--Sustainability

Comment: One commenter suggested that the Department revise Selection Criterion F to consider the extent to which the proposed project can be integrated into the fabric of LEAs, schools, and nonprofit partners. The commenter suggested that a promise of money to operate the project beyond the length of the grant does not reflect the spirit of innovation. The commenter also suggested that the Department foster a ”doing more with less” approach rather than an approach that would “add on” projects, which the commenter stated would not foster investments in true innovation.

Discussion: We believe that the criterion sufficiently addresses the commenter’s concerns. Under Selection Criterion F(2) (proposed Selection Criterion F(2)(b)), the Secretary considers the potential and planning for the incorporation of project purposes, activities, or benefits into the ongoing work of the eligible applicant and any other partners at the end of the grant. Under Selection Criterion F, the Secretary will also consider the adequacy of resources to continue the proposed project after the grant period ends, which would include the expenses associated with the continued management of projects.

Changes: None.

Comment: Several commenters recommended that the Department revise Selection Criterion F(2)(a) to include additional stakeholders such as parents, students, local government, community-based organizations, faith-based organizations, institutions of higher education, research institutes, and entities that may not typically be considered education stakeholders. The commenters stated that support from these stakeholders may help demonstrate the sustainability of the proposed project.

Discussion: The list of potential stakeholders in Selection Criterion F(1) (proposed Selection Criterion F(2)(a)) is not intended to be exhaustive. We cannot include all potential stakeholders in the criterion and so decline to make the additions recommended. In addressing this criterion, eligible applicants may provide evidence of support from other stakeholders including those mentioned by the commenters.

Changes: None.

Comment: One commenter expressed concern that it would be difficult for States and LEAs that currently have budget problems to sustain funded projects in the future.

Discussion: We agree with the commenter that budget problems may create challenges for some States and LEAs to sustain projects. These budgetary concerns, however, emphasize the importance of LEAs and States learning from each other and sharing those practices that have improved project outcomes in a cost-effective manner. This program aims both to promote this kind of sharing and to better leverage public and private sector investments in education. The Cost Sharing or Matching requirement is intended to help address the challenges faced by grantees and increase the sustainability of projects by securing matching funds from the private sector.

Changes: None.

Comment: Two commenters recommended that the Department award additional points to applicants with previous experience in obtaining or leveraging funding from private sources.

Discussion: Eligible applicants that have a record of securing funding from private sources or that have new funding already secured can demonstrate those qualities in response to this criterion and other selection criteria, including Selection Criterion C (Experience of the Eligible Applicant) and Selection Criterion E (Strategy and Capacity to Bring to Scale (in the case of Scale-up and Validation grants); Strategy and Capacity to Further Develop and Bring to Scale (in the case of Development grants)). As a result, we do not believe it is necessary to add a criterion (with additional points) to account for the consideration of this information.

Changes: None.

Selection Criterion G--Quality of the Management Plan and Personnel

Comment: Several commenters recommended that the Department modify Selection Criterion G(2)(b)) to include consideration of the qualifications of key partner personnel in addition to the qualifications of the project directors and key project personnel. Another commenter recommended that the Department modify Selection Criterion G to include consideration of partnerships that are strategic for management and personnel purposes.

Discussion: In response to Selection Criterion G, an eligible applicant may include personnel from those partners (official partners or other partners) who are important to achieving the proposed project’s objectives and may discuss the responsibilities of those personnel. Accordingly, we believe that the selection criterion addresses these commenters’ concerns.

Changes: None.

Comment: One commenter suggested that the Department revise Selection Criterion G to include consideration of whether the proposed project includes one or more key personnel who can demonstrate understanding of and experience with programs and practices in rural schools or LEAs.

Discussion: The Department believes that the commenter’s concern is addressed in the general consideration of the qualifications of key personnel under Selection Criterion G. Because of the variety of applications that are likely to be submitted under this program, we do not believe it is appropriate to specifically consider whether eligible applicants include staff with experience working with specific types of schools or LEAs, such as rural schools or LEAs.

Changes: None.

EVIDENCE AND EVALUATION

Comment: The Department received a large number of comments on the standards of evidence for this program. Some commenters supported the Department’s emphasis on the proposed use and generation of evidence for the Development, Validation, and Scale-up grants.

Discussion: We appreciate the commenters’ support. To ensure that applications for Scale-up grants are supported by strong evidence (as defined in this notice), that applications for Validation grants are supported by moderate evidence (as defined in this notice), and that applications for Development grants are supported by reasonable hypotheses, we are revising the requirements for this program to explicitly address these evidence standards.

Changes: We are adding a requirement that to be eligible for an award, an application for a Scale-up grant must be supported by strong evidence (as defined in this notice), an application for a Validation grant must be supported by moderate evidence (as defined in this notice), and an application for a Development grant must be supported by a reasonable hypothesis.

Comment: None.

Discussion: To provide further clarity, we are adding a definition of the term well-implemented and well-designed, with respect to an experimental or quasi-experimental study.

Changes: We are adding that, for this program, well-designed and well-implemented means, with respect to an experimental or quasi-experimental study (as defined in this notice), that the study meets the What Works Clearinghouse evidence standards, with or without reservations (see http://ies.ed.gov/ncee/wwc/references/idocviewer/doc.aspx?docid=19&tocid=1 and in particular the description of “Reasons for Not Meeting Standards” at http://ies.ed.gov/ncee/wwc/references/idocviewer/Doc.aspx?docId=19&tocId=4#reasons).

Comment: None.

Discussion: To provide further clarity on what we will consider under Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect) with respect to the strength of the existing research, we are revising the criterion for all three types of grants.

Changes: We are revising Selection Criterion B (Strength of Research, Significance of Effect, and Magnitude of Effect) for Scale-up and Validation grants to clarify that the strength of the existing research evidence includes the internal validity (strength of causal conclusions) and external validity (generalizability) of the effects reported in prior research. We are also revising the criterion for Development grants to clarify that the strength of the existing research evidence includes reported practice, theoretical considerations, and the significance and magnitude of any effects reported in prior research.

Comment: Some commenters argued that well-conducted experimental studies--including delayed-treatment studies or studies that use lotteries to allocate slots for oversubscribed programs--provide definitive evidence of the effectiveness of innovations and should receive a competitive preference over quasi-experimental or non-experimental studies. Other commenters recommended that evidence from one well-designed and well-implemented experimental study, when feasible, be a prerequisite for receiving a Scale-up grant. One commenter recommended that similar criteria be applied to applications for Validation grants.

Discussion: This notice defines strong evidence in a way that gives more weight to a large, well-designed and well-implemented (as defined in this notice), multisite experimental study (as defined in this notice) than to a corresponding quasi-experimental study (as defined in this notice). This emphasis is justified, because a large-scale experimental study is likely to yield evidence with greater confidence and a stronger claim to internal validity than a similarly sized quasi-experiment. Nonetheless, we do not favor giving a further preference to applicants relying on experimental evidence, for example by making a well-designed and well-implemented (as defined in this notice) experiment (where feasible) a prerequisite for receiving a Scale-Up grant. Such preferences would risk discounting valid evidence from quasi-experimental studies and could exclude from funding and further study promising innovations for which experimental evaluations are less feasible.

Changes: None.

Comment: Many commenters argued that the proposed definitions of moderate evidence and strong evidence are too narrow and restrictive given the focus of the grants on supporting innovation. Commenters criticized what they perceived to be an unduly exclusive, inflexible, and expensive focus on experimental and quasi-experimental designs to the exclusion of other research designs, such as correlational and longitudinal outcomes analyses utilizing available public data. These commenters also expressed concern that many organizations with experience developing education interventions to help struggling students may be relatively small and may lack experience with the costly data infrastructure required for experimental or quasi-experimental studies. The commenters expressed concern that excluding such organizations from Scale-up and Validation grants would be counterproductive to the goals of the Investing in Innovation program.

Discussion: The Department does not believe that the definitions of moderate evidence and strong evidence are too narrow and restrictive. A program’s evidence of effectiveness should be commensurate with the scale on which the program will be implemented: thus, we are requiring strong evidence for implementation at the State, regional, or national level (Scale-up grants), and moderate evidence for implementation at the State or regional level (Validation grants). Where strong or moderate evidence is lacking, study of a promising program through a Development grant may be appropriate.

While strong evidence focuses on findings from well-designed and well-implemented (as defined in this notice) experimental and quasi-experimental studies, moderate evidence includes not only evidence from experimental and quasi-experimental studies, but also correlational research with strong statistical controls for selection bias and for discerning the influence of internal factors. Analysis of the outcomes over time reported in public data can occur in the context of quasi-experimental studies, such as the interrupted time series studies described in this notice.

For the purpose of submitting applications to the Department, eligible applicants who lack experience with the data collection required for experimental or quasi-experimental evaluations can form official or other partnerships with entities offering such experience. In many instances, much of the data required for the evaluation will already be collected by the agencies implementing the innovation, for example by districts as part of their school accountability and student progress monitoring systems. Because experimental studies require smaller sample sizes than do other studies to detect the same magnitude of effects, data collection costs for experiments may be less than data collection costs for quasi-experiments and correlational studies.

Changes: None.

Comment: One commenter proposed that “robust, quantifiable” findings be viewed as a source of strong evidence on the effectiveness of a defined practice, strategy, or program when competing explanations for changes in outcomes have been ruled out. The commenter also proposed that qualitative data on the relationship between a defined practice, strategy, or program, and proven and promising interventions, be viewed as a source of moderate evidence that the practice, strategy, or program is effective.

Discussion: The Department believes that, regardless of whether prior research studies include a qualitative component, ruling out competing explanations for differences in outcomes is necessary for either strong or moderate evidence of effectiveness to be present. Accounting for any differences between program participants and non-participants can be accomplished by random assignment to treatment and control groups, or through a variety of quasi-experimental or statistical methods. Studies utilizing these designs and methods can provide strong or moderate evidence for the purposes of this program. The identification of any significant associations between qualitative measures of program implementation and outcomes can only provide moderate evidence of effectiveness if this research includes strong statistical controls for selection bias and for discerning the influence of internal factors that could be responsible for differences in outcomes.

Changes: None.

Comment: Some commenters supported the use of quasi-experimental and mixed method evaluation strategies, stating that even well-implemented experimental designs can suffer weaknesses and limitations in their external validity.

Discussion: We agree with the commenters that all evaluations can suffer weaknesses and limitations in their external validity, regardless of whether they are experimental or quasi-experimental in nature. A large, well-designed, well-implemented, randomized, controlled, multisite trial is likely to have strong external validity as well as internal validity. Concerns about external validity also can be addressed with evidence from more than one well-designed and well-implemented (as defined in this notice) experimental study (as defined in this notice) or quasi-experimental study (as defined in this notice) supporting the effectiveness of the practice, strategy, or program for different populations. The evaluation requirements in this notice allow for mixed method strategies, for example to provide implementation data, performance feedback, progress assessment, and information relevant for replication in other settings. A well-designed evaluation of a Scale-up or Validation project would take into consideration both external validity and internal validity when specifying either an experimental or quasi-experimental design.

Changes: None.

Comment: Several commenters expressed concern that experimental and quasi-experimental research designs may be inappropriate for evaluating complex innovations, including innovations with multiple components adapted to a local context. Many commenters expressed concern that the evidence definitions would favor, at most, small, narrowly targeted, short-term interventions as opposed to bold, comprehensive, multiple-component, long-term school- or LEA-wide innovations. As examples of comprehensive innovations unsuited to analysis through random assignment, commenters pointed to turnaround programs implemented in particular schools, LEA-wide initiatives in curriculum and instruction, and family and neighborhood engagement strategies.

Discussion: The Department appreciates the importance of the commenters’ concern, but disagrees that experimental and quasi-experimental methods are ill-suited to study complex innovations. Over the past thirty years, numerous multiple-component social programs, including those involving education reforms, have been evaluated rigorously using experimental and quasi-experimental methods, and some have been found to be effective. We believe that a range of experimental and quasi-experimental methods can be considered to identify potentially effective, comprehensive programs and to evaluate those programs when implemented on a larger scale, for example at a State or regional level.

The evidence standards established in this notice permit the consideration of systemic LEA and whole-school initiatives, as well as interventions targeted for specific groups of children within schools. For example, school-wide innovations can be studied through the random assignment of entire schools to implement specified practices or combinations of practices. Other LEA-wide or school-wide innovations can be studied through quasi-experimental methods, such as interrupted time series comparisons of outcomes before and after a program begins. Substantively significant findings can arise when even a small number of LEAs and schools are included in a study. However, studies involving larger numbers of LEAs and schools have stronger external validity and greater likelihood of detecting effects at a given level of statistical significance.

Changes: None.

Comment: A few commenters recommended that the Department fund the scaling up and validation of comprehensive strategies (or combinations of strategies) that are associated with “extraordinary” student learning gains and that the applicant plans to evaluate rigorously. One commenter suggested that the Department define an “escape clause” that would permit a Scale-up grant to be awarded to support an innovation that was exceptionally promising on theoretical grounds but that lacked support from a randomized study.

Discussion: The Department believes that, given the magnitude of public investment being planned for Scale-up grants and the number of students who would be affected, we need to require strong empirical evidence of significant learning gains before awarding a Scale-up grant. Likewise, moderate empirical evidence of significant learning gains should be required before a Validation grant is awarded; this evidence could be experimental or quasi-experimental. Learning gains that appear extraordinary, but that lack strong or moderate evidence of being caused by the innovation in question, therefore, would not justify funding for State or regional implementation through a Scale-up or Validation grant, but could justify funding at the level of a Development grant.

Changes: None.

Comment: Some commenters criticized the Department’s proposed definition of strong evidence because under the definition a single, well-designed study could provide sufficient evidence when the same study would be insufficient for the Department’s What Works Clearinghouse.

Discussion: For the purposes of this program, the Department considers a single, large, well-implemented, multisite, randomized, controlled trial with evidence of effectiveness as equivalent to two separate quasi-experimental studies or two smaller experimental studies. Scale-up funding will permit researchers to test whether an innovation that has already been validated with strong evidence of effectiveness for diverse populations maintains its effectiveness when implemented on a State, regional, or national scale. The evidence standards of the What Works Clearinghouse were not developed for the purpose of evaluating effectiveness under conditions of scale-up implementation, but rather for the purpose of synthesizing research evidence, often from multiple, small-scale efficacy studies, rather than large, multisite evaluations.

Changes: None.

Comment: Some commenters recommended reducing the number of evidence levels, and the corresponding number of grant categories, from three to two. These commenters proposed combining the strong and moderate evidence criteria under Scale-up grants, and supporting a wider range of projects under Validation or Development grants.

Discussion: The Department believes that the distinction between strong evidence of effectiveness and moderate evidence of effectiveness is a meaningful distinction with respect to both the funding of innovations and the purpose of the funding: namely, scaling up effective practices, strategies, and programs (Scale-up grants), as opposed to validating claims of effectiveness (Validation grants). The multiple tiers of evidence corresponding with the three categories of grants under this program will permit the Department to support a wide range of projects. Development grants will permit promising innovations to be tested, while the larger Validation and Scale-up grants will support the implementation and evaluation of innovations at levels commensurate with the corresponding evidence of effectiveness.

Changes: None.

Comment: One commenter proposed that the principles of scientific research in education identified by the National Research Council in 2002 be applied to all three types of grants.

Discussion: Many eminent organizations have proposed definitions of scientific evidence in education. The six principles identified by the National Research Council in 2002 provide a general foundation and framework for understanding scientific research in education, but do not focus specifically on criteria for identifying effective education practices, strategies, and programs. The evidence criteria and the definitions for this program were developed to be reasonable and specific given the purposes of this grant program to support the development, validation, and scaling up of effective innovations.

Changes: None.

Comment: A few commenters asked the Department to clarify how the evidence of effectiveness will be defined and the quality of a research design determined. One commenter asked whether applicants will be required to meet the evidence criterion in the January 25, 2005, notice on “Scientifically Based Evaluation Methods” (70 FR 3585).

Discussion: Evidence of effectiveness will be assessed relative to the internal validity and external validity of such claims utilizing a peer review process that will include experts with strong backgrounds in research and evaluation. We are establishing the evidence standards and evaluation requirements for this program in this notice; the 2005 notice regarding scientifically based evaluation methods is not being used for this program.

Changes: None.

Comment: One commenter asked the Department to provide descriptions of what constitutes high internal and external validity for Scale-up grants. The same commenter also requested an explanation of how the strong evidence required for Scale-up grants will be distinguished from the moderate evidence required for Validation grants and from the evidence required for Development grants.

Discussion: The Department has revised Table 1 (“Differences Between the Three Types of Investing in Innovation Grants in Terms of the Evidence Required to Support the Proposed Practice, Strategy, or Program”) to provide more detailed summary information contrasting the evidence criteria for each type of grant. Internal validity refers to confidence regarding causal inferences and external validity refers to confidence regarding generalizability of findings. Scale-up grants will support practices, strategies, and programs for which there are few threats to either internal or external validity of claims of effectiveness. Validation grants will support practices, strategies, and programs with evidence of effectiveness, even if some threats to internal or external validity arise from the limitations of previous studies, such as small sample sizes or lack of baseline equivalence between treatment and comparison groups. Development grants will support further study of promising practices, strategies, and programs for which evidence of effectiveness is lacking.

Changes: None.

Comment: Many commenters requested that the Department provide applicants with clear guidance on the evidence standards that will be used to evaluate applications, including (1) examples of case studies or actual research in the absolute priority areas that meet the moderate and strong evidence requirements and (2) specifications of desired outcome measures and appropriate program performance metrics, including how the program goals should vary by grade level across projects.

Discussion: Because of the diversity of practices, strategies, and programs that may be supported through the different categories of grants, the Department does not wish to over-emphasize any particular area in the competition by citing to specific examples or case studies, or by defining specific outcome measures beyond those mentioned in this notice.

Changes: None.

Comment: Some commenters asked the Department to clarify whether (1) evidence from an experimental or quasi-experimental study of a similar solution in a similar setting could be used as evidence to support a Scale-up grant application, (2) evidence from a large, multisite, experimental evaluation of a component of the program and peer-reviewed publications on other components could be used as evidence to support a Validation grant application, and (3) a Scale-up grant applicant may have been, or should have been, a subject of the prior study. One commenter proposed that the Department permit a consortium of organizations to submit and receive credit for research evidence from individual organizations within the consortium. Another commenter requested guidance on whether adding a new dimension to an existing program would preclude the project from meeting the criteria for a Scale-up grant. In reference to Validation grants, several commenters urged the Department to accept applications that adapt validated practices to new contexts. Other commenters asked whether modifications to well-tested models would receive points in both the Validation and the Scale-up grant categories.

Discussion: Evidence of the effectiveness of a proposed practice, strategy, or program will be stronger in terms of internal validity if the prior research applies to the same innovation the eligible applicant is proposing, rather than to a similar innovation or to a component of the proposed strategy or program. Evidence of effectiveness will be stronger in terms of external validity if the previous studies included at least some schools associated with the eligible applicant, and if these schools were similar to the schools in which the proposed innovation would be implemented. Eligible applicants (including consortium partners) that were involved in the actual implementation of the previously studied innovation would have a more credible application to bring to scale that innovation, than would applicants replicating an innovation previously implemented by others. Modification and adaptation of existing, well-tested practices for new contexts may mean that strong evidence of effectiveness in the original context is only moderate evidence of effectiveness in the new context. Eligible applicants must determine whether the weight of evidence for the internal and external validity of claims of effectiveness is sufficient to apply for a Scale-up grant as opposed to a Validation grant. In general, innovations that are similar to, but not the same as, those that have been evaluated previously with strong evidence of effectiveness will not be eligible for a Scale-up grant, but may be eligible for a Validation grant.

Changes: None.

Comment: One commenter proposed that a third category, procedural validity, be used in addition to internal validity and external validity. The commenter proposed defining procedural validity as the extent to which the developer followed scientifically approved methodology in the development, piloting, and implementation of the innovation.

Discussion: While we appreciate the commenter’s suggestion, the Department believes that applications for Scale-up and Validation grants should be based on how the innovations have been implemented in the past, rather than on how they could have been implemented. Issues of procedural validity in the implementation of similar practices, strategies, or programs could be considered as part of the justification for a Development project.

Changes: None.

Comment: One commenter recommended that program success be measured by statistically significant improvements in social and behavioral outcomes in addition to academic achievement.

Discussion: Social and behavioral measures could be intermediate outcomes that contribute to student educational achievement and attainment and, thus, already are targeted under this program. Applications should include citations of relevant research that establishes a direct correlation between intermediate outcomes and the outcomes described in this notice. This research should include research designs or statistical controls for selection bias and for discerning the relationship between intermediate outcomes and the outcomes described in this notice.

Changes: None.

Comment: One commenter argued that a single, high-quality, quasi-experimental study should be sufficient to provide “strong evidence” of effectiveness, because any study of this quality is likely to be expensive and because requiring more than one study would rule out otherwise qualified applicants. Other commenters argued in favor of using multiple sites and multiple studies to generate evidence, and criticized the Department’s proposal to require only one acceptable experimental study. According to these commenters, the Department’s proposed approach would decrease the evidentiary standard for Scale-up grants.

Discussion: In general, the Department supports the principle that strong evidence of effectiveness should be established through multiple studies in multiple sites. Scale-up implementation at the State, regional, or national level may be justified if an innovation has evidence of effectiveness in multiple settings and for different populations. The evidence standard for this grant program, summarized in Table 1, makes an exception in the case of a large, well-designed and well-implemented (as defined in this notice) randomized controlled, trial in multiple sites. Threats to the internal validity of claims of effectiveness are greater for quasi-experimental evaluations than for experimental evaluations. In particular, compared with more straightforward findings from large-scale experiments, findings from large-scale quasi-experimental studies may be sensitive to decisions concerning analysis methods such as statistical matching and regression modeling, and therefore need to be confirmed through multiple studies.

Changes: None.

Comment: Some commenters suggested that multiple method studies be defined and encouraged in the standards of evidence for Scale-up, Validation, and Development grants.

Discussion: The Department agrees that multiple method studies can help researchers understand the context and implementation of a program. These studies may be especially useful for the evaluations of Scale-up, Validation, and Development projects. The Department does not believe it is necessary or efficient to incorporate a potentially costly multiple method requirement into the standards of evidence for Scale-up and Validation grants, because the qualitative data collection needs of an evaluation are likely to depend on the type of innovation being studied.

Changes: None.

Comment: One commenter recommended that, under the definitions, the discussion of “randomized control trials” include an emphasis on minimizing overall and differential attrition, and that the discussion of matched comparison group designs include a discussion of establishing baseline equivalence.

Discussion: Studies with high levels of overall or differential attrition, or without baseline equivalence between treatment and comparison groups, would not meet the standard of strong evidence, as defined in this notice. A well-designed and otherwise well-implemented study with a flaw in one of these areas would likely be considered moderate evidence of effectiveness. The issues of differential attrition and baseline equivalence are discussed in the Department’s What Works Clearinghouse Procedures and Standards Handbook (see

http://ies.ed.gov/ncee/wwc/references/idocviewer/

doc.aspx?docid=19&tocid=1, and also the IES/NCEE Technical Methods papers at http://ies.ed.gov/ncee/tech_methods/).

Changes: None.

Comment: One commenter recommended that the Department define “multisite” as including multiple schools, LEAs, or cities.

Discussion: The definition of multisite depends on the level at which the innovative practice, strategy, or program will be implemented and on the units that will be assigned to the treatment. For example, in the case of a school-level intervention, multisite would include separate schools; in the case of an LEA-level intervention, multisite would include separate LEAs. For this reason, a definition of multisite that is limited to a specific level of implementation would be inaccurate.

Changes: None.

Comment: One commenter noted the difficulty of defining how projects can be “innovative and comprehensive in scope” and “show a cumulative effect over time” as specified in the NPP (74 FR 52216). The commenter stated that larger grants should not invest in innovations that are ineffective or that cannot be evaluated within the grant period. The commenter recommended that the Department consult with stakeholders to define what cumulative effects would mean in each area of a student’s growth. Another commenter noted that while narrowly focused programs may result in short-term gains, the relative efficacy of larger, macro-level efforts to engage stakeholders may require more time before the full impact is revealed. The commenter recommended that language be added to the notice to reflect this concern.

Discussion: The Department is interested in supporting projects with great potential to make meaningful improvements in students’ lives on a long term basis. For purposes of this grant program, however, project evaluations will only be able to detect impacts on outcomes measured during the grant period, and not on the longer-term outcomes on which programs may be focused. For this reason, eligible applicants for Scale-up and Validation grants will need to identify, in consultation with researchers and key stakeholders, intermediate outcomes directly correlated with the long-term outcomes of importance, on which their innovations are likely to have statistically significant effects before the grant period ends. Because of the variety of practices, strategies, and programs that we anticipate will be proposed by eligible applicants under the priorities identified by the Department, the definition of specific effects will need to be proposed separately by each eligible applicant, rather than specified in this notice.

Changes: None.

Comment: Several commenters proposed broadening the range of outcomes measures for which evidence of effectiveness would be documented. Several commenters highlighted the importance of technological and other skills needed for college attainment and success in the 21^st Century workplace, recommended that improved career readiness be added to the list of desired program outcomes in the selection criteria, and expressed concerns about limiting measures of performance to mathematics and to reading and language arts. One commenter recommended that student achievement in subjects such as science, civics, the arts, and the impact of school climate, school attendance, attendance rates, and student engagement on school achievement and graduation rates should also be measured.

Discussion: By placing such importance on student achievement, student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, and increasing college enrollment and completion rates, the Department is emphasizing the attainment of those skills and skill levels, and the conditions that contribute to attaining those skills, that are critical for student success in school and in careers. Eligible applicants can propose other outcomes if they contribute to the outcomes identified in this notice.

Changes: None.

Comment: A few commenters suggested that applicants adhering explicitly to research-based principles and findings should be considered for funding under this program. One commenter specifically suggested that applicants relying on the Department’s own research compendia (the What Works Clearinghouse, the Doing What Works website, and the Institute of Education Sciences reports) should be considered “pre-qualified” to meet the research evidence requirements in the notice.

Discussion: Research studies or reports released by the Department can be included as evidence of effectiveness for the practices, strategies, or programs proposed for funding under this program. However, whether such studies constitute strong or moderate evidence of effectiveness depends not only on the internal and external validity of the studies, but also on the correspondence between the practices, strategies, or programs proposed by the eligible applicant and the practices, strategies, and programs included in the reports released by the Department. The evidence of effectiveness documented in reports released by the Department varies in strength. Measured according to the criteria summarized in Table 1, this evidence would not necessarily qualify as strong or even moderate evidence of effectiveness for those innovations that an applicant would propose to implement in particular settings. Therefore, it would not be appropriate to pre-qualify eligible applicants, as recommended by the commenter.

Changes: None.

Comment: Several commenters expressed concern about the quality of child assessments that would be included in projects supported under this program. One commenter argued that the Department should support applicants that use multiple measures of developmental and academic outcomes for children. Other commenters criticized the emphasis the Department placed in the NPP on State-developed formative and interim assessments. These commenters argued instead for curriculum-embedded formative and summative assessments aligned with college-ready standards. Other commenters argued that reliance on the existing State and local formative and summative assessments would be more relevant to practice, less time-consuming, less disruptive of student learning time, and less expensive than relying on formal, in-depth standardized assessments for research purposes. One commenter noted that, for projects previously implemented and evaluated in multiple States, it would be reasonable to expect a nationally-normed standardized assessment to be used instead of State tests. The commenter recommended that, in a “large, well-designed and well-implemented randomized controlled, multisite trial, that effectiveness of the practice, strategy, or program” include a “randomized controlled, multi-state trial that uses a nationally-normed standardized assessment that is valid and reliable” for purposes of the Scale-up grants and the demonstration of improved student achievement. Another commenter questioned the validity of State achievement measures and recommended the priority consider assessments that are not tied to AYP determinations.

Discussion: This notice requires using State assessments for those grades and subjects assessed under section 1111(b)(3) of the ESEA to measure student achievement, and also permits alternative measures of student learning and performance to be used, especially for non-tested grades and subjects. Examples of these alternative measures include interim assessments or formative, classroom-based assessments. In projects spanning multiple States, commonality of measures of student learning and performance across all relevant grades and subjects is desirable, so a nationally-normed standardized assessment that is valid and reliable would be a reasonable measure of project performance. The Department’s intent is to contribute to improvements both in the reliability and validity of student assessments and in how these data are used to improve instruction for each student, not to add to the burden of schools in assessing students.

Changes: None.

Comment: One commenter argued that the Department should require projects proposed for Scale-up and Validation grants to study, for at least one year in at least 10 schools, the effects of the proposed project on student outcomes using measures other than those inherent to the treatments.

Discussion: The Department appreciates this suggestion to promote the external validity of findings from Scale-up and Validation projects. Because of the range of projects that could be supported by this program, the Department believes that decisions regarding the minimum sample size, the length of the study, or the choice of assessment measures should be made by the grantee according to the type of project being proposed.

Changes: None.

Comment: One commenter expressed concern about how to define treatment conditions and to aggregate data across schools or over time when different schools implement different innovations that also change over time. The commenter recommended that researchers with expertise in small-scale statistics provide guidance to support claims of effectiveness. The commenter also recommended that the Department provide incentives for LEAs to release student performance data (with appropriate privacy protections) that could be utilized in quasi-experimental analyses that would compare school outcomes.

Discussion: Elsewhere in this notice, we provide references to information and guidance that eligible applicants can use to support claims of effectiveness. An individual project’s evaluations should include the information needed to replicate or test the project in other settings. This information can include data on corresponding student outcomes, if appropriate privacy protections are in place.

Changes: None.

Comment: Some commenters argued that a sufficiently large effect size should be required by the Department, especially for the Scale-up and Validation grants. A few commenters argued that the Department should specify a 0.20 minimum effect size as a threshold for identifying educationally significant effects on student achievement. Many other commenters argued against using a single minimum effect size, and recommended instead that the Department evaluate the effect size, as reported by applicants in the context of the type of intervention, target population, outcomes being measured, and the existing research on anticipated effects. Some commenters argued that the expected effect size should differ by grant year and should be valued according to its long-term benefit to students.

Discussion: The Department appreciates the comments we received in response to our request for input on whether we should set a minimum effect size for this program. We are compelled by the arguments from commenters that a one-size-fits-all effect size would not be appropriate for this program given that the target effect size for a given practice, strategy, or program can vary because of factors such as the age and grade of the children receiving services, the nature of the outcome variable, and the cost of the innovation. Accordingly, eligible applicants should justify their claims regarding which magnitude of effect is reasonable and substantively important for their proposed project. Because the Department has decided to not specify a single effect size, eligible applicants are free to specify an anticipated effect size that differs by year for each year that would be included in the project evaluation. The specification of the anticipated effect sizes should be informed by the evidence of effectiveness for the innovation. Any differences from previously documented effect sizes should be discussed, particularly in the case of Scale-up grants for which evidence of effectiveness should be strong.

Changes: None.

Comment: Several commenters stated that any effect size standards adopted by the Department should take into account both the program costs and the anticipated effect size per unit cost in order to promote cost-effective innovations.

Discussion: Although cost is an important consideration when interpreting the importance of an effect size, the Department believes that the cost information will be more useful to reviewers of applications--and ultimately to researchers, practitioners, policymakers, and the public--if it is reported separately from the effect sizes, especially for innovations targeting multiple outcomes.

Changes: None.

Comment: Several commenters disagreed on the usefulness of intermediate outcome variables such as school attendance, parental engagement, teacher satisfaction, or school climate. Some commenters expressed concern that focusing on intermediate variables would detract from student achievement or attainment. In contrast, other commenters argued that removing or “downgrading” intermediate outcome would ignore research on the relationship between these outcomes and student achievement and attainment.

Discussion: Because of the limited time period of the Department’s innovation grants, the Department believes that it may be necessary for eligible applicants to identify and target key intermediate outcomes in order to understand the impact of projects in the short term. The Department believes that eligible applicants should carefully select intermediate outcomes that have a strong theoretical basis and empirical evidence of their direct connection with long-term student outcomes. Eligible applicants should collect data on intermediate outcomes only when data collection on longer-term outcomes is not feasible.

Changes: We are revising the selection criterion to clarify that an applicant choosing to demonstrate success through an intermediate variable must use an intermediate variable that is strongly correlated with the proposed project’s long-term student outcomes.

Comment: One commenter recommended that Validation grants support proposed practices, strategies, or programs for which there is a statistically significant association between the innovation and an intermediate variable that is highly correlated with the outcomes of interest. Another commenter stated that intermediate outcomes were needed because of validity and reliability issues with assessing the learning of children between birth and the third grade. One commenter argued that gathering data on “secondary effects” is also useful in understanding a project’s impact. A few commenters emphasized the need for evidence that the intermediate variables targeted by projects truly impact, and are not merely correlated with, student outcomes of importance, and that such a causal connection should have both theoretical and empirical support. Another commenter argued that it was important that any intermediate measures be reliable predictors of student learning outcomes, and that the learning outcomes be aligned with State standards and the range of skills and engagement predictive of student success. The commenter expressed concern that the Department support projects that provide clear presentations of the context and populations for which the effectiveness of supported innovations is being measured.

Discussion: The Department believes that defining intermediate outcome variables is necessary because of the limited duration of the grants provided under this program and because not all long-term outcomes targeted by projects will be measurable during the grant period. On the basis of the research evidence, eligible applicants should specify intermediate outcomes that are likely to be affected by the proposed practices, strategies, and programs, and that contribute to, or at least predict, improvements in the longer-term outcomes identified by the Department. “Secondary effects” estimates should be held to the same standards of evidence as effectiveness on long-term outcomes. However, strong evidence of effects on secondary, intermediate outcomes does not, in itself, constitute evidence of effects on long-term outcomes with which the secondary outcomes are correlated.

Changes: None.

Comment: Several commenters requested that the Department: (1) provide clarification of the meaning of “school climate” and its relationship to personal safety, gang presence, or drug presence; (2) list “family engagement” as an intermediate outcome alongside school climate; and (3) specify that the list of variables is not exhaustive.

Discussion: By listing examples of intermediate outcomes directly correlated with longer-term student outcomes, the Department left open the possibility of eligible applicants proposing other intermediate outcomes, including family engagement. The types of intermediate outcomes proposed by an eligible applicant, and the specific measures used for a variable, would depend on the type of practice, strategy, or program being proposed, the long-term student outcomes being targeted, and the settings in which the innovation would be implemented.

The Department does not wish to privilege some types of innovations over others by specifying a detailed list of intermediate outcome measures. Therefore, we are removing “improvements in school climate” as an example of an intermediate variable because we find it is not necessary to the effective use of the selection criterion.

Changes: We are removing “improvements in school climate” as an example of an intermediate variable in the selection criterion.

Comment: One commenter asked for clarification of the meaning of an effect that has a magnitude that is “substantial and important.”

Discussion: The meaning of “substantial and important” will vary depending on the context, such as the age and grade level of the students being served, and the cost of the innovation. Eligible applicants should describe why the expected effects are substantial and important for attaining the goals of this program.

Changes: None.

Comment: One commenter recommended that the reference to the “significance” of an effect for Development grants be changed to read “statistical significance.” Another commenter recommended changing “statistically significant” to “significant” when discussing the strength of research evidence regarding innovation effectiveness.

Discussion: Development grants may not always support innovations implemented on a scale that would produce statistically significant effects, so the omission of the adjective “statistical” is intentional. The NPP and this notice refer to “statistically significant” with regard to the significance of the effect that a practice, strategy, or program is expected to have if supported through a Scale-up or Validation grant. The magnitude of effect reported in prior studies--whether statistically significant or not--should support an eligible applicant’s claim that the effect of the practice, strategy, or program is likely to be detected as statistically significant in the sample included in the proposed Scale-up or Validation project. Small sample sizes in prior studies make the detection of statistically significant effects less likely and also weaken the external validity of findings, reducing the likelihood of the findings qualifying as the strong evidence required for a Scale-up grant. (Applicants should refer to Table 1 and its detailed summary of evidence criteria for the three types of grants.)

Changes: None.

Comment: Some commenters asked that “promising results” be changed to “positive results” in this final notice.

Discussion: “Promising results” refers, in the context of Development grants, to outcomes from practices, strategies, or programs for which there is not yet even moderate evidence of effectiveness. “Positive results” refers more generally to outcomes or goals consistent with the goal of the project, and encompasses both promising results suggesting that more formal and systematic study of efficacy may be warranted, and results qualifying as moderate or strong evidence of effectiveness. “Promising results” therefore is the more appropriate term for the Department to use in describing Development grants.

Changes: None.

Comment: A few commenters asked whether an applicant must name an independent evaluator in its application. Commenters asked whether it would be sufficient for an applicant to budget for an independent evaluator for only Scale-up and Validation grants. One commenter expressed concern that there is no capacity to have independent evaluators in place prior to a grant award.

Discussion: The quality of the evaluation proposed for each project, including the methods of evaluation planned and the resources proposed for evaluation, will be considered by the Department when awarding grants under this program. Whether an independent evaluator has been selected at the time of application will not, in itself, disadvantage an applicant. Applications should include the name of qualified independent evaluators of projects, if these have already been selected, and should in all cases demonstrate the applicant’s commitment to ensure a high-quality and independent evaluation of the proposed project.

Changes: None.

Comment: One commenter recommended that the Department offer technical assistance to grantees and their evaluators to ensure that high-quality independent evaluations are conducted of projects funded under all three types of grants. A few commenters asked the Department to explore how the required evaluations of funded projects can occur in an independent and statistically valid manner and the results collected, analyzed, and disseminated in a coordinated way that builds both stakeholder knowledge and the capacity of State and LEA evaluators. Several commenters emphasized the need for the Department to ensure rigorous, independent evaluations, scientific reporting, and the sharing of data on the effectiveness of grantee interventions. The commenters suggested that the Department require applications to include information about how project participants will support and cooperate with the independent evaluator, and use experimental or quasi-experimental methods where feasible. A few commenters expressed concern that the independent evaluation of a grantee’s project not be duplicative of the evaluation work submitted in the application.

Discussion: The Department’s Institute of Education Sciences (IES) will be involved in evaluating the Investing in Innovation program, in providing technical assistance to evaluators of individual funded projects, and in synthesizing evidence from multiple supported projects. The IES role will be defined in a way that will not duplicate the individual project evaluations under this program and that also encourages the independent evaluators to add to existing knowledge on the efficacy and effectiveness of the innovations being studied. Data will be collected and maintained by grantees. However, we agree with commenters that it is valuable to share the data from these evaluations. Thus, the data from the evaluations of Scale-up and Validation projects must be made available to third-party researchers. To support the sharing of data with third parties, the Department will work with grantees to set up procedures to make data available to other researchers while safeguarding privacy.

Changes: We are revising the Evaluation requirement under this program to specify that, in addition to making the results of any evaluation broadly available, Scale-up and Validation grantees must also ensure that the data from their evaluations are made available to third-party researchers consistent with applicable privacy requirements.

Comment: Commenters disagreed on the prioritization for experimental designs for the evaluations of grantee projects. One commenter argued that the evaluation requirements, not only for Scale-up grants, but also for Validation grants, should, wherever feasible, be experimental studies led by independent evaluators experienced with such studies. One commenter agreed with our proposal in the NPP specifying that Scale-up grants be evaluated by experimental or quasi-experimental means. Another commenter argued that evaluating Scale-up grants experimentally may not be feasible because of the lack of a control group and may not be necessary if the evidence for the innovation is sufficiently strong to scale it up.

Discussion: The evaluation requirements for Scale-up and Validation grants specify the use of independent evaluators and well-designed experimental or quasi-experimental studies. Because Validation grants would need to be supported by only moderate evidence, a large, well-implemented quasi-experimental evaluation may be sufficient to expand knowledge of the program’s effectiveness. Because Scale-up grants would already be supported by strong evidence, an experimental evaluation is preferable, when feasible, to assess how and under what conditions the program is effective when it is implemented in a fuller range of settings than prior to the awarding of the grant. Control or comparison groups can be identified for Scale-up projects from sites that have not yet implemented the innovation.

Changes: None.

Comment: Some commenters expressed concerns that the costs of evaluation could leave too few funds available to support implementation of the innovation, not only in the case of Development grants, but also in the case of Validation and Scale-up grants.

Discussion: Applicants should budget appropriate amounts for the evaluation of their project. The use of available data and measures that LEAs and schools already collect can help minimize new data collection costs and ensure that the innovations themselves are funded adequately. Evaluation dollars are well spent if they inform future decisions about whether to implement particular innovations more broadly.

Changes: None.

Comment: A few commenters emphasized the importance of measuring the extent and the quality of the implementation of grantee innovations, as well as on providing sufficient information to facilitate replication or testing of the innovation in other settings.

Discussion: We appreciate the commenters’ emphasis on the importance of evaluating grantee innovations. The measurement of program implementation and provision of information to facilitate replication or testing in other settings are required under the evaluations that will be conducted of each project funded under this program. This information will be especially important for understanding whether and under what circumstances innovations are implemented with fidelity.

Changes: None.

Comment: One commenter argued that program developers and implementers should be involved in evaluating the project, but should not be the sole evaluators. Another commenter argued that neither developers nor implementers should evaluate the impact of the project.

Discussion: The impact evaluation of Validation and Scale-up projects must be conducted by a qualified evaluator distinct from the program developer and project implementer. An autonomous research or evaluation office within a large organization could qualify as an independent evaluator if its reporting of findings and conclusions is not subject to approval by the office responsible for developing or implementing the program. In this way, impact evaluations of these projects would be independent, objective, and of greater use to all stakeholders. The Department encourages independent evaluators to consult with developers and implementers about knowledge that would inform evaluation design and reporting. For Development projects, developers and implementers can participate in the evaluation if they are qualified to do so because such participation may be necessary for the innovations to be implemented with fidelity as part of a small-scale study of efficacy.

Changes: None.

Comment: One commenter recommended that the Department pay LEAs for the cost of staff time that would be associated with the implementation of the evaluation.

Discussion: The cost of LEA staff time associated with the implementation of project evaluations can be included in each applicant’s evaluation budget.

Changes: None.

Comment: One commenter recommended that the Department provide funding for robust research studies and for a clearinghouse to describe the funded innovations.

Discussion: Under the requirements for this program, any eligible applicant receiving funds must conduct an independent evaluation of its proposed project and comply with the requirements of any evaluation of the program conducted by the Department (see Evaluation requirement). Therefore, the cost of the evaluation may be included in the applicant’s budget for its proposed project. The existing What Works Clearinghouse at IES is funded to review and synthesize evidence of effectiveness from education practices, strategies, or programs, including those that may be supported with Investing in Innovation grants.

Changes: None.

Comment: One commenter recommended that the Department treat data systems as measurement infrastructure for evaluating the effectiveness of other interventions, rather than as a separate intervention that is subject to the evidentiary standards for Scale-up grants.

Discussion: While data systems can be part of the measurement infrastructure for other interventions, the Department does not want to preclude the possibility of an applicant proposing a data system as a separate intervention.

Changes: None.

Comment: One commenter recommended that Development grants only be funded if there is a clear theory of action and if the associated research literature suggests that the hypothesized action on the intended outcome is likely to occur.

Discussion: This notice already specifies that applicants for Development grants provide a rationale for the proposed practice, strategy, or program that is based on research findings or reasonable hypotheses, including related research or theories in education or other sectors. Therefore, it is not necessary to add the requirement recommended by the commenter.

Changes: None.

Comment: A number of commenters expressed concerns about the application of evidence standards to small LEAs and rural LEAs. A few commenters expressed concern about the difficulty of small LEAs qualifying for Scale-up or Validation grants under the proposed priorities given the evidence requirements for applicants and the time that would be required to serve 100,000 or 250,000 students. One commenter recommended that the Department provide a competitive preference priority to applications where regional partnerships have been identified to scale up practices across schools and LEAs. Another commenter recommended that applicants be required to address “the limited human, fiscal, and technology capacity of rural LEAs and schools to collect data on the innovation and for independent evaluation.”

Discussion: The Department recognizes the particular challenges faced by small LEAs and rural LEAs in implementing and evaluating innovations. According to the evidence criteria described in Table 1, it may be possible, under the category of Validation grant funding, for rural LEAs to apply for funding to implement innovations with evidence of effectiveness in non-rural settings, since this evidence could have high internal validity but only moderate external validity. Challenges faced by rural LEAs in the areas of data collection and evaluation may be addressed by applicants applying under Competitive Preference Priority 8.

Changes: None.

Comment: One commenter recommended removing the reference to the What Works Clearinghouse procedures, standards, and technical methods papers because the commenter thought this reference was too limited.

Discussion: Knowledge of the What Works Clearinghouse procedures, standards, and technical methods papers may be useful to applicants in developing their project evaluation plans, but these resources are meant to be informative, not prescriptive, of evaluation decisions. Accordingly, we decline to remove that reference.

Changes: None.

FINAL PRIORITIES:

Types of Priorities

When inviting applications for a competition using one or more priorities, we designate the type of each priority as absolute, competitive preference, or invitational. Under an absolute priority, as specified by 34 CFR 75.105(c)(3), we consider only applications that meet the priority. Under a competitive preference priority, we give competitive preference to an application by (1) awarding additional points, depending on the extent to which the application meets the priority (34 CFR 75.105(c)(2)(i)); or (2) selecting an application that meets the priority over an application of comparable merit that does not meet the priority (34 CFR 75.105(c)(2)(ii)). With an invitational priority, we signal our interest in receiving applications that meet the priority; however, consistent with 34 CFR 75.105(c)(1), we do not give an application that meets an invitational priority preference over other applications.

Final Priorities

The Secretary establishes the following priorities for the Investing in Innovation Fund. We may apply these priorities in any year in which this program is in effect.

Absolute Priorities

Absolute Priority 1--Innovations that Support Effective Teachers and Principals.

Under this priority, the Department provides funding to support practices, strategies, or programs that are designed to increase the number or percentages of teachers or principals who are highly effective teachers or principals or reduce the number or percentages of teachers or principals who are ineffective, especially for teachers of high-need students, by identifying, recruiting, developing, placing, rewarding, and retaining highly effective teachers or principals (or removing ineffective teachers or principals). In such initiatives, teacher or principal effectiveness should be determined through an evaluation system that is rigorous, transparent, and fair; performance should be differentiated using multiple rating categories of effectiveness; multiple measures of effectiveness should be taken into account, with data on student growth as a significant factor; and the measures should be designed and developed with teacher and principal involvement.

Absolute Priority 2--Innovations that Improve the Use of Data.

Under this priority, the Department provides funding to support strategies, practices, or programs that are designed to (a) encourage and facilitate the evaluation, analysis, and use of student achievement or student growth data by educators, families, and other stakeholders in order to inform decision-making and improve student achievement, student growth, or teacher, principal, school, or LEA performance and productivity; or (b) enable data aggregation, analysis, and research. Where LEAs and schools are required to do so under the Elementary and Secondary Education Act of 1965, as amended (ESEA), these data must be disaggregated using the student subgroups described in section 1111(b)(3)(C)(xiii) of the ESEA (i.e., economically disadvantaged students, students from major racial and ethnic groups, migrant students, students with limited English proficiency, students with disabilities, and student gender).

Absolute Priority 3--Innovations that Complement the Implementation of High Standards and High-Quality Assessments.

Under this priority, the Department provides funding for practices, strategies, or programs that are designed to support States’ efforts to transition to standards and assessments that measure students’ progress toward college- and career-readiness, including curricular and instructional practices, strategies, or programs in core academic subjects (as defined in section 9101(11) of the ESEA) that are aligned with high academic content and achievement standards and with high-quality assessments based on those standards.⁸ Proposed projects may include, but are not limited to, practices, strategies, or programs that are designed to: (a) increase the success of under-represented student populations in academically rigorous courses and programs (such as Advanced Placement or International Baccalaureate courses; dual-enrollment programs; “early college high schools;” and science, technology, engineering, and mathematics courses, especially those that incorporate rigorous and relevant project-, inquiry-, or design-based contextual learning opportunities); (b) increase the development and use of formative assessments or interim assessments, or other performance-based tools and “metrics” that are aligned with high student content and academic achievement standards; or (c) translate the standards and information from assessments into classroom practices that meet the needs of all students, including high-need students.

Under this priority, an eligible applicant must propose a project that is based on standards that are at least as rigorous as its State’s standards. If the proposed project is based on standards other than those adopted by the eligible applicant’s State, the applicant must explain how the standards are aligned with and at least as rigorous as the eligible applicant’s State’s standards as well as how the standards differ.

Absolute Priority 4--Innovations that Turn Around Persistently Low-Performing Schools.

Under this priority, the Department provides funding to support strategies, practices, or programs that are designed to turn around schools that are in any of the following categories: (a) persistently lowest-achieving schools (as defined in the final requirements for the School Improvement Grants program)⁹; (b) Title I schools that are in corrective action or restructuring under section 1116 of the ESEA; or (c) secondary schools (both middle and high schools) eligible for but not receiving Title I funds that, if receiving Title I funds, would be in corrective action or restructuring under section 1116 of the ESEA. These schools are referred to as Investing in Innovation Fund Absolute Priority 4 schools.

Proposed projects must include strategies, practices, or programs that are designed to turn around Investing in Innovation Fund Absolute Priority 4 schools through either whole-school reform or targeted approaches to reform. Applicants addressing this priority must focus on either:

(a) Whole-school reform, including, but not limited to, comprehensive interventions to assist, augment, or replace Investing in Innovation Fund Absolute Priority 4 schools, including the school turnaround, restart, closure, and transformation models of intervention supported under the Department’s School Improvement Grants program (see Final Requirements for School Improvement Grants as Amended in January 2010 (January 28, 2010) at http://www2.ed.gov/programs/sif/faq.html); or

(b) Targeted approaches to reform, including, but not limited to: (1) providing more time for students to learn core academic content by expanding or augmenting the school day, school week, or school year, or by increasing instructional time for core academic subjects (as defined in section 9101(11) of the ESEA); (2) integrating “student supports” into the school model to address non-academic barriers to student achievement; or (3) creating multiple pathways for students to earn regular high school diplomas (e.g., by operating schools that serve the needs of over-aged, under-credited, or other students with an exceptional need for support and flexibility pertaining to when they attend school; awarding credit based on demonstrated evidence of student competency; and offering dual-enrollment options).

Competitive Preference Priorities

Competitive Preference Priority 5--Innovations for Improving Early Learning Outcomes.

We give competitive preference to applications for projects that would implement innovative practices, strategies, or programs that are designed to improve educational outcomes for high-need students who are young children (birth through 3^rd grade) by enhancing the quality of early learning programs. To meet this priority, applications must focus on (a) improving young children’s school readiness (including social, emotional, and cognitive readiness) so that children are prepared for success in core academic subjects (as defined in section 9101(11) of the ESEA); (b) improving developmental milestones and standards and aligning them with appropriate outcome measures; and (c) improving alignment, collaboration, and transitions between early learning programs that serve children from birth to age three, in preschools, and in kindergarten through third grade.

Competitive Preference Priority 6--Innovations that Support College Access and Success.

We give competitive preference to applications for projects that would implement innovative practices, strategies, or programs that are designed to enable kindergarten through grade 12 (K-12) students, particularly high school students, to successfully prepare for, enter, and graduate from a two- or four-year college. To meet this priority, applications must include practices, strategies, or programs for K-12 students that (a) address students’ preparedness and expectations related to college; (b) help students understand issues of college affordability and the financial aid and college application processes; and (c) provide support to students from peers and knowledgeable adults.

Competitive Preference Priority 7–-Innovations to Address the Unique Learning Needs of Students with Disabilities and Limited English Proficient Students.

We give competitive preference to applications for projects that would implement innovative practices, strategies, or programs that are designed to address the unique learning needs of students with disabilities, including those who are assessed based on alternate academic achievement standards, or the linguistic and academic needs of limited English proficient students. To meet this priority, applications must provide for the implementation of particular practices, strategies, or programs that are designed to improve academic outcomes, close achievement gaps, and increase college- and career-readiness, including increasing high school graduation rates (as defined in this notice), for students with disabilities or limited English proficient students.

Competitive Preference Priority 8--Innovations that Serve Schools in Rural LEAs.

We give competitive preference to applications for projects that would implement innovative practices, strategies, or programs that are designed to focus on the unique challenges of high-need students in schools within a rural LEA (as defined in this notice) and address the particular challenges faced by students in these schools. To meet this priority, applications must include practices, strategies, or programs that are designed to improve student achievement or student growth, close achievement gaps, decrease dropout rates, increase high school graduation rates, or improve teacher and principal effectiveness in one or more rural LEAs.

FINAL REQUIREMENTS

The Secretary establishes the following requirements for the Investing in Innovation Fund. We may apply these requirements in any year in which this program is in effect.

Providing Innovations that Improve Achievement for High-Need Students: All eligible applicants must implement practices, strategies, or programs for high-need students (as defined in this notice).

Eligible Applicants: Entities eligible to apply for Investing in Innovation Fund grants include: (a) an LEA or (b) a partnership between a nonprofit organization and (1) one or more LEAs or (2) a consortium of schools. An eligible applicant that is a partnership applying under section 14007(a)(1)(B) of the ARRA must designate one of its official partners (as defined in this notice) to serve as the applicant in accordance with the Department’s regulations governing group applications in 34 CFR 75.127 through 75.129.

Eligibility Requirements: To be eligible for an award, an eligible applicant must--except as specifically set forth in the Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization that follows:

(1)(A) Have significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA (economically disadvantaged students, students from major racial and ethnic groups, students with limited English proficiency, students with disabilities); or

(B) Have demonstrated success in significantly increasing student academic achievement for all groups of students described in that section;

(2) Have made significant improvements in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and principals, as demonstrated with meaningful data;

(3) Demonstrate that it has established one or more partnerships with the private sector, which may include philanthropic organizations, and that the private sector will provide matching funds in order to help bring results to scale; and

(4) In the case of an eligible applicant that includes a nonprofit organization, provide in the application the names of the LEAs with which the nonprofit organization will partner, or the names of the schools in the consortium with which it will partner. If an eligible applicant that includes a nonprofit organization intends to partner with additional LEAs or schools that are not named in the application, it must describe in the application the demographic and other characteristics of these LEAs and schools and the process it will use to select them as either official or other partners. An applicant must identify its specific partners before a grant award will be made.

Note about LEA Eligibility: For purposes of this program, an LEA is an LEA located within one of the 50 States, the District of Columbia, or the Commonwealth of Puerto Rico.

Note about Eligibility for an Eligible Applicant that Includes a Nonprofit Organization: The authorizing statute (as amended) specifies that an eligible applicant that includes a nonprofit organization is considered to have met the requirements in paragraphs (1) and (2) of the eligibility requirements for this program if the nonprofit organization has a record of significantly improving student achievement, attainment, or retention. For an eligible applicant that includes a nonprofit organization, the nonprofit organization must demonstrate that it has a record of significantly improving student achievement, attainment, or retention through its record of work with an LEA or schools. Therefore, an eligible applicant that includes a nonprofit organization does not necessarily need to include as a partner for its Investing in Innovation Fund grant an LEA or a consortium of schools that meets the requirements in paragraphs (1) and (2).

In addition, the authorizing statute (as amended) specifies that an eligible applicant that includes a nonprofit organization is considered to have met the requirements of paragraph (3) of the eligibility requirements for this program if the eligible applicant demonstrates that it will meet the requirement relating to private-sector matching.

Evidence Standards: To be eligible for an award, an application for a Scale-up grant must be supported by strong evidence (as defined in this notice), an application for a Validation grant must be supported by moderate evidence (as defined in this notice), and an application for a Development grant must be supported by a reasonable hypothesis.

Funding Categories: An applicant must state in its application whether it is applying for a Scale-up, Validation, or Development grant. An applicant may not submit an application for the same proposed project under more than one type of grant. An applicant will be considered for an award only for the type of grant for which it applies.

Cost Sharing or Matching: To be eligible for an award, an eligible applicant must demonstrate that it has established one or more partnerships with an entity or organization in the private sector, which may include philanthropic organizations, and that the entity or organization in the private sector will provide matching funds in order to help bring project results to scale. An eligible applicant must obtain matching funds or in-kind donations equal to at least 20 percent of its grant award. Selected eligible applicants must submit evidence of the full 20 percent private-sector matching funds following the peer review of applications. An award will not be made unless the applicant provides adequate evidence that the full 20 percent private-sector match has been committed or the Secretary approves the eligible applicant’s request to reduce the matching-level requirement.

The Secretary may consider decreasing the 20 percent matching requirement in the most exceptional circumstances, on a case-by-case basis. An eligible applicant that anticipates being unable to meet the 20 percent matching requirement must include in the application a request to the Secretary to reduce the matching-level requirement, along with a statement of the basis for the request.

Subgrants: In the case of an eligible applicant that is a partnership between a nonprofit organization and (1) one or more LEAs or (2) a consortium of schools, the partner serving as the applicant may make subgrants to one or more official partners (as defined in this notice).

Limits on Grant Awards: No grantee may receive more than two grant awards under this program. In addition, no grantee may receive more than $55 million in grant awards under this program in a single year’s competition.

Evaluation: A grantee must comply with the requirements of any evaluation of the program conducted by the Department. In addition, the grantee is required to conduct an independent evaluation (as defined in this notice) of its project and must agree, along with its independent evaluator, to cooperate with any technical assistance provided by the Department or its contractor. The purpose of this technical assistance will be to ensure that the evaluations are of the highest quality and to encourage commonality in evaluation approaches across funded projects where such commonality is feasible and useful. Finally, the grantee must make broadly available through formal (e.g., peer-reviewed journals) or informal (e.g., newsletters) mechanisms, and in print or electronically, the results of any evaluations it conducts of its funded activities. For Scale-up and Validation grants, the grantee must also ensure the data from their evaluations are made available to third-party researchers consistent with applicable privacy requirements.

Participation in “Communities of Practice”: Grantees are required to participate in, organize, or facilitate, as appropriate, communities of practice for the Investing in Innovation Fund. A community of practice is a group of grantees that agrees to interact regularly to solve a persistent problem or improve practice in an area that is important to them. Establishment of communities of practice under the Investing in Innovation Fund will enable grantees to meet, discuss, and collaborate with each other regarding grantee projects.

FINAL DEFINITIONS:

The Secretary establishes the following definitions for the Investing in Innovation Fund. We may apply these definitions in any year in which this program is in effect.

Definitions Related to Evidence

Strong evidence means evidence from previous studies whose designs can support causal conclusions (i.e., studies with high internal validity), and studies that in total include enough of the range of participants and settings to support scaling up to the State, regional, or national level (i.e., studies with high external validity). The following are examples of strong evidence: (1) more than one well-designed and well-implemented (as defined in this notice) experimental study (as defined in this notice) or well-designed and well-implemented (as defined in this notice) quasi-experimental study (as defined in this notice) that supports the effectiveness of the practice, strategy, or program; or (2) one large, well-designed and well-implemented (as defined in this notice) randomized controlled, multisite trial that supports the effectiveness of the practice, strategy, or program.

Moderate evidence means evidence from previous studies whose designs can support causal conclusions (i.e., studies with high internal validity) but have limited generalizability (i.e., moderate external validity), or studies with high external validity but moderate internal validity. The following would constitute moderate evidence: (1) at least one well-designed and well-implemented (as defined in this notice) experimental or quasi-experimental study (as defined in this notice) supporting the effectiveness of the practice, strategy, or program, with small sample sizes or other conditions of implementation or analysis that limit generalizability; (2) at least one well-designed and well-implemented (as defined in this notice) experimental or quasi-experimental study (as defined in this notice) that does not demonstrate equivalence between the intervention and comparison groups at program entry but that has no other major flaws related to internal validity; or (3) correlational research with strong statistical controls for selection bias and for discerning the influence of internal factors.

Well-designed and well-implemented means, with respect to an experimental or quasi-experimental study (as defined in this notice), that the study meets the What Works Clearinghouse evidence standards, with or without reservations (see http://ies.ed.gov/ncee/wwc/references/idocviewer/doc.aspx?docid=19&tocid=1 and in particular the description of “Reasons for Not Meeting Standards” at http://ies.ed.gov/ncee/wwc/references/idocviewer/Doc.aspx?docId=19&tocId=4#reasons).

Experimental study means a study that employs random assignment of, for example, students, teachers, classrooms, schools, or districts to participate in a project being evaluated (treatment group) or not to participate in the project (control group). The effect of the project is the average difference in outcomes between the treatment and control groups.

Quasi-experimental study means an evaluation design that attempts to approximate an experimental design and can support causal conclusions (i.e., minimizes threats to internal validity, such as selection bias, or allows them to be modeled). Well-designed quasi-experimental studies include carefully matched comparison group designs (as defined in this notice), interrupted time series designs (as defined in this notice), or regression discontinuity designs (as defined in this notice).

Carefully matched comparison group design means a type of quasi-experimental study that attempts to approximate an experimental study. More specifically, it is a design in which project participants are matched with non-participants based on key characteristics that are thought to be related to the outcome. These characteristics include, but are not limited to: (1) prior test scores and other measures of academic achievement (preferably, the same measures that the study will use to evaluate outcomes for the two groups); (2) demographic characteristics, such as age, disability, gender, English proficiency, ethnicity, poverty level, parents’ educational attainment, and single- or two-parent family background; (3) the time period in which the two groups are studied (e.g., the two groups are children entering kindergarten in the same year as opposed to sequential years); and (4) methods used to collect outcome data (e.g., the same test of reading skills administered in the same way to both groups).

Interrupted time series design¹⁰ means a type of quasi-experimental study in which the outcome of interest is measured multiple times before and after the treatment for program participants only.  If the program had an impact, the outcomes after treatment will have a different slope or level from those before treatment.  That is, the series should show an “interruption” of the prior situation at the time when the program was implemented.  Adding a comparison group time series, such as schools not participating in the program or schools participating in the program in a different geographic area, substantially increases the reliability of the findings.

Regression discontinuity design study means, in part, a quasi-experimental study design that closely approximates an experimental study. In a regression discontinuity design, participants are assigned to a treatment or comparison group based on a numerical rating or score of a variable unrelated to the treatment such as the rating of an application for funding. Another example would be assignment of eligible students, teachers, classrooms, or schools above a certain score (“cut score”) to the treatment group and assignment of those below the score to the comparison group.

Independent evaluation means that the evaluation is designed and carried out independent of, but in coordination with, any employees of the entities who develop a practice, strategy, or program and are implementing it. This independence helps ensure the objectivity of an evaluation and prevents even the appearance of a conflict of interest.

Other Definitions

Applicant means the entity that applies for a grant under this program on behalf of an eligible applicant (i.e., an LEA or a partnership in accordance with section 14007(a)(1)(B) of the ARRA).

Official partner means any of the entities required to be part of a partnership under section 14007(a)(1)(B) of the ARRA.

Other partner means any entity, other than the applicant and any official partner, that may be involved in a proposed project.

Consortium of schools means two or more public elementary or secondary schools acting collaboratively for the purpose of applying for and implementing an Investing in Innovation Fund grant jointly with an eligible nonprofit organization.

Nonprofit organization means an entity that meets the definition of “nonprofit” under 34 CFR 77.1(c), or an institution of higher education as defined by section 101(a) of the Higher Education Act of 1965, as amended.

Formative assessment means assessment questions, tools, and processes that are embedded in instruction and are used by teachers and students to provide timely feedback for purposes of adjusting instruction to improve learning.

Interim assessment means an assessment that is given at regular and specified intervals throughout the school year, is designed to evaluate students’ knowledge and skills relative to a specific set of academic standards, and produces results that can be aggregated (e.g., by course, grade level, school, or LEA) in order to inform teachers and administrators at the student, classroom, school, and LEA levels.

Highly effective principal means a principal whose students, overall and for each subgroup as described in section 1111(b)(3)(C)(xiii) of the ESEA (i.e., economically disadvantaged students, students from major racial and ethnic groups, migrant students, students with disabilities, students with limited English proficiency, and students of each gender), achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth. Eligible applicants may include multiple measures, provided that principal effectiveness is evaluated, in significant part, based on student growth. Supplemental measures may include, for example, high school graduation rates; college enrollment rates; evidence of providing supportive teaching and learning conditions, support for ensuring effective instruction across subject areas for a well-rounded education, strong instructional leadership, and positive family and community engagement; or evidence of attracting, developing, and retaining high numbers of effective teachers.

Highly effective teacher means a teacher whose students achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth. Eligible applicants may include multiple measures, provided that teacher effectiveness is evaluated, in significant part, based on student growth. Supplemental measures may include, for example, multiple observation-based assessments of teacher performance or evidence of leadership roles (which may include mentoring or leading professional learning communities) that increase the effectiveness of other teachers in the school or LEA.

High-need student means a student at risk of educational failure, or otherwise in need of special assistance and support, such as students who are living in poverty, who attend high-minority schools, who are far below grade level, who are over-age and under-credited, who have left school before receiving a regular high school diploma, who are at risk of not graduating with a regular high school diploma on time, who are homeless, who are in foster care, who have been incarcerated, who have disabilities, or who are limited English proficient.

National level, as used in reference to a Scale-up grant, describes a project that is able to be effective in a wide variety of communities and student populations around the country, including rural and urban areas, as well as with the different groups of students described in section 1111(b)(3)(C)(xiii) of the ESEA (i.e., economically disadvantaged students, students from major racial and ethnic groups, migrant students, students with disabilities, students with limited English proficiency, and students of each gender).

Regional level, as used in reference to a Scale-up or Validation grant, describes a project that is able to serve a variety of communities and student populations within a State or multiple States, including rural and urban areas, as well as with the different groups of students described in section 1111(b)(3)(C)(xiii) of the ESEA (i.e., economically disadvantaged students, students from major racial and ethnic groups, migrant students, students with disabilities, students with limited English proficiency, and students of each gender). To be considered a regional-level project, a project must serve students in more than one LEA. The exception to this requirement would be a project implemented in a State in which the State educational agency is the sole educational agency for all schools and thus may be considered an LEA under section 9101(26) of the ESEA. Such a State would meet the definition of regional for the purposes of this notice.

Rural LEA means an LEA that is eligible under the Small Rural School Achievement (SRSA) program or the Rural and Low-Income School (RLIS) program authorized under Title VI, Part B of the ESEA. Eligible applicants may determine whether a particular LEA is eligible for these programs by referring to information on the following Department Web sites. For the SRSA: www.ed.gov/programs/reapsrsa/eligible09/index.html. For the RLIS: www.ed.gov/programs/reaprlisp/eligibility.html.

Student achievement means—

(a) For tested grades and subjects: (1) a student’s score on the State’s assessments under section 1111(b)(3) of the ESEA; and, as appropriate, (2) other measures of student learning, such as those described in paragraph (b) of this definition, provided they are rigorous and comparable across classrooms; and

(b) For non-tested grades and subjects: alternative measures of student learning and performance such as student scores on pre-tests and end-of-course tests; student performance on English language proficiency assessments; and other measures of student achievement that are rigorous and comparable across classrooms.

Student growth means the change in student achievement data for an individual student between two or more points in time. Growth may be measured by a variety of approaches, but any approach used must be statistically rigorous and based on student achievement data, and may also include other measures of student learning in order to increase the construct validity and generalizability of the information.

High school graduation rate means a four-year adjusted cohort graduation rate consistent with 34 CFR 200.19(b)(1) and may also include an extended-year adjusted cohort graduation rate consistent with 34 CFR 200.19(b)(1)(v) if the State in which the proposed project is implemented has been approved by the Secretary to use such a rate under Title I of the ESEA.

Regular high school diploma means, consistent with 34 CFR 200.19(b)(1)(iv), the standard high school diploma that is awarded to students in the State and that is fully aligned with the State’s academic content standards or a higher diploma and does not include a General Education Development (GED) credential, certificate of attendance, or any alternative award.

SELECTION CRITERIA:

The Secretary establishes the following selection criteria for evaluating an application under the Investing in Innovation Fund. We may apply these criteria in any year in which this program is in effect. In the notice inviting applications, we will announce the maximum possible points assigned to each criterion.

1. Scale-up Grants.

A. Need for the Project and Quality of the Project Design.

The Secretary considers the need for the project and quality of the design of the proposed project.

In determining the need for the project and quality of the design of the proposed project, the Secretary considers the following factors:

(1) The extent to which the proposed project represents an exceptional approach to the priorities the eligible applicant is seeking to meet (i.e., addresses a largely unmet need, particularly for high-need students, and is a practice, strategy, or program that has not already been widely adopted).

(2) The extent to which the proposed project has a clear set of goals and an explicit strategy, with actions that are (a) aligned with the priorities the eligible applicant is seeking to meet, and (b) expected to result in achieving the goals, objectives, and outcomes of the proposed project.

B. Strength of Research, Significance of Effect, and Magnitude of Effect.

The Secretary considers the strength of the existing research evidence,¹¹ including the internal validity (strength of causal conclusions) and external validity (generalizability) of the effects reported in prior research, on whether the proposed project will improve student achievement or student growth, close achievement gaps, decrease dropout rates, increase high school graduation rates, or increase college enrollment and completion rates. Eligible applicants may also demonstrate success through an intermediate variable that is strongly correlated with improving these outcomes, such as teacher or principal effectiveness.

In determining the strength of the existing research evidence, the Secretary considers the following factors:

(1) The extent to which the eligible applicant demonstrates that there is strong evidence (as defined in this notice) that its implementation of the proposed practice, strategy, or program will have a statistically significant, substantial, and important effect on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates.

(2) The importance and magnitude of the effect expected to be obtained by the proposed project, including the extent to which the project will substantially and measurably improve student achievement or student growth, close achievement gaps, decrease dropout rates, increase high school graduation rates, or increase college enrollment and completion rates. The evidence in support of the importance and magnitude of the effect would be the research-based evidence provided by the eligible applicant to support the proposed project.

C. Experience of the Eligible Applicant.

The Secretary considers the experience of the eligible applicant in implementing the proposed project.

In determining the experience of the eligible applicant, the Secretary considers the following factors:

(1) The past performance of the eligible applicant in implementing large, complex, and rapidly growing projects.

(2) The extent to which an eligible applicant provides information and data demonstrating that--

(a) In the case of an eligible applicant that is an LEA, the LEA has--

(i) Significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or significantly increased student achievement for all groups of students described in such section; and

(ii) Made significant improvements in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and principals, as demonstrated with meaningful data; or

(b) In the case of an eligible applicant that includes a nonprofit organization, the nonprofit organization has significantly improved student achievement, attainment, or retention through its record of work with an LEA or schools.

D. Quality of the Project Evaluation.

The Secretary considers the quality of the evaluation to be conducted of the proposed project.

In determining the quality of the evaluation, the Secretary considers the following factors:

(1) The extent to which the methods of evaluation will include a well-designed experimental study or, if a well-designed experimental study of the project is not possible, the extent to which the methods of evaluation will include a well-designed quasi-experimental study.

(2) The extent to which, for either an experimental study or a quasi-experimental study, the study will be conducted of the practice, strategy, or program as implemented at scale.

(3) The extent to which the methods of evaluation will provide high-quality implementation data and performance feedback, and permit periodic assessment of progress toward achieving intended outcomes.

(4) The extent to which the evaluation will provide sufficient information about the key elements and approach of the project so as to facilitate replication or testing in other settings.

(5) The extent to which the proposed project plan includes sufficient resources to carry out the project evaluation effectively.

(6) The extent to which the proposed evaluation is rigorous, independent, and neither the program developer nor the project implementer will evaluate the impact of the project.

Note: We encourage eligible applicants to review the following technical assistance resources on evaluation: (1) What Works Clearinghouse Procedures and Standards Handbook: http://ies.ed.gov/ncee/wwc/references/idocviewer/doc.aspx?docid=19&tocid=1; and (2) IES/NCEE Technical Methods papers: http://ies.ed.gov/ncee/tech_methods/.

E. Strategy and Capacity to Bring to Scale.

The Secretary considers the quality of the eligible applicant’s strategy and capacity to bring the proposed project to scale on a national, regional, or State level.

In determining the quality of the strategy and capacity to bring the proposed project to scale, the Secretary considers:

(1) The number of students proposed to be reached by the proposed project and the capacity of the eligible applicant and any other partners to reach the proposed number of students during the course of the grant period.

(2) The eligible applicant’s capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to bring the proposed project to scale on a national, regional, or State level working directly, or through partners, either during or following the end of the grant period.

(3) The feasibility of the proposed project to be replicated successfully, if positive results are obtained, in a variety of settings and with a variety of student populations. Evidence of this ability includes the proposed project’s demonstrated success in multiple settings and with different types of students, the availability of resources and expertise required for implementing the project with fidelity, and the proposed project’s evidence of relative ease of use or user satisfaction.

(4) The eligible applicant’s estimate of the cost of the proposed project, which includes the start-up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project. The eligible applicant must include an estimate of the costs for the eligible applicant or others (including other partners) to reach 100,000, 500,000, and 1,000,000 students.

(5) The mechanisms the eligible applicant will use to broadly disseminate information on its project so as to support replication.

F. Sustainability.

The Secretary considers the adequacy of resources to continue the proposed project after the grant period ends.

In determining the adequacy of resources for the proposed project, the Secretary considers the following factors:

(1) The extent to which the eligible applicant demonstrates that it has the resources to operate the project beyond the length of the Scale-up grant, including a multi-year financial and operating model and accompanying plan; the demonstrated commitment of any other partners; and evidence of broad support from stakeholders (e.g., State educational agencies, teachers’ unions) critical to the project’s long-term success.

(2) The potential and planning for the incorporation of project purposes, activities, or benefits into the ongoing work of the eligible applicant and any other partners at the end of the Scale-up grant.

G. Quality of the Management Plan and Personnel.

The Secretary considers the quality of the management plan and personnel for the proposed project.

In determining the quality of the management plan and personnel for the proposed project, the Secretary considers:

(1) The adequacy of the management plan to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, timelines, and milestones for accomplishing project tasks, as well as tasks related to the sustainability and scalability of the proposed project.

(2) The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing large, complex, and rapidly growing projects.

(3) The qualifications, including relevant expertise and experience, of the project director and key personnel of the independent evaluator, especially in designing and conducting large-scale experimental and quasi-experimental studies of educational initiatives.

2. Validation Grants.

A. Need for the Project and Quality of the Project Design.

The Secretary considers the need for the project and quality of the design of the proposed project.

In determining the need for the project and quality of the design of the proposed project, the Secretary considers the following factors:

(1) The extent to which the proposed project represents an exceptional approach to the priorities the eligible applicant is seeking to meet (i.e., addresses a largely unmet need, particularly for high-need students, and is a practice, strategy, or program that has not already been widely adopted).

(2) The extent to which the proposed project has a clear set of goals and an explicit strategy, with actions that are (a) aligned with the priorities the eligible applicant is seeking to meet, and (b) expected to result in achieving the goals, objectives, and outcomes of the proposed project.

(3) The extent to which the proposed project is consistent with the research evidence supporting the proposed project, taking into consideration any differences in context.

B. Strength of Research, Significance of Effect, and Magnitude of Effect.

The Secretary considers the strength of the existing research evidence, including the internal validity (strength of causal conclusions) and external validity (generalizability) of the effects reported in prior research, on whether the proposed project will improve student achievement or student growth, close achievement gaps, decrease dropout rates, increase high school graduation rates, or increase college enrollment and completion rates. Eligible applicants may also demonstrate success through an intermediate variable that is strongly correlated with improving these outcomes, such as teacher or principal effectiveness.

In determining the strength of the existing research evidence,¹² the Secretary considers the following factors:

(1) The extent to which the eligible applicant demonstrates that there is moderate evidence (as defined in this notice) that the proposed practice, strategy, or program will have a statistically significant, substantial, and important effect on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates.

(2) The importance and magnitude of the effect expected to be obtained by the proposed project, including the likelihood that the project will substantially and measurably improve student achievement or student growth, close achievement gaps, decrease dropout rates, increase high school graduation rates, or increase college enrollment and completion rates. The evidence in support of the importance and magnitude of the effect would be the research-based evidence provided by the eligible applicant to support the proposed project.

C. Experience of the Eligible Applicant.

The Secretary considers the experience of the eligible applicant in implementing the proposed project.

In determining the experience of the eligible applicant, the Secretary considers the following factors:

(1) The past performance of the eligible applicant in implementing complex projects.

(2) The extent to which an eligible applicant provides information and data demonstrating that--

(a) In the case of an eligible applicant that is an LEA, the LEA has--

(i) Significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or significantly increased student achievement for all groups of students described in such section; and

(ii) Made significant improvements in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and principals, as demonstrated with meaningful data; or

(b) In the case of an eligible applicant that includes a nonprofit organization, the nonprofit organization has significantly improved student achievement, attainment, or retention through its record of work with an LEA or schools.

D. Quality of the Project Evaluation.

The Secretary considers the quality of the evaluation to be conducted of the proposed project.

In determining the quality of the evaluation, the Secretary considers the following factors:

(1) The extent to which the methods of evaluation will include a well-designed experimental study or well-designed quasi-experimental study.

(2) The extent to which the methods of evaluation will provide high-quality implementation data and performance feedback, and permit periodic assessment of progress toward achieving intended outcomes.

(3) The extent to which the evaluation will provide sufficient information about the key elements and approach of the project so as to facilitate replication or testing in other settings.

(4) The extent to which the proposed project plan includes sufficient resources to carry out the project evaluation effectively.

(5) The extent to which the proposed evaluation is rigorous, independent, and neither the program developer nor the project implementer will evaluate the impact of the project.

Note: We encourage eligible applicants to review the following technical assistance resources on evaluation: (1) What Works Clearinghouse Procedures and Standards Handbook: http://ies.ed.gov/ncee/wwc/references/idocviewer/doc.aspx?docid=19&tocid=1; and (2) IES/NCES Technical Methods papers: http://ies.ed.gov/ncee/tech_methods/.

E. Strategy and Capacity to Bring to Scale.

The Secretary considers the quality of the eligible applicant’s strategy and capacity to bring the proposed project to scale on a State or regional level.

In determining the quality of the strategy and capacity to bring the proposed project to scale, the Secretary considers:

(1) The number of students proposed to be reached by the proposed project and the capacity of the eligible applicant and any other partners to reach the proposed number of students during the course of the grant period.

(2) The eligible applicant’s capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to bring the proposed project to scale on a State or regional level (as appropriate, based on the results of the proposed project) working directly, or through other partners, either during or following the end of the grant period.

(3) The feasibility of the proposed project to be replicated successfully, if positive results are obtained, in a variety of settings and with a variety of student populations. Evidence of this ability includes the availability of resources and expertise required for implementing the project with fidelity, and the proposed project’s evidence of relative ease of use or user satisfaction.

(4) The eligible applicant’s estimate of the cost of the proposed project, which includes the start-up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project. The eligible applicant must include an estimate of the costs for the eligible applicant or others (including other partners) to reach 100,000, 250,000, and 500,000 students.

(5) The mechanisms the eligible applicant will use to broadly disseminate information on its project to support further development, expansion, or replication.

F. Sustainability.

The Secretary considers the adequacy of resources to continue to develop the proposed project.

In determining the adequacy of resources for the proposed project, the Secretary considers the following factors:

(1) The extent to which the eligible applicant demonstrates that it has the resources, as well as the support of stakeholders (e.g., State educational agencies, teachers’ unions), to operate the project beyond the length of the Validation grant.

(2) The potential and planning for the incorporation of project purposes, activities, or benefits into the ongoing work of the eligible applicant and any other partners at the end of the Validation grant.

G. Quality of the Management Plan and Personnel.

The Secretary considers the quality of the management plan and personnel for the proposed project.

In determining the quality of the management plan and personnel for the proposed project, the Secretary considers:

(1) The adequacy of the management plan to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, timelines, and milestones for accomplishing project tasks, as well as tasks related to the sustainability and scalability of the proposed project.

(2) The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing complex projects.

(3) The qualifications, including relevant expertise and experience, of the project director and key personnel of the independent evaluator, especially in designing and conducting experimental and quasi-experimental studies of educational initiatives.

3. Development Grants.

A. Need for the Project and Quality of the Project Design.

The Secretary considers the need for the project and quality of the design of the proposed project.

In determining the need for the project and quality of the design of the proposed project, the Secretary considers the following factors:

(1) The extent to which the proposed project represents an exceptional approach to the priorities the eligible applicant is seeking to meet (i.e., addresses a largely unmet need, particularly for high-need students, and is a practice, strategy, or program that has not already been widely adopted).

(2) The extent to which the proposed project has a clear set of goals and an explicit strategy, with the goals, objectives, and outcomes to be achieved by the proposed project clearly specified and measurable and linked to the priorities the eligible applicant is seeking to meet.

B. Strength of Research, Significance of Effect, and Magnitude of Effect.

The Secretary considers the strength of the existing research evidence,¹³ including reported practice, theoretical considerations, and the significance and magnitude of any effects reported in prior research, on whether the proposed project will improve student achievement or student growth, close achievement gaps, decrease dropout rates, increase high school graduation rates, or increase college enrollment and completion rates. Eligible applicants may also demonstrate success through an intermediate variable that is strongly correlated with improving these outcomes, such as teacher or principal effectiveness.

In determining the strength of the existing research evidence, the Secretary considers the following factors:

(1) The extent to which the eligible applicant demonstrates that there are research-based findings or reasonable hypotheses that support the proposed project, including related research in education and other sectors.

(2) The extent to which the proposed project has been attempted previously, albeit on a limited scale or in a limited setting, with promising results that suggest that more formal and systematic study is warranted.

(3) The extent to which the eligible applicant demonstrates that, if funded, the proposed project likely will have a positive impact, as measured by the importance or magnitude of the effect, on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, increasing high school graduation rates, or increasing college enrollment and completion rates.

C. Experience of the Eligible Applicant.

The Secretary considers the experience of the eligible applicant in implementing the proposed project or a similar project.

In determining the experience of the eligible applicant, the Secretary considers the following factors:

(1) The past performance of the eligible applicant in implementing projects of the size and scope proposed by the eligible applicant.

(2) The extent to which an eligible applicant provides information and data demonstrating that--

(a) In the case of an eligible applicant that is an LEA, the LEA has--

(i) Significantly closed the achievement gaps between groups of students described in section 1111(b)(2) of the ESEA, or significantly increased student achievement for all groups of students described in such section; and

(ii) Made significant improvements in other areas, such as graduation rates or increased recruitment and placement of high-quality teachers and principals, as demonstrated with meaningful data; or

(b) In the case of an eligible applicant that includes a nonprofit organization, the nonprofit organization has significantly improved student achievement, attainment, or retention through its record of work with an LEA or schools.

D. Quality of the Project Evaluation.

The Secretary considers the quality of the evaluation to be conducted of the proposed project.

In determining the quality of the evaluation, the Secretary considers the following factors.

(1) The extent to which the methods of evaluation are appropriate to the size and scope of the proposed project.

(2) The extent to which the methods of evaluation will provide high-quality implementation data and performance feedback, and permit periodic assessment of progress toward achieving intended outcomes.

(3) The extent to which the evaluation will provide sufficient information about the key elements and approach of the project to facilitate further development, replication, or testing in other settings.

(4) The extent to which the proposed project plan includes sufficient resources to carry out the project evaluation effectively.

Note: We encourage eligible applicants to review the following technical assistance resources on evaluation: (1) What Works Clearinghouse Procedures and Standards Handbook: http://ies.ed.gov/ncee/wwc/references/idocviewer/doc.aspx?docid=19&tocid=1; and (2) IES/NCEE Technical Methods papers: http://ies.ed.gov/ncee/tech_methods/.

E. Strategy and Capacity to Further Develop and Bring to Scale.

The Secretary considers the quality of the eligible applicant’s strategy and capacity to further develop and bring to scale the proposed project.

In determining the quality of the strategy and capacity to further develop and bring to scale the proposed project, the Secretary considers:

(1) The number of students proposed to be reached by the proposed project and the capacity of the eligible applicant and any other partners to reach the proposed number of students during the course of the grant period.

(2) The eligible applicant’s capacity (e.g., in terms of qualified personnel, financial resources, or management capacity) to further develop and bring to scale the proposed practice, strategy, or program, or to work with others (including other partners) to ensure that the proposed practice, strategy, or program can be further developed and brought to scale, based on the findings of the proposed project.

(3) The feasibility of the proposed project to be replicated successfully, if positive results are obtained, in a variety of settings and with a variety of student populations. Evidence of this ability includes the availability of resources and expertise required for implementing the project with fidelity, and the proposed project’s evidence of relative ease of use or user satisfaction.

(4) The eligible applicant’s estimate of the cost of the proposed project, which includes the start-up and operating costs per student per year (including indirect costs) for reaching the total number of students proposed to be served by the project. The eligible applicant must include an estimate of the costs for the eligible applicant or others (including other partners) to reach 100,000, 250,000, and 500,000 students.

(5) The mechanisms the eligible applicant will use to broadly disseminate information on its project so as to support further development or replication.

F. Sustainability.

The Secretary considers the adequacy of resources to continue to develop or expand the proposed practice, strategy, or program after the grant period ends.

In determining the adequacy of resources for the proposed project, the Secretary considers the following factors:

(1) The extent to which the eligible applicant demonstrates that it has the resources, as well as the support from stakeholders (e.g., State educational agencies, teachers’ unions) to operate the project beyond the length of the Development grant.

(2) The potential and planning for the incorporation of project purposes, activities, or benefits into the ongoing work of the eligible applicant and any other partners at the end of the Development grant.

G. Quality of the Management Plan and Personnel.

The Secretary considers the quality of the management plan and personnel for the proposed project.

In determining the quality of the management plan and personnel for the proposed project, the Secretary considers:

(1) The adequacy of the management plan to achieve the objectives of the proposed project on time and within budget, including clearly defined responsibilities, timelines, and milestones for accomplishing project tasks.

(2) The qualifications, including relevant training and experience, of the project director and key project personnel, especially in managing projects of the size and scope of the proposed project.

Table 3.¹⁴  Differences Between the Three Types of Investing in Innovation Fund Grants in Terms of the Evidence Required to Support the Proposed Practice, Strategy, or Program

	Scale-up grants	Validation grants	Development grants
Strength of Research	Strong evidence	Moderate evidence	Reasonable hypotheses
Internal Validity (Strength of Causal Conclusions) and External Validity (Generalizability)	High internal validity and high external validity	(1) High internal validity and moderate external validity; or (2) moderate internal validity and high external validity	Theory and reported practice suggest the potential for efficacy for at least some participants and settings
Prior Research Studies Supporting Effectiveness or Efficacy of the Proposed Practice, Strategy, or Program	(1) More than one well-designed and well-implemented experimental study or well-designed and well- implemented quasi-experimental study; or (2) one large, well-designed and well- implemented randomized controlled, multisite trial	(1) At least one well-designed and well-implemented experimental or quasi-experimental study, with small sample sizes or other conditions of implementation or analysis that limit generalizability; (2) at least one well-designed and well-implemented experimental or quasi-experimental study that does not demonstrate equivalence between the intervention and comparison groups at program entry but that has no other major flaws related to internal validity; or (3) correlational research with strong statistical controls for selection bias and for discerning the influence of internal factors	(1) Evidence that the proposed practice, strategy, or program, or one similar to it, has been attempted previously, albeit on a limited scale or in a limited setting, and yielded promising results that suggest that more formal and systematic study is warranted; and (2) a rationale for the proposed practice, strategy, or program that is based on research findings or reasonable hypotheses, including related research or theories in education and other sectors
Practice, Strategy, or Program in Prior Research	The same as that proposed for support under the Scale-up grant	The same as , or very similar to, that proposed for support under the Validation grant	The same as, or similar to, that proposed for support under the Development grant
Participants and Settings in Prior Research	Participants and settings included the kinds of participants and settings proposed to receive the treatment under the Scale-up grant	Participants or settings may have been more limited than those proposed to receive the treatment under the Validation grant	Participants or settings may have been more limited than those proposed to receive the treatment under the Development grant
Significance of Effect	Effect in prior research was statistically significant, and would be likely to be statistically significant in a sample of the size proposed for the Scale-up grant	Effect in prior research would be likely to be statistically significant in a sample of the size proposed for the Validation grant	Practice, strategy, or program warrants further study to investigate efficacy
Magnitude of Effect	Based on prior research, substantial and important for the target population for the Scale-up project	Based on prior research, substantial and important, with the potential of the same for the target population for the Validation project	Based on prior implementation, promising for the target population for the Development project

Executive Order 12866

Under Executive Order 12866, the Secretary must determine whether this regulatory action is “significant” and, therefore, subject to the requirements of the Executive Order and subject to review by OMB. Section 3(f) of Executive Order 12866 defines a “significant regulatory action” as an action likely to result in a rule that may (1) have an annual effect on the economy of $100 million or more, or adversely affect a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local, or tribal governments, or communities in a material way (also referred to as an “economically significant” rule); (2) create serious inconsistency or otherwise interfere with an action taken or planned by another agency; (3) materially alter the budgetary impacts of entitlement grants, user fees, or loan programs or the rights and obligations of recipients thereof; or (4) raise novel legal or policy issues arising out of legal mandates, the president's priorities, or the principles set forth in the Executive order. Pursuant to the Executive order, it has been determined that this regulatory action will have an annual effect on the economy of more than $100 million because the amount of government transfers provided through the Investing in Innovation Fund will exceed that amount. Therefore, this action is “economically significant” and subject to OMB review under section 3(f)(1) of the Executive order.

The potential costs associated with this regulatory action are those resulting from statutory requirements and those we have determined as necessary for administering this program effectively and efficiently.

In assessing the potential costs and benefits--both quantitative and qualitative--of this regulatory action, we have determined that the benefits of the final priorities, requirements, definitions, and selection criteria justify the costs.

We have determined, also, that this regulatory action does not unduly interfere with State, local, and tribal governments in the exercise of their governmental functions.

Need for Federal Regulatory Action

These final priorities, requirements, definitions, and selection criteria are needed to implement the Investing in Innovation Fund. The Secretary does not believe that the statute, by itself, provides a sufficient level of detail to ensure that the program achieves the greatest national impact in promoting educational innovation. The authorizing language is very brief and provides only broad parameters governing the program. The final priorities, requirements, definitions, and selection criteria established in this notice provide greater clarity on the types of activities the Department seeks to fund, and permit the Department to fund projects that are closely aligned with the Secretary’s priorities.

In the absence of specific selection criteria for the Investing in Innovation Fund, the Department would use the general selection criteria in 34 CFR 75.210 in selecting grant recipients. The Secretary does not believe the use of those general criteria would be appropriate for the Investing in Innovation Fund grant competition, because they do not focus on the educational reform and innovation activities most likely to improve student achievement and attainment outcomes and eliminate persistent disparities in these outcomes among different populations of students.

Regulatory Alternatives Considered

The Department considered a variety of possible priorities, requirements, definitions, and selection criteria before deciding to establish those included in this notice. The final priorities, requirements, definitions, and selection criteria are those that the Secretary believes best capture the purposes of the program while clarifying what the Secretary expects the program to accomplish and ensuring that program activities are aligned with Departmental priorities. The final priorities, requirements, definitions, and selection criteria also provide eligible applicants with flexibility in selecting activities to apply to carry out under the program. The Secretary believes that the final priorities, requirements, definitions, and selection criteria thus appropriately balance a limited degree of specificity with broad flexibility in implementation.

Summary of Costs and Benefits

The Secretary believes that the final priorities, requirements, definitions, and selection criteria do not impose significant costs on eligible applicants. The Secretary also believes that the benefits of the final priorities, requirements, definitions, and selection criteria outweigh any associated costs.

The Secretary believes that the final priorities, requirements, definitions, and selection criteria would result in the selection of high-quality applications to implement activities that are most likely to have a significant national impact on educational reform and improvement. The final priorities, requirements, definitions, and selection criteria are intended to provide clarity as to the scope of activities the Secretary expects to support with program funds and the expected burden of work involved in preparing an application and implementing a project under the program. The pool of possible applicants is very large; during school year 2007-08, 9,729 LEAs across the country (about 65 percent of all LEAs) made AYP. Although not every one of those LEAs would necessarily meet all the eligibility requirements, the number of LEAs that would meet them is likely to be in the thousands. Eligible applicants would need to consider carefully the effort that will be required to prepare a strong application, their capacity to implement a project successfully, and their chances of submitting a successful application.

The Secretary believes that the costs imposed on applicants by the final priorities, requirements, definitions, and selection criteria would be limited to paperwork burden related to preparing an application and that the benefits of the final priorities, requirements, definitions, and selection criteria outweigh any costs incurred by applicants. The costs of carrying out activities will be paid for with program funds and with matching funds provided by private-sector partners. Thus, the costs of implementation would not be a burden for any eligible applicants, including small entities. However, under the final selection criteria the Secretary will assess the extent to which an eligible applicant is able to sustain a project once Federal funding through the Investing in Innovation Fund is no longer available. Thus, eligible applicants should propose activities that they will be able to sustain without funding from the program and, thus, in essence, should include in their project plan the specific steps they will take for sustained implementation of the proposed project.

The final priorities provide flexibility on the topics and types of grant activities applicants may propose. The use of three types of grants--Scale-up, Validation, and Development grants--will allow potential eligible applicants to determine which type of grant they are best suited to apply for, based on their own priorities, resources, and capacity to implement grant activities.

Accounting Statement

As required by OMB Circular A-4 (available at

http://www.Whitehouse.gov/omb/Circulars/a004/a-4.pdf), in the following table, we have prepared an accounting statement showing the classification of the expenditures associated with the provisions of this final regulatory action. This table provides our best estimate of the Federal payments to be made to eligible applicants under this program as a result of this final regulatory action. Expenditures are classified as transfers to LEAs and nonprofit organizations.

Table--Accounting Statement Classification of Estimated Expenditures

Category Transfers (in millions)

Annual Monetized Transfers $643

From Whom to Whom Federal Government to LEAs

and nonprofit organizations

Paperwork Reduction Act of 1995

The requirements and selection criteria established in this notice require the collection of information that is subject to review by the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 (44 U.S.C. 3501-3520). The Department has received emergency approval for the information collections described below under Information Collection Reference Number XX.

Estimates for Scale-up Grants: We estimate 100 applicants for Scale-up grants, and that each applicant would spend approximately 120 hours of staff time to address the application requirements and criteria, prepare the application, and obtain necessary clearances. The total number of hours for all Scale-up applicants is an estimated 12,000 hours (100 applicants times 120 hours equals 12,000 hours).

Estimates for Validation Grants: We estimate 500 applicants for Validation grants, and that each applicant would spend approximately 120 hours of staff time to address the application requirements and criteria, prepare the application, and obtain necessary clearances. The total number of hours for all Validation applicants is an estimated 60,000 hours (500 applicants times 120 hours equals 60,000 hours).

Estimates for Development Grants: We estimate 1000 full applications for Development grants, and that each applicant would spend approximately 120 hours of staff time to address the application requirements and criteria, prepare the application, and obtain necessary clearances. The total number of hours for all Development applicants is an estimated 120,000 hours (1000 applicants times 120 hours equals 120,000 hours)).

Total Estimates: Across the three grant types, we estimate the average total cost per hour of the LEA and nonprofit organization staff who carry out this work to be $25.00 an hour. The total estimated cost for all applicants would be $4,800,000 ($25.00 times 192,000 (12,000 + 60,000 + 120,000) hours equals $4,800,000).

Regulatory Flexibility Act Certification

The Secretary certifies that this final regulatory action will not have a significant economic impact on a substantial number of small entities. The small entities that this final regulatory action will affect are small LEAs or nonprofit organizations applying for and receiving funds under this program. The Secretary believes that the costs imposed on applicants by the final priorities, requirements, definitions, and selection criteria would be limited to paperwork burden related to preparing an application and that the benefits of the final priorities, requirements, definitions, and selection criteria outweigh any costs incurred by applicants.

Participation in this program is voluntary. For this reason, the final priorities, requirements, definitions, and selection criteria would impose no burden on small entities in general. Eligible applicants would determine whether to apply for funds, and have the opportunity to weigh the requirements for preparing applications, and any associated costs, against the likelihood of receiving funding and the requirements for implementing projects under the program. Eligible applicants would most likely apply only if they determine that the likely benefits exceed the costs of preparing an application. The likely benefits include the potential receipt of a grant as well as other benefits that may accrue to an entity through its development of an application, such as the use of that application to spur educational reforms and improvements without additional Federal funding.

The U.S. Small Business Administration Size Standards define as “small entities” for-profit or nonprofit institutions with total annual revenue below $7,000,000 or, if they are institutions controlled by small governmental jurisdictions (that are comprised of cities, counties, towns, townships, villages, school districts, or special districts), with a population of less than 50,000. The Urban Institute’s National Center for Charitable Statistics reported that of 203,635 nonprofit organizations that had an educational mission and reported revenue to the Internal Revenue Service by July 2009, 200,342 (or about 98 percent) had revenues of less than $5 million. In addition, there are 12,484 LEAs in the country that meet the definition of small entity. However, the Secretary believes that only a small number of these entities would be interested in applying for funds under this program, thus reducing the likelihood that the final priorities, requirements, definitions, and selection criteria in this notice would have a significant economic impact on small entities.

In addition, the Secretary believes that the final priorities, requirements, definitions, and selection criteria do not impose any additional burden on small entities applying for a grant than they would face in the absence of the proposed action. That is, the length of the applications those entities would submit in the absence of the regulatory action and the time needed to prepare an application would likely be the same.

Further, this final regulatory action may help small entities determine whether they have the interest, need, or capacity to implement activities under the program and, thus, prevent small entities that do not have such an interest, need, and capacity from absorbing the burden of applying.

This final regulatory action would not have a significant economic impact on small entities once they receive a grant because they would be able to meet the costs of compliance using the funds provided under this program and with any matching funds provided by private-sector partners.

Intergovernmental Review: This program is subject to Executive Order 12372 and the regulations in 34 CFR 79. One of the objectives of the Executive Order is to foster an intergovernmental partnership and a strengthened federalism. The Executive Order relies on processes developed by State and local governments for coordination and review of proposed Federal financial assistance.

This document provides notification of our specific plans and actions for this program.

Accessible Format: Individuals with disabilities can obtain this document in an accessible format (e.g., braille, large print, audiotape, or computer diskette) on request to the program contact person listed under FOR FURTHER INFORMATION CONTACT.

Electronic Access to This Document: You can view this document, as well as all other documents of this Department published in the Federal Register, in text or Adobe Portable Document Format (PDF) on the Internet at the following site: www.ed.gov/news/fedregister.

To use PDF you must have Adobe Acrobat Reader, which is available free at this site.

Note: The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available on GPO Access at: www.gpoaccess.gov/nara/index.html.

Dated:

________________________

Arne Duncan,

Secretary of Education

1 Under the final requirements for the School Improvement Grants program, “persistently lowest-achieving schools” means, as determined by the State, (a)(1) any Title I school in improvement, corrective action, or restructuring that (i) is among the lowest-achieving five percent of Title I schools in improvement, corrective action, or restructuring or the lowest-achieving five Title I schools in improvement, corrective action, or restructuring in the State, whichever number of schools is greater; or (ii) is a high school that has had a graduation rate as defined in 34 CFR 200.19(b) that is less than 60 percent over a number of years; and (2) any secondary school that is eligible for, but does not receive, Title I funds that (i) is among the lowest-achieving five percent of secondary schools or the lowest-achieving five secondary schools in the State that are eligible for, but do not receive, Title I funds, whichever number of schools is greater; or (ii) is a high school that has had a graduation rate as defined in 34 CFR 200.19(b) that is less than 60 percent over a number of years. See http://www2.ed.gov/programs/sif/faq.html. The definition of this term is used also by the State Fiscal Stabilization Fund and Race to the Top Fund programs.

2 A single LEA could submit a group application on behalf of itself and other eligible LEAs under section 14007(a)(1)(A) of the ARRA. In that case, each of the other eligible LEAs included in the group application must meet the eligibility requirements of this program. Because an LEA that submits an application on its own has flexibility to work with other LEAs as other partners (as defined in this notice), the Department sees no advantage to an LEA submitting a group application in this manner. For this reason, we do not address the applicability of requirements to group applications submitted by LEAs under section 14007(a)(1)(A) of the ARRA in this notice.

3 Because the Department makes a grant award to the grantee, we interpret the term “fiscal agent” as used in section 14007(d) of the ARRA as referring to the applicant receiving an award, namely the grantee. We recognize that the grantee may rely on another entity to manage its grant funds, and that the grantee or others may consider that entity as the fiscal agent of the grant. For the Department’s purposes, under this program, we do not consider such entities as fiscal agents; because the Department’s funding relationship is with the grantee, who is responsible for ensuring the grant is administered in accordance with program regulations.

4 Note that this requirement pertains to the entities that are eligible to apply for funding under this program. In order to receive funding, entities that meet the Eligible Applicants requirement must also meet the eligibility requirements discussed elsewhere in this notice.

5 For example, in a partnership between a nonprofit organization and one or more LEAs for which the nonprofit organization is the fiscal agent, the nonprofit organization may make subgrants only to the LEAs in the partnership.

6 Pursuant to the Department’s grants regulations, multiple eligible LEAs could also apply as a group. However there is no advantage for multiple LEAs to apply as a group.

7 Note, however, that, under section 14011 of the ARRA, no recipient of ARRA funding, including a grantee under this program, may provide financial assistance to students to attend private elementary or secondary schools, unless funds are used to provide special education and related services to children with disabilities, as authorized by the Individuals with Disabilities Education Act (20 U.S.C. 1400 et seq.).

8 Consistent with the Race to the Top Fund, the Department interprets the core academic subject of “science” under section 9101(11) to include STEM education (science, technology, engineering, and mathematics) which encompasses a wide-range of disciplines, including science.

9 Under the final requirements for the School Improvement Grants program, “persistently lowest-achieving schools” means, as determined by the State, (a)(1) any Title I school in improvement, corrective action, or restructuring that (i) is among the lowest-achieving five percent of Title I schools in improvement, corrective action, or restructuring or the lowest-achieving five Title I schools in improvement, corrective action, or restructuring in the State, whichever number of schools is greater; or (ii) is a high school that has had a graduation rate as defined in 34 CFR 200.19(b) that is less than 60 percent over a number of years; and (2) any secondary school that is eligible for, but does not receive, Title I funds that (i) is among the lowest-achieving five percent of secondary schools or the lowest-achieving five secondary schools in the State that are eligible for, but do not receive, Title I funds, whichever number of schools is greater; or (ii) is a high school that has had a graduation rate as defined in 34 CFR 200.19(b) that is less than 60 percent over a number of years. See http://www2.ed.gov/programs/sif/faq.html.

10 A single subject or single case design is an adaptation of an interrupted time series design that relies on the comparison of treatment effects on a single subject or group of single subjects. There is little confidence that findings based on this design would be the same for other members of the population. In some single subject designs, treatment reversal or multiple baseline designs are used to increase internal validity. In a treatment reversal design, after a pretreatment or baseline outcome measurement is compared with a post treatment measure, the treatment would then be stopped for a period of time, a second baseline measure of the outcome would be taken, followed by a second application of the treatment or a different treatment. A multiple baseline design addresses concerns about the effects of normal development, timing of the treatment, and amount of the treatment with treatment-reversal designs by using a varying time schedule for introduction of the treatment and/or treatments of different lengths or intensity.

11 For additional information on the evidence for Scale-up grants, see Table 3 later in this section.

12 For additional information on the evidence for Validation grants, see Table 3 later in this section.

13 For additional information on the evidence for Development grants, see Table 3 later in this section.

14 This table is identical to Table 1 earlier in this notice.

371