60D Comment Response

Memo

To:

From: Implementation of Title I/II-A Program Initiatives Study Team

Date: 12/23/2021

Subject: Response to public comment

Thank you for the public comments submitted regarding the Implementation of Title I/II-A Program Initiatives study (Docket No. ED-2021-SCC-0129) on September 28, 2021.

Comment:

TO: Coordinator of the Strategic Collections and Clearance Governance and Strategy Division, U.S. Department of Education

CC:

COMMENT: on Implementation of Title I/ II - A Program Initiatives

INTRO: [Summary: In accordance with the Paperwork Reduction Act of 1995, ED is proposing a new collection.] I wholeheartedly agree and support a new collection method for obtaining site-level aggregate data to help inform departmental decisions about the efficacy of core programs, particularly for Title I and Title II, to the Every Student Succeeds Act (ESSA).

A centralized, digital reporting platform for educational service providers is needed to take the onus off of teachers and schools to report and file their Professional Development Training hours. To do so, we should place the ‘Reporting Responsibility’ on the vendors who are providing the training and support services, and who have an added incentive to report the teachers and schools who are using their training services to the US Department of Education. The data should only have to be ‘signed off’ on by the school organizations. The department should create this online system for vendors to report who they are providing Professional Development services for and in what quantities.

Particularly as it relates to your solicited point number 5, “and (5) how might the Department minimize the burden of this collection on the respondents, including through the use of information technology.” I believe a focus should be on the states and districts’ vendor-partners to report to the department, in order that the department glean performance data under ESSA (Every Student Succeeds Act) for Title I/ II - A, with a particular focus on teacher professional development and/ or teacher professional training, including coaching.

Under the ESSA, one of its core programs, Title II - A, provides economic incentives (i.e. expenditure reimbursements) for teachers to gain continuous training, often called Professional Development. The reporting onus is often on the user (the teacher), rather than the provider (the vendors) of the training services.

The PD market is an odd ‘market’ in that the suppliers are oftentimes also and coterminously the purchaser. Meaning, the school districts themselves provide the training and fund themselves for the training. This proves to be a ‘closed market’, whereas only roughly 18% of the industry Professional Development and Training funding activity is allocated to third-party vendor suppliers. These vendors have more of the technological infrastructure and flexibility, (being oftentimes cutting-edge technology companies) than the school districts do themselves. Therefore, shifting the online reporting requirement infrastructure to the third-party vendors is also congruent with the state of technological infrastructure throughout the country’s school districts.

The vendors have an incentive to report PD, as more PD is coming in the form of ‘Coaching’. A Bill & Melinda Gates sponsored survey of 1,300 teachers, nationwide, showed that teachers are increasingly receiving ‘Coaching’ forms of training. These ‘Coaching’ formats often include independent consultants training teachers on lesson planning, delivery, and evaluation. For better transparency, the vendor-suppliers should report to the department on who and in what quantities they provided services to.

Although training decisions are often made at the school district level, schools are more equipped to know the particular and peculiar needs of each individual school. In the US, on average, teachers spend between 70 - 90 hours per year in professional training and development. The historical status quo has been to give more autonomy to the individual schools, which has caused more decentralized training systems nationwide. As a result, it would be more advantageous for the vendors to report to a unified system, in order that the department glean performance data.

The market for professional education training services is very fragmented. There currently exist no unified reporting repository for ‘coaching hours’ or service hours for independent service providers to report on.

Teachers have also reported a desire for a more ‘on-demand’ form of ‘Coaching’ service. This format would mark a significant change in the format of the professional development status quo, where training is designed ‘and then’ delivered to the teacher staff. Through an ‘on-demand’ model, the teacher would be placed in the center of the decision making process as to which vendors get pulled in and when.

Professional Development of K - 12 teachers in the US is an $18 Billion industry! Coordinating between the school entities and the service vendors will increase transparency with the department and enable a greater, more accurate evaluation of the return on investing in teacher training.

Response: In the absence of a centralized system to collect data on professional development, the study team plans to use a series of survey questions of all states and nationally representative groups of districts and schools to better understand the amount, nature, and content of teacher professional development during the 2021-22 school year.

Memo

To: All4Ed

From: Implementation of Title I/II-A Program Initiatives Study Team

Date: 12/23/2021

Subject: Response to public comment

Thank you for the public comments submitted regarding the Implementation of Title I/II-A Program Initiatives study (Docket No. ED-2021-SCC-0129) on November 2, 2021. We appreciate the comments and have incorporated changes into the surveys where possible. In general, we had to weigh the tradeoff between the topics and level of detail covered in the surveys with the burden on survey respondents. While we could not add information related to every suggestion, we tried to incorporate information in response to your comments to the extent feasible. The study team offers the below responses to your specific comments.

Relevant survey/ question	Public Comment Recommendation	Study Team Response
School 3-9 (same as new School 3-9)	Mention additional questions related specifically to high-school interventions: early warning indicator systems, college and career pathway programs, expanded access to advanced coursework (e.g., dual enrollment or dual credit courses, early college HS, AP, IB)	We added an item about early warning indicator systems as part of 3-5, which asks what data / information was used to identify students needing support. We added college and career pathway programs and expanded access to advanced coursework to 3-9, which asks about support for students to accelerate learning.
n/a	Examine distribution of Title I funds by elementary vs middle vs high school	We did not incorporate this suggestion. Funding amounts and allocations are not a focus of this survey, However, IES has another study, Study of District and School Uses of Federal Education Funds, that examines how funds are distributed and used from five major federal education programs, including Part A of Title I, as well as the CARES Act.

Memo

To: Reid Setzer, The Education Trust

From: Implementation of Title I/II-A Program Initiatives Study Team

Date: 12/23/2021

Subject: Response to public comment

Thank you for the public comments submitted regarding the Implementation of Title I/II-A Program Initiatives study (Docket No. ED-2021-SCC-0129) on November 2, 2021. We appreciate the comments and have incorporated changes into the surveys where possible. In general, we had to weigh the tradeoff between the topics and level of detail covered in the surveys with the burden on survey respondents. While we could not add information related to every suggestion, we tried to incorporate information in response to your comments to the extent feasible. The study team offers the below responses to your specific comments.

Relevant survey/ question	Public Comment Recommendation	Study Team Response
All survey levels	Throughout survey – definitions: In each relevant definition section, amend the definition of “Comprehensive Support and Improvement (CSI) schools” to include the third group of CSI schools that states are required to identify under ESSA – those with chronically underperforming subgroups – as follows: “Comprehensive Support and Improvement (CSI) schools are those in the bottom 5 percent of all Title I schools, and schools with graduation rates below 67 percent, and schools with chronically underperforming subgroups, as defined under ESEA for federal accountability.”	We have updated the CSI definition at all three levels of the survey.
SEA 1-1 to 1-4: During this school year (2021–22), did your state provide or fund professional development (PD) to teachers on the following topics?	Amend the response options (under “Professional Development (PD) Topic”) as follows: • (a): Separate “Curricula, standards, and subject matter content” into three separate options — “Curricula and instructional materials;” “Standards;” and “Subject matter content.” • (d): Amend this option as follows: “Other specific evidence-based strategies to help students catch up or accelerate learning (not focused on use of technology).” • (g): Separate “Engaging students and families” to ensure that information on the engagement of students and families is collected separately — “Engaging families;” and “Improving classroom management or relationships with students” (this option is included in a similar question in the School Survey). • Add two new topics: (1) “Restorative classroom management practices or other alternatives to punitive practices (e.g., suspensions, expulsion)” (this option is included in a similar question in the School Survey), and (2) “Conducting assessments and using data on student progress to inform instruction.”	We understand that there may be interest in these topics. However, given (1) the need to constrain the burden on the survey, and (2) the need to be able to collect information that can be easily summarized and reported, we decided not to add them. The suggestion to add “evidence-based” as a modifier to “strategies” would not be likely to change the responses, as respondents are likely to believe that their practices are evidence-based. It may even induce social desirability bias in their responses, leading more respondents to choose this item but not because they necessarily engaged in these practices. The suggested new topic on restorative classroom management practices is captured by an item on this list: “Student behavior, discipline, and safety strategies.” Finally, district survey question 3-1 captures use of state-mandated academic assessments to plan and target academic interventions and support to schools and students. In addition, school survey question 2-2 captures use of assessments to inform curriculum and instruction.
SEA 1-11 (new SEA 1-4): Within the past 12 months, which of the following types of information did the state use to assess the effectiveness of any of its teacher preparation programs for accountability or support? Please indicate if each type of information has been used for assessing effectiveness of traditional preparation programs only, alternative preparation programs only, both traditional and alternative programs, or neither.	Teacher preparation programs play an important role in efforts to recruit, support, and graduate new teachers of color. Therefore, we recommend adding response options (under “Type of Information”) that assess whether the state uses candidate diversity to evaluate the effectiveness of programs: • “The racial and ethnic diversity of the program’s graduates who earn certification” • “The racial and ethnic diversity of the program’s graduates placed in teaching jobs” • “The rates of retention in the profession of the program’s graduates who are teachers of color”	We agree that attention to racial and ethnic diversity should be reflected among the response options. However, we are cautious about adding additional burden to the survey. So, we have added one additional response option to question 1-4 on the racial and ethnic diversity of the program’s graduates who earn certification. Of the three proposed options, we have reason to believe that this one is most likely to be used by states.
SEA survey: New question proposed	After question 1-12, insert an additional question to identify whether states are reporting information about the racial and ethnic diversity of the state’s teacher preparation programs. • Additional Question: “Within the past 12 months, has your state reported information about the racial and ethnic diversity of the teachers they prepared to traditional and alternative preparation programs that the teachers attended or to the public using information listed in [question 1-11]?” • Response Options (“Type of Information”): For each, require states to select “Yes” or “No.” o “State reported information about diversity to traditional preparation programs” o “State reported information about diversity of traditional preparation programs to the public [If available, please provide link to report]:” 13 o “State reported information about diversity to alternative preparation programs” o “State reported information about diversity of alternative preparation programs to the public [If available, please provide link to report]:”	We have addressed this concern by adding a response option to question 1-4 about the racial and ethnic diversity of teacher preparation programs.
SEA 1-18 (new SEA 1-11)	Because the experiences of students from low-income backgrounds and students of color are not synonymous, the responses to this question should be broken into separate options. In each instance, replace, “The examination revealed that low-income students and students of color tended to ….” with two separate questions as follows: • “The examination revealed that low-income students tended to …” • “The examination revealed that students of color tended to …”	To allow states to answer separately about findings of access to effective teachers for low-income students and students of color, we have created a grid for question 1-11 that asks about low-income students in the top row and students of color in the bottom row.
SEA 1-19	Add the following response options (under “Action Taken”): • “State assisted districts with less-qualified or less-effective teachers and those with greater teacher shortages to hire teachers earlier”	We removed this question (1-19) and now instead ask all districts (not just those that have found inequitable access to effective teachers) a question (1-2) about strategies for teacher hiring and retention. Our new question includes a response option on states assisting districts with early hiring, consistent with this suggestion.
SEA 2-14	Question 2-14: The current response options include responses that would violate federal law (e.g., canceling statewide assessments in 2021–2022). Therefore, the Department should remove the following option: • “N/A Assessment Was Not Administered”	This question was deleted with the plan to obtain this information from extant sources.
SEA 2-18	To ensure this data is useful in identifying whether individual groups of students in different instructional settings were able to participate in the state-mandated academic assessments, and consistent with ESSA’s existing requirement that participation rates be disaggregated, we recommend the Department add response options to capture whether states are disaggregating by both instructional setting and student group: • “Participation rate by instructional setting (e.g., percentage of students learning in remote or distance, hybrid, or in-person classes), disaggregated for each student group” • “Percent proficient or advanced by instructional setting (e.g., percentage of students learning in remote or distance, hybrid, or in-person classes), disaggregated for each student group.”	We agree ESSA requires SEAs to report participation rates disaggregated by ethnicity and special populations. However, ESSA does not require reporting state-mandated assessment results by instructional setting. Additionally, information from Ed.Gov suggests that most schools are in-person for the 2021-22 school year. Given concerns about respondent burden, we did not add this question to the survey.
SEA 2-19 (new SEA 2-5, row e)	One primary purpose of the statewide assessments required under Title I of the ESSA is to provide parents and families with information about their student’s progress against grade level standards. Therefore, we recommend the Department ask states to identify whether they provided written guidance, individualized support, or group professional development to districts and schools about using results to inform and support parents and families Additional option: “How to use state-mandated academic assessment data to provide opportunities for families to be involved in students’ academic progress.”	We agree that this is an important purpose of the statewide assessments and have added a row to the question.
SEA 3-47, 48, 49, 51 (new SEA 3-19, 20, 25, 26) LEA 3-4, 7, 9 (new LEA 3-2, 5, 6) School 3-7, 10 (same as new School 3-7, 10)	Clearly define what “small groups” means to match research and avoid ambiguity.	On all of these questions, we have modified the row on tutoring so it says “Provide tutoring (in groups of one to four students)"
SEA 3-78 (new SEA 3-45) LEA 3-31 (new LEA 3-23)	Define what "meaningful differences" in spending might mean.	Although we agree there will be ambiguity in responses, since different respondents may have different implicit definitions, imposing a specific standard seems likely to increase burden substantially. Therefore, we did not make any changes.
SEA Survey: New question proposed	As noted in the attached letter, a growing body of research shows that having access to teachers of color benefits all students, and many state and local efforts to diversify the workforce can be supported with funding under Titles I and II-A. Therefore, after Question 1-19, there should be an additional question about the actions taken by the district to address a lack of diversity in the educator workforce (similar to existing question 1-19 about teacher quality and effectiveness). • Additional Question: “What actions has your state taken to diversify the teacher workforce in the past 12 months?” • Action Taken: (States required to select “Yes” or “No” for each action). a) “State set a clear numerical goal for increasing the racial diversity of the educator workforce” b) “State developed a task force, advisory group, or role within the LEA to examine, create, and monitor strategies to increase the racial diversity of the workforce” c) “State Invested in Grow Your Own programs or teacher academies to increase the diversity of the workforce” d) “State partnered with teacher preparation programs to ensure targeted recruitment and hiring” e) “State invested in cultural competence and anti-bias trainings for hiring managers and school leaders” f) “State invested in residency models to support and prepare candidates of color” g) “State invested in opportunities for teachers of color to grow and develop in their abilities and qualification for leadership roles, including targeted PD and cohort models” h) “Others:”	We have incorporated a similar question at the district level (LEA 1-15), where we think it is most likely that we will find these changes.
LEA 0-6 (new LEA 0-6 and 0-7)	Break this question into two questions, requiring states to separately identify whether any of their low-performing schools are designated for Targeted Support and Improvement (TSI) or Additional Targeted Support and Improvement (ATSI).	We now ask separately whether any of the schools are TSI or ATSI.
LEA 1-1 to 1-3: During … what type of instruction did MOST students receive from your district and its schools?	To the greatest extent possible, the Department should align these questions and response options with those provided in the School Pulse Panel Data Collection. It should define “most” as “more than 50%” of students to ensure the collection of comparable information across school districts.	Because the Department of Education’s School Pulse survey collects these data, we have decided to remove these items from our survey.
LEA 1-5 to 1-8 and 1-13 to 1-14 (various) (LEA 1-5 is now LEA 1-2) (LEA 1-8 is now LEA 1-3)	Amend the response options (under “Professional Development [PD] Topic”) as follows: • (a): Separate “Curricula, standards, and subject matter content” into three options — “Curricula and instructional materials;” “Standards;” and “Subject matter content.” • (d): Amend this option as follows: “Other specific evidence-based strategies to help students catch up or accelerate learning (not focused on use of technology).” • (g): Separate “Engaging students and families,” so information on the engagement of students and families can be collected separately — “Engaging families;” and “Improving classroom management or relationships with students” (this option is included in a similar question in the School Survey). • Add two new topics: (1) “Restorative classroom management practices or other alternatives to punitive practices (e.g., suspensions, expulsion)” (this option is included in a similar question in the School Survey), and (2) “Conducting assessments and using data on student progress to inform instruction.”	We understand that there may be interest in collecting data at this level of detail, but (1) to minimize burden and (2) to gather data that can be summarized and reported easily, we think it’s still valuable to collect them in aggregate. Adding more topics would lengthen the survey beyond the target time limit, adding burden on district staff. The suggestion to add “evidence-based” as a modifier to “strategies” would not be likely to change the responses, as respondents are likely to believe that their practices are evidence-based. It may even induce social desirability bias in their responses, leading more respondents to choose this item but not because they necessarily engaged in these practices. To address interest in restorative justice, we have added “improving classroom management or relationships with students,” an item on the school survey, to questions about PD topics on the SEA survey (1-6 to 1-7) and LEA surveys (1-2 to 1-3). Finally, district survey question 3-1 captures use of state-mandated academic assessments to plan and target academic interventions and support to schools and students. In addition, school survey question 2-2 captures use of assessments to inform curriculum and instruction.
LEA 1-5 to 1-14 (various)	In general, the professional development opportunities and experiences of school leaders are underemphasized in this survey. We recommend duplicating questions 1-5 to 1-14 to focus 4 on school leaders, in addition to retaining the current questions focused solely on teachers. In addition, for questions related to the professional development and support offered to school leaders, we suggest adding the following topics: • “Master scheduling” • “Supporting inexperienced or low-performing teachers” • “Providing feedback to teachers and evaluating them” • “Connecting with community-based organizations.”	We understand that it may be helpful to collect data on the professional development of school leaders as well as teachers. However, asking the same set of questions twice, about school leaders as well as teachers, would substantially increase burden, so we think it is necessary to choose one or the other. We believe it is appropriate to prioritize teacher PD over school leader PD, so we have limited the professional development questions to those that concern teachers.
LEA 1-9: During this school year (2021–22), including last summer (2021), on which of the following topics did your district provide general education teachers with professional development (PD) to help students with disabilities succeed in general education settings?	Add the following two response options (under “Topic”): • “Engaging with families of students with disabilities” • “Supporting the social, emotional, and mental health needs of students with disabilities”	Elsewhere in the survey we collect data on professional development on this topic and so prefer not to add burden by repeating questions in the context of students with disabilities.
LEA 1-10: During this school year (2021–22), including last summer (2021), on which of the following topics did your district provide general education teachers with professional development (PD) to help English learners succeed?	Add the following response option (under “Topic”): • “Engaging with families of English learners”	Elsewhere in the survey we collect data on professional development on this topic and so prefer not to add burden by repeating questions in the context of English learners.
LEA 1-15: During the 2021–22 school year, including last summer (2021), did your district provide tuition reimbursement, professional development (PD) time, or another form of support for teachers to earn certifications, credentials, or endorsements in the following topic areas?	Add the following two areas, which are included in other parts of the survey as well: • “Culturally responsive practices” • “Implicit bias”	Due to concerns about the burden on district-level respondents, we have streamlined the survey by removing this question. We will focus on collecting data based on the list of PD topics, which already includes these two areas.
LEA 1-22 (new LEA 1-12): For this school year (2021–22), how many full-time teachers did your district hire from the following programs?	We appreciate the Department’s effort to collect data on which programs districts are using to recruit and hire teachers of color. We recommend revising option (d) to include a broader set of alternative programs: • (d): “Alternative Certification programs (e.g., Teach for America, Urban Teachers, TNTP, other AmeriCorps programs)”	We agree that it makes sense to collect data on other alternative certification programs and so have amended the list of alternative programs as suggested (question 1-12).
LEA 1-25: What information was used to define teacher quality or effectiveness in the examination of the distribution of teachers?	Consistent with prior questions and responses focused on district support for professional certifications (e.g., National Board Certification), add “National Board Certification” as an option in the list of “Source(s) of Information” used to inform how the district defines teacher quality or effectiveness.	Due to concerns about the burden on district-level respondents, we have removed this question, as it is not as essential to collecting information about access to effective teachers. We have retained a question on what districts found as a result of their investigation.
LEA 1-26 (new LEA 1-17): According to your district’s examination of the distribution of teacher quality or effectiveness, to what extent are there more effective teachers in schools serving fewer low-income students or fewer students of color compared to schools serving more low-income students or more students of color?	Because the experiences of students from low-income backgrounds and students of color are not synonymous, the responses to this question should be broken into separate options, as follows: • “The examination revealed that low-income students tended to …” • “The examination revealed that students of color tended to …”	To allow districts to answer separately about findings of access to effective teachers for low-income students and students of color, we have created a grid for question 1-17 that asks about low-income students in the top row and students of color in the bottom row.
LEA 1-27 and 1-28: What actions has your district taken to address any inequities found in teacher quality or effectiveness in the past 12 months?	Add another response option (under “Action Taken”) to include an additional option for school leaders that is parallel to existing action (g) for teachers: • “Offering more professional development for principals/school leaders in schools with lower levels of teacher quality or effectiveness compared to other schools.”	We have decided to remove these questions and instead ask all districts (not just those that have found inequitable access to effective teachers) a question (1-11) about strategies for teacher hiring and retention. In the list of strategies, we do not include an option for offering PD to administrators in these schools, as it seems like a very indirect route by which to improve the hiring, recruitment, or retention of teachers.
LEA 2-9 and 2-13: Which schools are required to use the particular instructional planning resource for ELA or math for the 2021-22 school year? Same comment apply to SEA 2-9, 2-13, and 2-21	Add response option “TSI schools”	Questions on the targeting of guidance for instructional planning resources (LEA 2-9 and SEA 2-9) and supports for instructional processes aligned to the state content standards (LEA 2-13 and SEA 2-13) and guidance on use of assessment data (SEA 2-21) were dropped to reduce respondent burden. Also, collecting this information was considered less salient than questions that target state and district efforts on school recovery efforts. Instead, the LEA survey asks two questions (new 3-20 and 3-21) on the targeting of professional development topics and targeting of support to CSI schools compared with other schools. To keep survey burden reasonable, we have focused on CSI schools, for which LEAs are more likely to impose requirements.
LEA 3-2: “…several other states did not require all school districts to administer statewide assessments last school year (2020-21)” (e.g., CA, NM)	Therefore, the Department should add this response option to question 3-2: “The district did not administer the state ELA or math assessments for last school year (2020-21)	We have dropped the prior 3-2 from the LEA survey in an effort to reduce burden.
LEA 3-2	Add response option "The district did not administer the state ELA and math assessments for last school year (2020 – 2021)."	We have dropped 3-2 from the LEA survey in an effort to reduce burden.
LEA question removed	Ask about additional resources for TSI schools.	This question has been removed in an effort to reduce burden.
LEA question removed School 3-13 (new school 3-12)	Add row: "All students in this school are required to participate in tutoring."	The school survey question has been revised to include this additional option.
LEA 3-13 (new LEA 3-10)	Add item: “Students participated in culturally reflective or identity affirming activities.”	This seems likely to be subject to social desirability bias and the terms may not be easily defined and therefore may be difficult to interpret. Therefore, we did not make this change.
LEA 3-17h (new LEA 3-13h) School 3-21 (new School 3-18)	Add "Native American students" alongside Black and Latinx	We have made this change here and in other relevant questions.
LEA 3-20 (new LEA 3-10) School 3-17 (new School 3-15)	Distinguish teacher-tutors from paraprofessional tutors	Research evidence largely suggests similar results for paraprofessionals and teachers in tutoring roles (as distinguished from other, non-professional tutors), so we have not made this change.
LEA survey: New questions proposed	As noted in the attached letter, a growing body of research shows that having access to teachers of color benefits all students and many state and local efforts to diversify the workforce can be supported with funding under Titles I and II-A. Therefore, after Question 1-28, there should be an additional question about the actions taken by the district to address the lack of diversity in the educator workforce (similar to existing question 1-27 about teacher quality and effectiveness). • Additional Question: “What actions has your district taken to diversify the teacher workforce in the past 12 months?” • Actions Taken: (Districts required to select “Yes” or “No” for each action). a) “District set a clear, numeric goal for increasing the racial diversity of the educator workforce” b) “District developed a task force, advisory group, or role within the LEA to examine, create, and monitor strategies to increase the racial diversity of the workforce” c) “District Invested in Grow Your Own programs or teacher academies to increase the diversity of the workforce” 5 d) “District partnered with teacher preparation programs to ensure targeted recruitment and hiring” e) “District invested in cultural competence and anti-bias trainings for hiring managers and school leaders” f) “District invested in residency models to support and prepare candidates of color” g) “District invested in opportunities for teachers of color to grow and develop in their abilities and qualification for leadership roles, including targeted PD and cohort models” h) “Others:” After the proposed new question above, there should be an additional question about the primary actions taken by the district to address the lack of diversity in the educator workforce (similar to existing question 1-28 about teacher quality and effectiveness).	We agree that this would be important information to capture and so have included this question in the survey in multiple parts. First, did your district use strategies to increase the racial or ethnic diversity of the teacher workforce (question 1-14)? Second, which strategies did your district use to diversify the teacher workforce and which were the three primary strategies (question 1-15)?
School survey (throughout)	Throughout survey – definitions: Section 1 defines “Novice teachers” as those in their first, second, or third year of teaching. We recommend the Department ensure the definition for “novice teacher” used here aligns with the definition that the Department uses in other circumstances (e.g., the Civil Rights Data Collection collects data on teachers in their first or second year of teaching)	We have made this change.
School 1-3 and 1-4: During the 2021–22 school year, including last summer (2021), for each topic area, please indicate the type of supports that the school provided to teachers (or that this topic was not addressed). Which type of support was the main type of support for each topic?	Amend the response options (under “Professional Development (PD) Topic”) as follows: • (a): Separate “Curricula, standards, and subject matter content” into three options — “Curricula and instructional materials;” “Standards;” and “Subject matter content.” • (d): Amend this option as follows: “Other specific evidence-based strategies to help students catch up or accelerate learning (not focused on use of technology).” • (g): Separate “Engaging students and families” to ensure that information on the engagement of students and families is collected separately — “Engaging families;” and “Improving classroom management or relationships with students” (this option is included in a similar question in the School Survey). • Add two new topics: (1) “Restorative classroom management practices or other alternatives to punitive practices (e.g., suspensions, expulsion)” (this option is included in a similar question in the School Survey), and (2) “Conducting assessments and using data on student progress to inform instruction.”	We understand that there may be interest in collecting data at this level of detail, but (1) to minimize burden and (2) to gather data that can be summarized and reported easily, we think it’s still valuable to collect them in aggregate. Adding more topics would lengthen the survey beyond the target time limit, adding burden on principals. The suggestion to add “evidence-based” as a modifier to “strategies” would not be likely to change the responses, as respondents are likely to believe that their practices are evidence-based. It may even induce social desirability bias in their responses, leading more respondents to choose this item but not because they necessarily engaged in these practices. We have addressed interest in restorative justice in the item “improving classroom management or relationships with students.” We have also added this item to questions about PD topics on the SEA survey (1-6 to 1-7) and LEA surveys (1-2 to 1-3). Finally, district survey question 3-1 captures use of state-mandated academic assessments to plan and target academic interventions and support to schools and students. In addition, school survey question 2-2 captures use of assessments to inform curriculum and instruction.
School 2-11 (new School 2-1)	We recommend the Department amend the answer options so it can identify which schools administered the state-mandated academic assessment in the spring or summer of 2021 versus the fall of 2021. This difference in timing likely has significant implications for how the results of the assessments were used in planning for this school year (2021-22). The revised answers should be as follows:	We agree that having this detail on the timing of the 2021 administration can be helpful context and have revised the response options accordingly.
School 3-30 (new School 3-24)	Add rows on "scheduling staff and students effectively" and "recruiting and retaining diverse teachers"	We added the item on scheduling. Recruiting and retaining diverse teachers already is addressed in LEA question 1-22 of the survey.
Proposed new School question	We recommend the Department ask schools to identify how they communicated about the results of these assessments with parents and families. Additional question: “Prior to or during the 2021 – 22 school year, how did your school communicate state-mandate academic assessment results from last year (administered in the spring, summer or fall of 2021) to parents and families?” Answer options (check all that apply): a) “Mailed results to families” b) “Offered families online access to their student’s results (e.g., via an online student portal)” c) “Instructed teachers to share results with parents and families” d) “Our school district sent results to parents and families”	We appreciate the interest in understanding through what method assessment results are communicated to families. The school survey provides insight into school efforts to build families’ competence and capacity to understand their child’s assessment results (new 2-3). Given concerns about respondent burden, we believe this information more directly gets at the underlying question of interest than asking about the delivery methods.

Memo

To: Aaron Ridings, GLSEN

From: Implementation of Title I/II-A Program Initiatives Study Team

Date: 12/23/2021

Subject: Response to public comment

Thank you for the public comments submitted regarding the Implementation of Title I/II-A Program Initiatives study (Docket No. ED-2021-SCC-0129) on November 2, 2021. We appreciate the comments.

Over the last several months, the Institute of Education Sciences (IES) made the decision to no longer administer the planned teacher survey in light of the significant demands on teachers as they work to help students catch up on or accelerate learning in response to the pandemic. We have also streamlined the remaining surveys in order to decrease the burden on respondents. To do so, we removed questions that were not central to the study’s goals. In particular, the school survey questions asking for principal demographics were removed since IES has recently collected information on the demographics of principals through other surveys.

The study team offers the below responses to your specific comments.

Relevant survey/ question	Public Comment Recommendation	Study Team Response
School 4-8 and teacher 4-10, and school 4-10	Add instruction “select all that apply” to school 4-8 and teacher 4-10 so respondents can select more than one gender identify and intersex status. Add instruction to “select all that apply” to school 4-10 so respondents can select more than one race	The suggestion for teacher 4-10 is no longer relevant. IES decided not to administer the planned teacher survey in light of the significant demands on teachers as they work to help students catch up on or accelerate learning in response to the pandemic. We removed the school survey questions asking for principal demographics since the Institute of Education Sciences has recently collected information on the demographics of principals through its National Teacher and Principal Survey.
New question	Suggestion to add a question about sexual orientation to teacher and school surveys.	Thank you for suggesting a voluntary question on sexual orientation. As noted above, IES decided to not administer the teacher survey in light of the significant demands on teachers. We understand information on sexual orientation could be useful for measuring the representation of marginalized groups among principals. However, this information is not central to the focus of the study, so we do not have plans to collect it. To keep the study instruments to a manageable length, we have needed to prioritize asking only those questions that are central to the focus of the study.

Memo

To: Julie Sugarman, Migration Policy Institute

From: Implementation of Title I/II-A Program Initiatives Study Team

Date: 12/23/2021

Subject: Response to public comment

Thank you for the public comments submitted regarding the Implementation of Title I/II-A Program Initiatives study (Docket No. ED-2021-SCC-0129) on November 1, 2021. We appreciate the comments and have incorporated changes into the surveys where possible. In general, we had to weigh the tradeoff between the topics and level of detail covered in the surveys with the burden on survey respondents. While we could not add information related to every suggestion, we tried to incorporate information in response to your comments to the extent feasible. The study team offers the below responses to your specific comments.

Relevant survey/ question	Public Comment Recommendation	Study Team Response
SEA 2-14 How were the state-mandated academic assessments for ELA or math or the English language proficiency (ELP) assessment administered during the 2021-22 school year?	Reword third column head to “in person to some students and remotely to other students”	This question was deleted from the survey with the plan to obtain this information from extant sources.
Proposed new question for SEA section 2, subsection “State-Mandated Academic Assessments”	Add a new question: “Did the state provide updated guidance on accommodations for ELs for the ELA, math, or ELP exams?” [Response: Yes/No]	We do not think such a question is needed given annual training requirements. We understand that SEAs annually review and revise guidance documents addressing administration protocol including appropriately identifying and providing accommodations for ELs and students with disabilities. Additionally, we know that SEAs annually provide training for all licensed educators that may participate in administering the annual state-mandated assessments. Annual training sessions routinely cover revisions to guidance for ensuring that accommodations for English learners or students with disabilities comply with state requirements. This training would apply to remote and in-person assessment administrations. Also, knowing if an SEA provided updated guidance on use of accommodations does not, on its own, provide insight into the validity of the results. In addition, for burden reasons, we are unable to add more content related to ELs on things like what states' accommodations for ELs are.
SEA 3-39, 3-40 (new SEA 3—14, 3-15)	Add row on available performance measures: "English language proficiency assessment"	We added this row to the response options for these questions.
SEA 3-44 (new SEA 3-17)	Add a row on changing assessment accommodations for ELs	We added this item to the list, but we are planning to obtain information for this question predominantly from public data on waivers rather than through the surveys.
SEA 3-60	Add a row saying "Provide resources for digital literacy training to help parents support remote learning."	This question has been removed from the survey in an effort to reduce burden on respondents by removing content that is less central to the study goals.
SEA 3-63 (new SEA 3-33)	Add a row on evidence of effectiveness for subgroups.	We appreciate this comment, but we did not add this row. We think our existing rows on evidence of effectiveness are sufficient. Evaluations typically are designed to assess effectiveness for all students. Subgroups are often omitted from evaluations due to insufficient statistical power or the absence of a hypothesis related to subgroup-specific impacts.
LEA 1-10 (new LEA 1-5): During this school year (2021–22), including last summer (2021), on which of the following topics did your district provide general education teachers with professional development (PD) to help English learners succeed?	It is just as important for general education teachers to help ELs develop oral language as written language, and teachers sometimes find oral language more challenging. As a result we suggest adding a new topic to parallel choice c. Add topic: “Providing regular, structured opportunities for English learners to develop oral language skills.”	We have incorporated oral language skills into choice c in question 1-5. The revised choice c is “Providing regular, structured opportunities for English learners to develop oral and written language skills.” We have chosen not to add a separate topic to avoid adding burden by lengthening the survey.
	Co-teaching is becoming more common in EL classrooms. We suggest repeating choice b from question 1-9 in 1-10. Add topic: “Co-teaching or collaboration between an EL teacher and a general education teacher.”	We added this topic to question 1-5.
LEA 2-4 To what extent did the district use any of the following review strategies when selecting or recommending instructional materials or supports for ELA or math for the 2021-22 school year? LEA 2-7 Indicate which review criteria the district required schools had to meet before selecting and using instructional materials or supports for ELA or math for the 2021-22 school year.	Add review strategy: “Assess the availability and adequacy of curricular materials versions in a language other than English”	Questions on the process of selecting instructional materials and supports were considered less salient than questions that target state, district, or school recovery efforts. As a result, LEA 2-4 and 2-7, SEA 2-4 and 2-7, and school 2-3 were deleted to reduce respondent burden.
LEA 3-5, 3-7 (new LEA 3-3, 3-5) School 3-8, 3-10 (same as new 3-8, 3-10) SEA 3-47, 3-48, 3-53 (new SEA 3-19, 3-20, 3-25)	Under family engagement strategies, add item on "Staff work with community-based organizations to conduct outreach to students who dropped out or lost contact with school"	We added a row that reads: “School or district staff work in partnership with community-based organizations to conduct outreach to students who dropped out or lost contact with school.” With this phrasing, we are trying to ensure school staff are working in partnership, not outsourcing the activity.
LEA 3-10 (new LEA 3-7)	Add a row asking "Did tutors working with English learners receive training on how to effectively support them?"	We added a slightly revised version of this row: “Did tutors working with English learners receive any specific training on working with English learners?”
School 1-13 (new school 1-7): Which teachers at your school did the teacher leaders provide individualized (one-on-one) coaching to during the 2021–22 school year? (Select all that apply.)	Add choice: “General education teachers of English learners.”	Instead of adding this choice (which is likely to vary based on the school’s population of ELs rather than a school’s policy), we opted to cover coaching of teachers of special populations by adding two new response options to question 1-7: “English learner specialists” and “special education teachers.”
School 3-16 (new School 3-14)	Add option after the first option: "The same materials used in the students’ support services such as reading interventions or English as a second language instruction."	In order to capture this idea but not increase burden, we added support services to regular classroom instruction (with an “or” clause) in the existing first option. We will use a hover definition for support services that includes the rest of the clause, to keep the item short.
LEA 3-13 (new LEA 3-10) School 3-17 (new School 3-15)	Add row: "English learners were offered instruction in English language development"	We added this row.
School 3-27 (new School 3-21)	Add a row: "Provided parent/family learning opportunities such as adult English classes, digital literacy, or orientations to community services"	We added this row.