Part A TALIS 2024 v13

The Teaching and Learning International Survey (TALIS) is an international survey of teachers and principals focusing on the working conditions of teachers and the teaching and learning practices in schools. The United States will administer TALIS for the third time in 2024, having participated in 2013 and 2018.

TALIS 2024 is sponsored by the Organization for Economic Cooperation and Development (OECD). TALIS is steered by the TALIS Governing Board (TGB), comprising representatives from the OECD member countries, and implemented internationally by organizations contracted by the OECD (referred to as the “international consortium” or “IC”). In the U.S., TALIS 2024 is conducted by the National Center for Education Statistics (NCES) of the Institute of Education Sciences, U.S. Department of Education. Each administration of TALIS is a collaborative effort by the participating countries, guided by their governments on the basis of shared policy driven interests.

TALIS 2024 is focused on teachers’ professional environment, teaching conditions, and their impact on school and teacher effectiveness. TALIS 2024 will address teacher training and professional development, teacher appraisal, school climate, school leadership, instructional approaches, pedagogical practices, and teaching experience with and support for teaching diverse populations.

OECD has scheduled the main study to occur in the Northern hemisphere from February through March 2024 and in the Southern hemisphere from June through August 2024. To prepare for the main study, several TALIS countries conducted pilot studies in February 2022; the U.S. did not participate. Countries also conducted a field test in the first quarter of 2023, primarily to evaluate newly developed questionnaire items and school recruitment materials; the U.S. participated in the field test.

TALIS 2024 includes the core TALIS teacher and principal questionnaires that are required for each participating country, as well as a new and optional Teacher Knowledge Survey (TKS). The TKS is intended to better understand the teacher pedagogical knowledge base at the national level. Results from the TKS can be used to understand the extent to which teachers are prepared with the knowledge and skills for 21st century teaching such as using technology in instruction, teaching in diverse classroom environments, and using modern pedagogical approaches. The US included the TKS in the TALIS 2024 field test and will evaluate these results to determine the feasibility of including TKS as part of the U.S. Main Study. The TKS is a new and proprietary instrument and will not be published for public comment as part of the Paperwork Reduction Act (PRA) review process.

The previous submission (OMB #1850-0888 v.8) requested approval for: (1) recruitment and pre-survey activities for the 2023 field test sample; (2) administration of the field test; and (3) school recruitment and pre-survey activities for the 2024 main study sample.^¹ That package was approved in August 2022. OMB approved the international versions of the field test instruments (OMB#1850-0888 v.9) and the final U.S. versions of the field test instruments (OMB#1850-0888 v.10) in December 2022. In May 2023, OMB approved an increase in the school sample size for the TALIS main study (OMB#1850-0888 v.11). In December 2023, OMB approved the final U.S. versions of the main study instruments (OMB#1850 V 12).

Because TALIS is a collaborative international study, the U.S. administration of TALIS operates under some constraints, particularly around the schedule and the availability of instruments, which are developed and negotiated internationally. In Appendix B, NCES has included the most up-to-date versions of the final U.S. versions of the principal and teacher instruments for the TALIS 2024 main study. Since the completion of the field test, the instruments have been revised with a subset of field test items retained for the main study.

A. JUSTIFICATION

A.1 Importance of Information

As part of a continuing cycle of international education studies, the U.S., through NCES, is currently and in the coming years participating in several international assessments and surveys. TALIS, sponsored by OECD, is one of these studies. TALIS 2024 is part of the larger international program that NCES has actively participated in through collaboration with and representation at the OECD. Through this active participation, NCES has sought to strengthen the quality, consistency, and timeliness of international data. To continue this effort, the U.S. must participate with well-organized and well-executed data gathering activities within the nation’s boundaries. These efforts allow NCES to build a comprehensive data network that reports on the status of education in the U.S. and informs decision-making at the national, state, and local levels.

Participating in TALIS 2024 will allow the U.S. to continue to compare with and learn from other countries working to improve their education systems using a variety of approaches. Reponses from school principals and teachers cover subjects such as professional development they have received; their teaching beliefs and practices; the review of teacher work and the feedback and recognition they received about their work; and various other school leadership, management, and workplace issues.

A.2 Purposes and Uses of Data

The OECD launched the Education Indicators Project (INES) to help create a system of indicators for cross-national comparisons in education for the use of policy makers, consumers, and private industry. INES achieves its purpose by collecting and analyzing a set of key indicators for international comparison; providing an international forum for the exchange of methods and practices of developing and using education indicators for national policymakers; and contributing to evaluation methodology and developing more valid, reliable, and comprehensive indicators for use in policymaking. TALIS has been designed to increase the international information available to OECD countries on teachers, teaching, and the impact that teachers can have on student learning. The overall objective of TALIS is to provide international indicators and policy-relevant analysis on teachers and teaching to help countries develop and review policies that create the conditions for effective schooling.

The TALIS 2024 administration will focus on the ISCED² Level 2 teacher workforce. ISCED Level 2 is also known as lower secondary education; it begins around age 11 and usually lasts between 2 and 6 years. Middle school and junior high school (grades 7 through 9) in the U.S. are classified as ISCED Level 2. The administration of TALIS 2024 will include both classroom teachers in lower secondary education school programs as well as the principals of their schools. Teachers who teach in special needs-only schools, teachers who teach exclusively adults, occasional or emergency teachers, or teachers who are on long-term leave and are not expected to be back teaching at the time of data collection will not be included in the sample.

TALIS Instruments: Every participating country must implement two core questionnaires for TALIS 2024: a principal questionnaire and a teacher questionnaire. These instruments have been developed to address the TALIS 2024 questionnaire frameworks, which define the themes described below in section A.4. Themes captured in the teacher and principal core questionnaires include practices and beliefs about teaching and learning, professional development, job satisfaction, school climate, teacher self-efficacy, and school leadership. These teacher and principal questionnaires include items that have been administered in multiple cycles of TALIS, allowing the investigation of patterns and trends over time. The TKS questionnaire will capture teacher’s pedagogical knowledge as well as background and contextual items.³ Sampled teachers will complete either the core TALIS questionnaire or the TKS questionnaire, but not both. Countries adapt the questions to fit their national context and the questionnaires are reviewed and professionally verified to ensure they remain comparable across countries.

A.3 Improved Information Technology (Reduction of Burden)

The TALIS 2024 design and procedures are prescribed internationally. Data collection will involve teachers completing either the core or TKS questionnaire, each estimated to be completed in 60 minutes. Principals will complete a 45-minute questionnaire. To reduce the amount of paper and printing costs, the TALIS instruments for the field test and main study will be administered online, although paper versions will be made available to participants upon request.

A communication website, MyTALIS.us, was used during the 2023 field test and will be used during the 2024 main study to provide a simple, single source of information to support engaging teachers and principals and maintain high levels of their involvement. This secure portal will inform schools, particularly school coordinators, of their tasks and to provide them with easy access to information tailored for their anticipated needs. We will gather teacher sampling information from participating schools electronically using an adaptation of Westat’s secure E-filing process through the MyTALIS.us portal. E-filing is an electronic system for submitting lists of teacher information, including limited background information from school records. E-filing has been used successfully in the National Assessment of Educational Progress (NAEP) for more than 10 years, and was used in TALIS 2018, as well as in the Trends in International Mathematics and Science Study (TIMSS) 2015 and 2019, the International Computer and Information Literacy Study (ICILS) 2018, and the Program for International Student Assessment (PISA) 2012, 2015, and 2018 assessments. The E-filing system provides advantageous features such as efficiency and data quality checks, and secure data transmission.

A.4 Efforts to Identify Duplication

The U.S., through NCES, has been conducting its own national survey of schools and school personnel called the National Teacher and Principal Survey (NTPS, formerly known as the Schools and Staffing Survey) since 1985. Additionally, the U.S. continues to conduct NAEP, which includes teacher and school questionnaires. The U.S. also participates in several international assessments that have teacher and school questionnaires, including TIMSS, the Progress in International Reading Literacy Study (PIRLS), and ICILS. The teacher and principal questionnaires used in TALIS differ from these studies’ questionnaires in several important ways.

NTPS focus on collecting foundational school and organizational teacher characteristics, such as teacher demand and shortage, teacher and administrator characteristics, school programs, general conditions in schools, perceptions of school climate, and problems in school, teacher compensation, and hiring practices. TALIS items, on the other hand, aim to form connections between these key characteristics and highlight the underlying mechanisms driving teacher practice. More specifically, where NTPS collect a large amount of information on a teacher’s background, training, and in-field or out-of-field credentials, TALIS investigates other factors that may impact instructional practice such as instructional beliefs and the likelihood of utilizing various pedagogical strategies. Notably, TALIS also focuses on gathering the information necessary to make connections between teacher appraisal and feedback and the type, frequency, and duration of professional development opportunities teachers receive.

In addition to conceptual differences in item design, TALIS and NTPS also differ in the measurement of items. In the areas where there is some overlap between these two survey programs’ instruments, the questions for teachers are quite different and not interchangeable. In 2016, NCES convened a task force to compare the most recent NTPS and TALIS questionnaires to identify potential areas of overlap and avoid duplicative efforts in the latest iterations of both surveys. For example, both the NTPS and TALIS measure preparedness for various teaching elements, but TALIS asks about whether various elements were included in the teachers’ formal education or training while NTPS does not. Both questionnaires measure teacher perception of preparedness using the same response options, but TALIS and NTPS focus on different elements. For example, TALIS asks teachers about their preparedness regarding the content of subjects taught as well as pedagogy for subjects taught and general pedagogy. NTPS measures preparedness for content only. Further, NTPS and TALIS measure working conditions differently. Both surveys use a similar definition for the reference period and measure the number of hours spent in instruction. However, TALIS also gathers information on hours spent on a wide range of non-instructional activities not gathered in detail by NTPS. The NAEP, TIMSS, ICILS, and PIRLS studies also have teacher and school questionnaires, but their focus is student achievement in concentrated subject areas. NAEP collects information about teachers’ backgrounds and instructional practices as they relate to student achievement, which can be used to investigate the relationship between students’ achievement and various school and teacher factors that may influence this achievement. TIMSS seeks information about teachers’ background, instructional practices, and attitudes toward teaching a specific subject (mathematics or science). ICILS captures information on the use of Information and Communication Technology (ICT) and student development of ICT literacy. PIRLS intends to supply information about teacher backgrounds, classroom resources, and instructional materials for teaching reading. Therefore, for both conceptual and measurement reasons, alternate sources for these data do not exist.

TALIS 2024 focuses on 11 themes motivated by current theory and research:

Teachers’ instructional practices and beliefs: teaching practices and beliefs with a focus on classroom management, teachers’ views on resources of effective teaching, teachers’ openness to adopting innovative teaching practices, and their views on school conditions and incentives to foster innovation.
School leadership: the profile of school leadership and management styles (including indicators on the roles and functions of school leaders) and distributed/team leadership.
Teachers’ professional practices: teaching practices, cooperation among teaching staff, teaching special education needs students, the pedagogical use of technology (while ensuring there is no overlap with other studies), instruction about sustainability, and student assessment practices.
Teacher education and initial preparation: duration of education, content of formal education or training, pedagogical content knowledge, general pedagogical knowledge, and practical experience.
Teacher feedback and development: in-service professional development (types of activities, participation rates, intensity of participation, mentoring, and induction programs), the needs and demands for in-service professional development, barriers preventing participation in in-service professional development, the perceived impact of in-service professional development, forms of professional development that impact teaching and other aspects of professional practice, connections between teacher feedback and development, feedback as a feature of effective professional development, and feedback supporting innovation in teaching practice.
School climate: disciplinary climate, teacher-student relations, teachers’ working time, and parent-teacher and parent-school relations.
Job satisfaction: overall job satisfaction, teacher perception of the value of the teaching profession, teachers’ views on the factors that would increase their job satisfaction, and perception of the value of the profession.
Teacher self-efficacy: the extent to which teachers feel capable of performing general teaching practices and specific instruction to foster 21^st century skills, the relations between self-efficacy, job satisfaction, and further relevant teacher outcomes, individual differences in self-efficacy with respect to teachers’ age, education, gender, and school environment.
Diversity: policies and practices surrounding the recognition of diversity and supports for inclusion at the school and in the classroom.
Educational use of technology: teachers’ education and professional learning in the use of technology, teaching and professional practices, occupational perceptions, teacher attitudes toward and beliefs about technology, and school leadership and resources.
Social and emotional learning: student task performance, emotional regulation, collaboration, open-mindedness, engaging with others, and motivational aspects of learning.

The TKS questionnaire includes six additional themes:

Teaching methods and lesson planning: productively utilizing instructional time through use of various teaching methods, knowing when and how to apply each method to promote students’ conceptual understanding of learning tasks, and structuring learning objectives, lessons, curricular units, and assessment.
Classroom management: maximizing instructional time through awareness of all classroom activity, handling multiple classroom events concurrently, pacing lessons appropriately to maintain momentum, providing clear directions, and maintaining student attention.
Learning and development: fostering individual learning through knowledge of various cognitive learning processes, including learning strategies, impact of prior knowledge, memory and information processing, causal attributions, effects and quality characteristics of praise, and opportunities for increasing student engagement.
Affective-motivational dispositions: knowledge of motivational learning processes and strategies to motivate a single student or whole group.
Evaluation and diagnosis procedures: knowledge of different forms and purposes of formative and summative classroom assessments, how various frames of reference impact student motivation, and quality of assessment.
Data and research literacy: knowledge of interpreting, evaluating, and using research and data to inform the teaching and learning process.

These themes encompass both new topics deemed important in the current teaching landscape and retain specific topics covered in TALIS 2018. The results of TALIS will inform education policy and spur further investigation into differences within and between countries. Alternate sources for these data do not exist. This submission represents participation of the U.S. in TALIS 2024 involving 53 countries, with eight of those countries also participating in TKS. The U.S. must collect the same information at the same time as the other nations for purposes of making international comparisons. No other study in the U.S. will be using the instruments developed by the international sponsoring organization; thus, no alternative sources of comparable data are available.

To participate in the international study, the U.S. must agree to administer the same core instruments that will be administered in the other countries. Because the items measuring the teaching workforce, the conditions of teaching, and the learning environments of schools have been developed with intensive international coordination, any changes to the TALIS 2024 instruments would also require international coordination. Thus, opportunities to impact the survey instruments are limited to key international dates.

A.5 Minimizing Burden for Small Entities

No small entities are part of the field test sample. The school sample for the TALIS field test will contain a wide range of school types, including private schools, and burden will be minimized wherever possible for all institutions participating in the data collection. For example, for the selection of schools for the TALIS 2024 field test (in February and March 2023) NCES attempted to avoid overlap with the selection of schools for TIMSS 2023 and ICILS 2023, both of which were in the field during spring of 2023. The selection of schools also attempted to avoid overlap with the selection of schools for High School and Beyond, 2022, which was in main study collection in the fall of 2022. Schools included in the field test will have a low likelihood of being included in the main study. In addition, contractor staff will perform as much of the organization and survey administration as possible within each school.

A.6 Frequency of Data Collection

TALIS 2024 is the third administration of TALIS in the U.S. The survey is conducted on a 5- to 6-year cycle, as prescribed by the international sponsoring organization, and adherence to this schedule is necessary to establish consistency in survey operations among the many participating countries.

A.7 Special Circumstances

No special circumstances exist in the data collection plan for TALIS 2024 that would necessitate unique or unusual manners of data collection.

A.8 Consultations Outside NCES

Consultations outside NCES have been extensive and will continue throughout the life of the project. The nature of the study requires this because international studies typically are developed as a cooperative enterprise involving all participating countries. TALIS 2024 is being developed and operated, under the auspices of the OECD, by a consortium of organizations:

Organization for Economic Cooperation and Development
IEA Hamburg (International Study Centre)
IEA Amsterdam
RAND Europe
ACER
cApStAn

A.9 Payments or Gifts to Respondents

School and teacher participation is vital to reaching the international standard for the target sample size needed for a country to be a part of the study. Currently, the minimum response rate targets required by OECD are at least 75 percent of schools after replacement and 75 percent of selected teachers, while the NCES statistical standards require a minimum response rate target of 85 percent at the school and teacher levels.

Historically, these high response rates have been difficult to achieve in school-based studies. In the 2013 cycle, the U.S. did not meet the OECD (or NCES) standards. For the 2018 cycle, NCES used a multi-pronged approach to address the challenge of gaining school and teacher cooperation, which included reviewing the 2013 experience to identify areas for improvement and consulting with colleagues who had expertise in effective approaches to school recruitment. One of the strategies that arose from this work was increasing the dollar values of the incentives for schools, school coordinators, and teachers. In the 2018 cycle, the U.S. met the OECD standards with 77 percent of schools participating after replacement and 90 percent of selected teachers participating.

For the 2024 cycle, NCES plans to use the incentive amounts used in the 2018 cycle:

Schools. Schools participating in TALIS will receive $200. To meet the minimum school response rates mandated by the TALIS international governing board, and to compensate the school for the increased disruption, we believe it is necessary to offer schools this incentive to encourage participation.

School coordinators. The school coordinator will be offered $200. The role of the school coordinator is critical for the success of the study. The coordinator is expected to coordinate logistics with the data collection contractor; supply a list of eligible teachers for sampling to the data collection contractor; and communicate with teachers about the study to encourage participation. The school coordinators are an integral part of the success of TALIS in schools. While we try to minimize their burden, they are our link to the school and teachers. We want to limit as much as we can the possibility of the school coordinator feeling burdened or unrewarded for their time and effort.

Teachers: The field test will implement core and TKS teacher questionnaires delivered on-line with paper versions available upon request. Selected teachers will be offered $25 for completing the questionnaire.

A.10 Assurance of Confidentiality

Data security and confidentiality protection procedures have been put in place for TALIS 2024 to ensure that the contractor and its subcontractors comply with all privacy requirements, including:

The statement of work of this contract;
Family Educational Rights and Privacy Act (FERPA) of 1974 (20 U.S.C. §1232(g));
Privacy Act of 1974 (5 U.S.C. §552a);
Privacy Act Regulations (34 CFR Part 5b);
Computer Security Act of 1987;
U.S.A. Patriot Act of 2001 (P.L. 107-56);
Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9573);
Cybersecurity Enhancement Act of 2015 (6 U.S.C. §151);
Foundations of Evidence-Based Policymaking Act of 2018, Title III, Part B, Confidential Information Protection;
The U.S. Department of Education General Handbook for Information Technology Security General Support Systems and Major Applications Inventory Procedures (March 2005);
The U.S. Department of Education Incident Handling Procedures (February 2009);
The U.S. Department of Education, ACS Directive OM: 5-101, Contractor Employee Personnel Security Screenings;
NCES Statistical Standards; and
All new legislation that impacts the data collected through the contract for this study.

Furthermore, the contractor will comply with the Department of Education’s IT security policy requirements as set forth in the Handbook for Information Assurance Security Policy and related procedures and guidance, as well as IT security requirements in the Federal Information Security Modernization Act (FISMA), Federal Information Processing Standards (FIPS) publications, Office of Management and Budget (OMB) Circulars, and the National Institute of Standards and Technology (NIST) standards and guidance. All data products and publications will also adhere to the revised NCES Statistical Standards, as described at the website: https://nces.ed.gov/statprog/2012/.

By law (20 U.S.C. §9573), a violation of the confidentiality restrictions is a felony, punishable by imprisonment of up to 5 years and/or a fine of up to $250,000. TALIS procedures for maintaining confidentiality include notarized nondisclosure affidavits obtained from all personnel who will have access to individual identifiers; personnel training regarding the meaning of confidentiality; controlled and protected access to computer files; built-in safeguards concerning status monitoring and receipt control systems; and a secure, staffed, in-house computing facility. TALIS follows detailed guidelines for securing sensitive project data, including, but not limited to physical/environment protections, building access controls, system access controls, system login restrictions, user identification and authorization procedures, encryption, and project file storage/archiving/destruction.

Additionally, the contractor will take security measures to protect the web data collection applications from unauthorized access. The Department of Education has established a policy regarding the personnel security screening requirements for all contractor employees and their subcontractors. The contractor must comply with these personnel security screening requirements throughout the life of the contract, including several requirements that the contractor must meet for each employee working on the contract for 30 days or more. Among these requirements are that each person working on the contract must be assigned a position risk level. The risk levels are high, moderate, and low based upon the level of harm that a person in the position can cause to the Department’s interests. Each person working on the contract must complete the requirements for a “Contractor Security Screening.” Depending on the risk level assigned to each person’s position, a follow-up background investigation by the Department will occur.

The laws pertaining to the use of personally identifiable information are clearly communicated in correspondence with states, districts, school administrators, and teachers. Letters and information materials describe the study, its voluntary nature, and the extent to which respondents and their responses will be kept confidential (see Appendix A). Recruitment letters, supporting materials, login pages, and the front cover of each data collection instrument, including questionnaires, indicate:

The National Center for Education Statistics (NCES) is authorized to conduct the Teaching and Learning International Survey (TALIS) by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). The data are being collected for NCES by Westat, a U.S.-based research organization. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

Login pages and the front cover of each data collection instrument, including questionnaires, also include the following statement:

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0888. The time required to complete this information collection is estimated to average up to [45/60/240] minutes per [school administrator/teacher/school coordinator], including the time to review instructions, gather the data needed, and complete and review the information collection. If you have any comments or concerns regarding the accuracy of the time estimate(s), suggestions for improving the form, or questions about the status of your individual submission of this form, write directly to: Teaching and Learning International Survey (TALIS), National Center for Education Statistics (NCES), Potomac Center Plaza, 550 12th Street, SW, 4^th floor, Washington, DC 20202.

OMB No. 1850-0888, Approval Expires 12/31/2025

NCES understands the legal and ethical need to protect the privacy of the TALIS respondents and has extensive experience in developing data files for release that meet the government’s requirements to protect individually identifiable data from disclosure. The contractor will conduct a thorough disclosure risk analysis of the TALIS 2024 main study data when preparing the data files for use by researchers, in compliance with ESRA (20 U.S.C. §9573). Schools with high disclosure risk will be identified and, to ensure that individuals may not be identified from the data files, a variety of masking strategies will be used. IES’s Disclosure Review Board (DRB) carefully reviews all datasets prior to release to ensure that disclosure risks have been properly addressed. The TALIS 2024 data will be reviewed and approved by the DRB prior to any public release, as has been the protocol for all previous rounds of TALIS.

A.11 Sensitive Questions

The TALIS 2024 field test did not include questions usually considered to be of a highly sensitive nature, such as items concerning religion, substance abuse, or sexual activity. The main study also does not include items of a sensitive nature.

A.12 Estimates of Burden

The burden to respondents for the TALIS 2024 field test and main study is calculated for the estimated time required of special handling districts staff and school staff (school administrator, school coordinators, and teachers) to complete recruitment, pre-survey, and survey activities (Table 1). Survey participation includes the time involved to complete teacher and school administrator questionnaires. Recruitment and pre-survey activities include the time involved for (a) special handling districts to review the TALIS research application and (b) schools to decide to participate and who will distribute questionnaire to the school administrators and teachers, completing teacher listing forms, and notifying sampled teachers.

The estimated response burden for schools is based on a 45 minute online school questionnaire for administrators and a 60 minute online teacher questionnaire (each including 15 minutes for instructions and each available in a paper and pencil version available upon request), 90 minutes for school administrators during the recruitment process, and an average of 4 hours for school coordinators to coordinate logistics with the data collection contractor, supply a list of eligible teachers, and encourage teachers to participate in the survey.

We estimate that there may be 9 special handling districts in the field test sample – those known to require completion of a research application before they will allow schools under their jurisdiction to participate in a study. These applications are typically reviewed by a district IRB panel. We assume 6 members per district IRB panel. Estimated burden hours for special handling districts are included under “Special Handling Districts IRB Staff” and “Special Handling Districts IRB Panel” and reflect estimated burden associated with school district staff reviewing and processing special handling district research application materials. Contacting special districts begins with updating district information based on what can be gleaned from online sources, followed by calls to verify the information about where to send the completed required research application forms, and, if necessary, to collect contact information for this process. During the call, inquiry is made about the amount of time the districts spend reviewing similar research applications. The estimated number of such districts represents those with particularly detailed application forms and lengthy processes for approval. This operation began in August 2022 to allow sufficient time for special districts’ review processes. We began contacting these districts upon receiving OMB’s approval and continued to work with them until we received a final response (approval or denial of request) up until March 03, 2023.

The international sample design for TALIS 2024 calls for a minimum of 30 schools in the field test and a minimum of 200 in the main study. Education systems are allowed to use substitute schools (selected during the sampling process) to increase the school response rate. One substitute school is allowed per sampled school in the field test and two per sampled school in the main study. To allow for reliable estimation, while allowing for some amount of non-response, the U.S. is selecting school samples slightly greater than the minimum required. Similarly, for teachers within schools, the minimum sample size was set at 35 teachers to allow for reliable estimation and account for some non-response.

Table 1. Burden estimates for TALIS 2024 field test (FT) and main study (MS)

	Sample size	Expected response rate¹	Number of respondents	Number of responses	Burden per respondent (minutes)	Total burden (hours)
Field Test Recruitment and Pre-Survey Activity
District Administrator	70	1.00	70	70	10	12
Special Handling Districts IRB Staff
Participating Districts	9	0.54	5	5	120	10
Non-Participating Districts	9	0.54	4	4	120	8
Special Handling Districts IRB Panel
Participating Districts	54	0.54	30	30	60	30
Non-Participating Districts	54	0.54	24	24	60	24
School Administrator
Participating Schools	61	0.58	35	35	90	53
Non-Participating Schools	61	0.58	26	26	30	13
School Coordinator	40	1.00	40	40	240	160
Burden – Field Test Recruitment			234	234		310
Field Test Survey Participation
School Administrator	40	0.90	36	36	45	27
Teacher (Core)	800	0.90	720	720	60	720
Teacher (TKS)	600	0.90	540	540	60	540
Burden – Field Test Data Collection			1,296	1,296		1,287
Total Burden Field Test			1,530	1,530		1,597
Main Study Recruitment and Pre-Survey Activity
District Administrator	423	1.00	423	423	10	71
Special Handling Districts IRB Staff
Participating Districts	49	0.54	26	26	120	52
Non-Participating Districts	49	0.54	23	23	120	46
Special Handling Districts IRB Panel
Participating Districts	294	0.54	159	159	60	159
Non-Participating Districts	294	0.54	135	135	60	135
School Administrator
Participating Schools	374	0.58	217	217	90	326
Non-Participating Schools	374	0.58	157	157	30	79
School Coordinator	244	1.00	244	244	240	976
Burden – MS Recruitment			1,384	1,384		1,844
Main Study Survey Participation
School Administrator	244	0.90	220	220	60	220
Teacher (Core)	4,880	0.90	4,392	4,392	60	4,392
Teacher (TKS)	3,660	0.90	3,294	3,294	60	3,294
Burden – MS Data Collection			7,906	7,906		7,906
Total Burden Main Study			9,290	9,290		9,750
Total Burden Requested			10,820	10,820		11,347
¹Expected response rates are based on the TALIS 2018 experience. For details, see U.S. Technical Report and User Guide for the 2018 Teaching and Learning International Survey (TALIS), available at https://nces.ed.gov/pubs2021/2021070.pdf.
NOTE: Details may not sum to totals due to rounding.

For the TALIS field test and main study, a total of 11,347 burden hours are anticipated, resulting in an estimated burden time cost to respondents of approximately $367,965 based on Bureau of Labor Statistics data.⁴

A.13 Total Annual Cost Burden

Other than the time burden cost associated with completing the TALIS questionnaires (estimated above in Section A.12), the field test imposes no additional cost to respondents.

A.14 Annualized Cost to Federal Government

The total cost to the Federal Government for conducting the TALIS 2024 field test as described in the current request is estimated to be $1,131,287. The total cost to the Federal Government for conducting the TALIS 2024 main study (including components not described in the current request) is estimated to be $2,331,787.

Table 2. Estimated costs for TALIS 2024 field test and main study

Components with breakdown	Estimated costs
FIELD TEST (2023)
Recruitment	$190,300
Preparations (e.g., adapting instruments, sampling)	$630,979
Data collection, scoring, and coding	$310,008
Total Cost Field Test	$1,131,287
MAIN STUDY (2024)
Recruitment	$581,232
Preparations (e.g., adapting instruments, sampling)	$143,832
Data collection, scoring, and coding	$979,819
Reporting and dissemination	$626,904
Total Cost Main Study	$2,331,787
Total Cost TALIS 2024	$3,463,074

A.15 Program Changes or Adjustments

This is a reinstatement of a previously approved collection and, as such, shows an increase in respondent burden. Also, the international design calls for an increase in the number of schools in the field test relative to 2018 and TALIS 2024 includes the TKS questionnaire. To allow for reliable estimation for both the core and TKS questionnaires, the sample within schools is increased from 20 to 35 for TALIS 2024. Both of these changes increase the teacher sample size for the field test from 700 teachers in TALIS 2018 to 1,400 in TALIS 2024.

During the field test, the U.S. learned that more schools than expected had fewer than 35 eligible teachers. Additionally, the U.S. experienced slightly lower response rates resulting in a lower-than-expected yield of teacher responses. To achieve the desired analytic power for the main study, the U.S has drawn a sample of 244 schools, 4,880 teachers for the core TALIS questionnaire, 3,660 teachers for the TKS questionnaire. The estimates in table 1 retain the original design assumptions of 35 teachers per school and a target response rate of 0.9 for teachers. This means that the table provides the upper limit of respondent burden for the main study.

A.16 Plans for Tabulation and Publication

For the TALIS 2024 field test there will be no published report of the findings. For the main study, an analysis of the U.S. and international data will be conducted to report on the U.S. national results in relation to the international results. Based on proposed analyses of the international data set by the IC, and the need for NCES to report results from the perspective of a U.S. constituency, a statistical analysis of the U.S. national data set as compared to the international data set will be conducted. Analysis of data will include examinations of the survey instruments of U.S. teachers and school principals in relation to their international counterparts. All reports and publications will be coordinated with the release of information from the international organizing body. Planned publications and reports for the TALIS 2024 main study include the following:

General Audience Report. This report will present information on the status of the teaching workforce, the conditions of teaching, and the learning environments of schools in the U.S. in comparison to the other international participants, written for a non-specialist, general U.S. audience. This report will present the results of analyses in a clear and non-technical way, conveying how U.S. results compare on the international stage, and what factors, if any, may be associated with the U.S. results.

Survey Operations/Technical Report. This report will document the procedures used in the main study (e.g., sampling, recruitment, data collection, scoring, weighting, and imputation) and discuss any problems encountered. The primary purpose of the main study survey operations/technical report is to document those steps taken by the U.S. in conducting and completing the study. This report will include an analysis of non-response bias, which will assess the presence and extent of bias due to nonresponse. Selected characteristics of respondent teachers and schools will be compared with those of non-respondent schools and teachers to provide information on whether and how they differ from respondents along dimensions for which we have data for the nonresponding units, as required by NCES standards.

Electronic versions of each publication will be made available on the NCES website. Schedules for tabulation and publication of TALIS 2024 results in the U.S. are dependent upon receiving data files from the international sponsoring organization. The expected data collection dates and a tentative reporting schedule are provided in table 3.

Table 3. Timeline of activities for TALIS 2024 field test and main study

Activity	Timeframe
Pilot study (the U.S. did not participate in the pilot study)	February 2022
Preparation for field trial data collection	April – December 2022
Begin field test recruitment	August 2022
Field test data collection	February – March 2023
Analysis of field trial data / Preparation for main study data collection	April – December 2023
Begin main study recruitment	June 2023
Main study data collection	February – April 2024
Analysis of main study data; preparation and review of national databases; preparation of reports and dissemination of products	July 2024 – September 2026

A.17 Display OMB Expiration Date

The OMB expiration date will be displayed on all data collection and recruitment materials.

A.18 Exceptions to Certification Statement

No exceptions are requested to the “Certification for Paperwork Reduction Act Submissions” of OMB Form 83-I.

1 The materials that will be used in the 2024 main study are based upon the field test materials included in the v.8 submission. Additionally, that submission was designed to adequately justify the need for and overall practical utility of the full study and to present the overarching plan for all of the phases of the data collection, providing as much detail about the measures to be used as is available at the time of that submission. As part of that submission, NCES published a notice in the Federal Register allowing first a 60- and then a 30-day public comment period. For subsequent substantive changes, including after the field test to finalize the details of the main study, NCES will publish notices in the Federal Register allowing additional 30-day public comment periods.

2 International Standard Classification of Education (ISCED) of the United Nations Educational, Scientific and Cultural Organization (UNESCO).

3 Because the TKS instrument is newly developed and proprietary, it is not included in the package for public comment and review. It will be reviewed and approved directly by OMB in order to protect the intellectual property rights of the holder.

4 In the May 2020 National Occupational and Employment Wage Estimates, sponsored by the Bureau of Labor Statistics, the average hourly earnings for staff types in TALIS are as follows: middle school teachers: $31.39; secondary school teachers: $32.33; noninstructional staff: $20.38; and principals/education administrators: $49.52. When mean hourly wage was not provided, it was computed assuming 2,080 hours per year. The occupation codes (“OCC_CODE”) used were as follows: Elementary and Middle School Teachers (25-2020); Secondary School Teachers (25-2030); Office and Administrative Support Occupations (43-0000); and Education Administrators, Elementary and Secondary Schools (11-9032). Information was retrieved on September 10, 2021, from https://www.bls.gov/oes/current/oes_nat.htm. To calculate the total estimated burden time cost for teachers, we assumed 25 percent of teachers will teach in high schools and 75 percent will teach in middle schools or other types of schools (e.g., K-8, K-12, etc.); this assumption is based on previous TALIS sampling designs.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Mark Low
File Modified	0000-00-00
File Created	2024-07-29