School Readiness Goals_Part A_revised 8_15

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval to conduct a research study with Head Start grantees in order to improve understanding of how local programs define, measure, and communicate school readiness goals, and how they use these goals in program planning to improve program functioning. The study design will include a telephone survey of key personnel at approximately 90 local Head Start and Early Head Start programs, followed by site visits to a subset of 12 of these grantees.

Study Background

The Head Start Act of 2007 mandates that Head Start grantees develop locally defined school readiness goals as part of their annual self-assessment. In addition, grantees’ establishment and use of school readiness goals is one of the seven components on which grantees are evaluated in the new Head Start Designation Renewal System. As a result, the question of how Head Start programs interpret and implement these new requirements is of great interest to ACF.

Little is known about the process through which local programs define, measure, and communicate school readiness goals, nor how programs use these goals in program planning or evaluate progress towards their established goals. Anecdotal information gathered from technical assistance providers and communication with grantees indicates Head Start grantees are proceeding with implementation of the new requirement, but there has not been a systematic study to learn how they are doing so. Defining school readiness goals is no easy task, and using goals and related data in ways that effectively improve program performance is even more challenging. Given the diversity of local Head Start programs and communities, and the complexities related to assessing school readiness and using goals to improve program quality, there is likely to be considerable diversity in how local programs are interpreting and implementing the new requirements.

In 2012, the Office of Planning, Research and Evaluation (OPRE) contracted with the Urban Institute to conduct a study to learn how local Head Start programs are interpreting and implementing the new requirements to develop and use school readiness goals. A better understanding of what is happening across local grantees operating in a variety of policy and community contexts will help to inform ACF of how the new requirements are working, and will provide preliminary information about how the requirement to set school readiness goals may be affecting Head Start program functioning and program quality. It also will help inform technical assistance efforts designed to assist local grantees with improving processes for setting and using school readiness goals, and help these efforts assist grantees in a way that recognizes and respects the diversity of Head Start and Early Head Start programs. Finally, the information will be used to design measures about the school readiness goal-setting process that can be incorporated into other Head Start research studies where appropriate to study how this process relates to other characteristics of programs, the quality of programs, and to the progress that children make in these programs.

Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose of Survey and Data Collection Procedures

Overview of Purpose and Approach

The goal of this information collection is to improve understanding of how local Head Start and Early Head Start grantees define, measure, and communicate school readiness goals, and how they use these goals in program planning to improve program functioning. The information from this study will provide ACF with a better understanding of how grantees in diverse local contexts are implementing the new national requirements to set and use school readiness goals. ACF hopes to gain some sense of how the process of setting and using school readiness goals is affecting program operations on the ground level. In addition to providing general information to ACF and the public as a whole, the study outcomes are expected to be of particular value for informing technical assistance around the goals requirements. In fact, several Head Start technical assistance centers have already shown great interest in the study and its outcomes as they continue to develop technical assistance related to school readiness goals and the use of data to drive decision-making in local grantees. Finally, as described below, a further purpose is to develop constructs that may be used in future research.

The study will combine a telephone survey of Head Start and Early Head Start program directors and other relevant staff from 90 grantees with follow-up site visits to a subset of 12 of these grantees. In addition, in-depth telephone interviews will be conducted with 4 Head Start directors of American Indian/Alaskan Native (AIAN) grantees. All data collection will focus on a description of school readiness goals set by local programs; the process used to set school readiness goals; contextual factors informing choices made about school readiness goals (e.g., needs of local children and families, program and staff characteristics, and community characteristics); how programs use and analyze data about school readiness goals; how programs report progress on goals; and how school readiness goals and data inform program planning and improvement efforts. Site visit respondents will include program directors and other key staff involved in developing and using school readiness goals, program front-line staff, members of Head Start governing bodies and policy councils, parents of Head Start and Early Head Start children, and representatives from local education agencies.

This study will be exploratory in nature, designed to learn more about how programs set and use school readiness goals, through a survey of program directors and related staff from a sample of programs that is sufficiently varied in nature to capture differences in the ways in which programs may respond to the new requirements. The goal is to talk with a diverse set of grantees in order to get a sense of the range of opportunities and challenges facing grantees and the different needs for technical assistance. Near the completion of this study, the contractor will provide a recommended set of constructs and survey items that could be included in a national survey or data collection instrument, with options for a five-, ten- or twenty-minute survey, for potential future use in other Head Start research studies.

Upon OMB approval, the telephone survey will be conducted in the fall of 2013 (October-November) and the site visits and open-ended telephone interviews with the AIAN grantees will be scheduled to be conducted in the winter of 2013-2014 (December 2013-February 2014). In the fall of 2014, a final report will be released, drawing upon and synthesizing these data to present key research findings about school readiness goals and Head Start program functioning. Two short research briefs drawing on key themes in the final report will also be prepared.

Research Questions

The first objective, addressed through both the telephone survey and site visits, is to study the process by which local Head Start programs develop their school readiness goals. This research objective will be guided by two research questions:

What school readiness goals do Head Start and Early Head Start programs set?
What does the process of setting school readiness goals look like?

The second research objective is to study how school readiness goals are used to drive program improvements, including gathering information about the data systems used to examine progress toward goals. This research objective, also addressed through the combination of the telephone survey and site visits, will be guided by three questions that build in their complexity.

How do programs use and analyze data to monitor progress towards goals?
How do programs report progress on goals?
How do grantees use school readiness goals and data to inform program planning and improvement efforts?

These five research questions are embedded in a conceptual model that depicts the research team’s understanding of how the new requirements for establishing school readiness goals can lead to improvements in program quality and child outcomes, following principles of performance management. As shown in the circular process sketched out in the bottom half of the conceptual model (shown below), grantees are expected to 1) establish school readiness goals; 2) develop and implement plans to achieve goals; 3) evaluate progress towards goals; and 4) refine or adopt plans for program improvement, and then, back at the beginning, consider whether to make refinements in school readiness goals and/or action plans for achieving goals. The four steps align with the four strategic steps outlined in the November 8, 2011 Program Instruction on School Readiness Goals (ACF-PI-HS-11-04). The entire process is complex and iterative: the evaluation of progress can motivate plans for improvement and further plans for action, but it also can feed back into refining goals for school readiness. Finally, the process revolves around program quality and child outcomes, because the process of setting goals and using data to evaluate goals is not being done for its own sake, but in order to influence program quality and improve child outcomes.

As shown in the top half of the conceptual model, the dual and inter-related processes of establishing and using school readiness goals are influenced by the context in which the Head Start or Early Head Start agency operates. Specifically, four key contextual factors influence the process:

The Head Start Child Development and Early Learning Framework (HSCDELF) and other Head Start and Early Head Start standards, guidance and technical assistance. Head Start regulations and guidance, including the Head Start Child Development and Early Learning Framework, as well as related TA materials, provide a common starting point and guidelines for establishing and using school readiness goals.
Child and family characteristics. The age of children served by the program (3-5, 0-3, or 0-5) is expected to influence the setting of school readiness goals, as is consideration of child and family needs, the presence of dual-language learners, and family culture, beliefs and expectations.
Program characteristics. Program structure varies widely across grantees, including service delivery model, program size, auspices, length of service day, funding streams and levels, and governing structure. The research team expects these characteristics to influence the processes of setting goals and using goals. Program leadership and staff also will be influential, including directors’ and staff members’ experience, education, capacity and comfort with data use, and participation in professional development; organizational structure and culture, values, and theory of change; and past choices, such as choice of curriculum and data systems or pre-existing systems of goals and measures.
Community context. Grantees will be making their goals in a specific state context (e.g., State early learning guidelines, quality rating and improvement systems) and local community context (e.g., kindergarten requirements and transition policies of local education agencies and feeder schools; community stakeholders, partners, and resources; community norms and values).

Exhibit 1. Conceptual Model for School Readiness Goals and Head Start Program Functioning

The telephone survey and follow-up visits to selected sites are designed to gather more information about the process of using and setting school readiness goals, and how this is influenced by contextual factors. Exhibit 2 identifies the primary and secondary sources of information we will use to address each research question, and also provides more detail on the secondary questions and probes under each research question.

Exhibit 2. Research Questions

Key Research Questions X = Main data source O = Supplemental data source	Phone Survey	Field Work	Document Review
1. What school readiness goals do Head Start and Early Head Start programs set?
How many goals do they set and in what domains? How do they align to the OHS goals from the HSCDELF?	O	O	X
What school readiness measures do they use and why? How do measures align with goals?	O	O	X
2. What does the process of setting school readiness goals look like?
What are the steps? When did grantees begin the process and first finalize a set of goals?	O	X
Who is involved in the process?	X	X
What role do various contextual factors play in influencing the process (e.g., HSCDELF, program characteristics, child and family characteristics, community context)? What are the challenges? What are the facilitating factors?	X	X
How do programs understand and approach the new requirements?	X	X
Do programs set priorities either in the goal-setting process, or in their efforts to monitor and make progress towards goals? Which goals do they prioritize and why?	O	X	O
3. How do programs use and analyze data to monitor progress towards goals?
How often, and what times of the year, do programs analyze child-level and aggregated data, who leads the analysis, and what types of analyses do they conduct? What are the challenges? What are the facilitating factors?	O	X	O
What criteria do they use to determine whether children are progressing towards goals?	O	X	O
4 How do programs report progress on goals?
How and when are the data shared?	X	X
What do they report to: staff, parents, the governing body, the policy council, and community stakeholders?	X	X	O
What input do these groups have on how the data are interpreted and used, both for individual and groups of children?		X
5. How do grantees use school readiness goals and data to inform program planning and improvement efforts?
To what extent do program directors make decisions about allocating resources and targeting strategies based on progress towards school readiness goals?	O	X
What facilitates and what acts as an obstacle to using school-readiness related goals for ongoing program planning and improvement?	O	X

Study Design

The study design is based on methods suitable for exploratory research, in which the primary purpose is to gather information about the lay of the land, rather than to test hypotheses or evaluate outcomes. Plans for data collection include: a telephone survey of Head Start and Early Head Start (EHS) grantees; site visits to a subset of the same grantees to learn more about how they set and use school readiness goals; telephone interviews with a small group of AIAN Head Start grantees; and document review.

The sample for the telephone survey will include 90 grantees, purposively selected to obtain information about how the school readiness requirements are implemented across the fullest possible range of service delivery models and policy and community contexts. To capture how the process operates in both Head Start and Early Head Start programs, the 90 grantees will include 25 grantees that operate Head Start programs only, 15 that operate Early Head Start programs only, and 50 that operate both Head Start and Early Head Start; in this latter group, 25 grantees will be asked about their Head Start programs and 25 about their Early Head Start programs. Data from the Head Start Program Information Reports (PIR data) will be used to array grantees by type of grant (Head Start, Early Head Start, or programs that operate both Head Start and Early Head Start), and then by other characteristics that program experts, past research, and the study’s conceptual framework suggest may be associated with variations in how local programs define, measure, and communicate school readiness goals, and how they use these goals in program planning to improve program functioning. These other characteristics include program options offered, program size, presence of delegates, agency auspices, ACF geographic regions, language, race and ethnicity of families served, choice of assessment tools, and state school readiness policy context.

The study team will survey one respondent from each of the 90 selected grantees. The respondent may be the Head Start or Early Head Start program director, the education services manager or coordinator, an assistant director, or whoever handles the responsibility of setting and using school readiness goals in the local grantee. After sending a letter to introduce the study and invite programs to participate, members of the research team will call the directors of the selected grantees to answer questions about the study, determine which individual should be surveyed, and schedule the survey date and time (see Appendices A-1 and A-2 for the program recruitment letter and telephone survey recruitment script).

The telephone survey will be guided by a protocol designed for the purpose of this study. The same protocol will be used for all grantees; a skip pattern will be used to customize the protocol so that grantees that operate only one program (Head Start only or Early Head Start only) will not be asked questions that are only pertinent to grantees operating two programs.(see Appendix A-3). Grantees that operate both Head Start and Early Head Start will be instructed to answer their questions for only one program (with random assignment so that approximately 25 respond to questions about school readiness goals in their Head Start program and 25 respond to questions about school readiness goals in their Early Head Start program). Each telephone survey is expected to last approximately 45 minutes. In this time period, there will not be sufficient time to fully explore the process of setting goals and using goals; more in-depth information will be collected in the follow-up site visits to 12 grantees.

The study contractor will purposively select a subset of 12 of surveyed grantees for follow-up site visits, including six Head Start-only grantees and six combined Head Start-EHS grantees. Data from the telephone survey will be used to identify and select programs that have differing processes for setting and using school readiness goals. The goal will be to select programs that offer a range of experience with the process of setting and using school readiness goals, and not limit the study to only high-performing/model programs or low-performing programs. Selected programs will be sent a letter informing them that they have been selected for a follow-up site visit and to ask for their participation in the site visit phrase of the study (see Appendix B-1 for the site visit recruitment letter).

Because the process of setting school readiness goals may differ for programs that serve children 0-3 or children 0-5, as opposed to programs that serve preschool-age children only, the site visit sample will be divided between programs that serve Head Start only and programs that include both Head Start and Early Head Start components. Site visits at Head Start-only grantees are expected to last 1 ½ days, while site visits at combined Head Start-EHS grantees will be ½ a day longer (2 days), to allow extra time on site to capture perspectives from both Early Head Start and Head Start staff and parents.

During the site visits, interviews with program directors and senior managers, one of whom responded to the telephone survey, will be supplemented by discussions with other services managers, program staff who directly serve children (i.e., classroom teachers and home visitors), parents of children attending Head Start and Early Head Start, and local community stakeholders in order to learn more about how local grantees are implementing the requirements to set and use school readiness goals to improve program functioning. These discussions will be conducted through one-on-one and small group interviews that are designed to last 45 to 90 minutes, depending on the respondent (see Appendices B-2, B-3, B-4, B-5, B-6 and B-7).

The site visits to Head Start/Early Head Start program locations will be conducted by two-person teams of Urban Institute researchers with extensive experience carrying out qualitative research of similar size and scope. The two site visitors will conduct the interviews as a team, with the senior researcher responsible for managing the overall relationship with the study site and leading the interviews, and junior researchers responsible for arranging logistics and taking notes during the interviews.

In advance of the visits, the project director and senior researchers will train together by reviewing the protocols for each interview, discussing the intent of each item and probe, and confirming the key pieces of information that will be consistently obtained across respondents and across sites. The protocols are designed to be flexible to fit the circumstances of the selected program, so the probes used for one respondent may vary from another, and certain points may be emphasized more in one location than another, in order to collect data that best reflects each program. Upon return from each site visit, the site visit teams will meet to debrief as a group to share important information or lessons learned, and they will use that time to discuss and resolve any issues with the protocols or approaches to data collection.

The research team also will conduct qualitative, open-ended telephone phone interviews with 4 directors of AIAN Head Start programs, in order to gather exploratory information about the how American Indian and Alaskan Native grantees set and use school readiness goals. (As described in Part B, AIAN grantees will not be in the universe for the telephone survey). Given the diversity of experiences across AIAN programs, case studies of four programs will not capture the full range of experiences of this type of grantee. Nevertheless, a short description of the process of setting and using school readiness goals in four AIAN programs will help begin building a knowledge base about the experiences of these grantees. The qualitative telephone interviews with the AIAN program directors are designed to last approximately one hour (see Appendix C).

The final component of data collection involves document review. When grantees are contacted to complete the phone survey, they will be asked for one copy of the school readiness goals adopted by their program and one copy of their organizational chart. The document specifying school readiness goals will be coded by the research team for the purpose of addressing research questions related to the scope and nature of the school readiness goals actually established by Head Start and Early Head Start programs. The organizational chart will be used to customize the telephone survey with staff titles that correspond to titles being used in the program, as well as to facilitate decision-making about the structure of the site visits.

Universe of Data Collection Efforts

The instruments to be used for collecting data are as follows:

Telephone Survey Recruitment Script (A-2): To speak with program directors who were sent a program recruitment letter, determine the appropriate survey respondent, and schedule a time for the telephone survey.

Telephone Survey Protocol (A-3): To collect from program directors or other relevant respondents information on their experience setting and implementing school readiness goals

Interview Guide for Program Directors and Managers (B-2): To follow-up with respondents from the telephone survey and other senior managers and collect more detailed information on their role and perceptions of setting and implementing school readiness goals process.

Interview Guide for Other Managers, Coordinators and Specialists (B-3): To collect information on their role and perceptions of setting and implementing school readiness goals.

Interview Guide for Staff (B-4): To collect information on their role and perceptions of setting and implementing school readiness goals.

Interview Guide for Governing Body or Policy Council Representatives (B-5): To collect information on their role and perceptions of setting and implementing school readiness goals.

Interview Guide for Local Education Agency Representative (B-6): To collect information on their role and perceptions of setting and implementing school readiness goals.

Interview Guide for Parents (B-7): To collect information on their role and perceptions of setting and implementing school readiness goals. Attached to the Interview Guide is a Receipt of Payment documenting that respondents to parent interviews have been offered a token of appreciation for their participation.

Interview Protocol for AIAN Head Start Program Directors (C). To collect information on their role and perceptions of setting and implementing school readiness goals.

Instruments A-2 and A-3 are provided as separate files, along with files containing other Telephone Survey materials (a program recruitment letter (A-1) and telephone interview receipt of payment form (A-4)). Instruments B-2 through B-7 also are provided as separate files, along with other Site Visit materials (the site visit recruitment letter (B-1) and consent forms for site visit interviews (B-8 and B-9)). Spanish translations of the parent interview, receipt of payment form and parent consent form will be prepared and used with Spanish speaking parents who lack English language proficiency. The Interview Protocol for AIAN Head Start Program Directors (C) is also a separate file.

A Program Observational Sheet, which will be used by the research team for collecting descriptive information of the program organization and operational context while on site, is provided in a separate appendix (D). The Program Observation Sheet, completed through direct observation, does not represent a burden to staff and is therefore not included as an information collection instrument in the burden table in section A.12. This is consistent with 44 USC, 5 CFR Ch. 11 (1-1-99 Edition), 1320.3, which indicates that “information” does not generally include facts or opinions obtained through direct observation by an employee or agent of the sponsoring agency or through nonstandardized oral communication in connection with such direct observations.

A3. Improved Information Technology to Reduce Burden

Whenever possible, information technology will be used in data collection efforts to reduce burden on study participants. To facilitate data collection, entry, and management, the final telephone survey protocol will be programmed into a web application, such as Check Box, that interviewers will access on a secure server maintained by the study contractor. This application will prompt interviewers through the protocol and also serve as the mechanism by which telephone interviewers record responses to both closed-ended and open-ended survey items. Because the site visit data collection involves in-person interviews and more open-ended questions than the phone survey, data will be captured differently than in the telephone survey. Each site visit interview session will involve two members of the study team, with one asking questions and a second typing close to verbatim notes capturing key quotes and responses on a laptop. An audio recorder will be used with permission to later confirm direct quotes or other details from the sessions.

Notes taken during the interviews will be analyzed later with the assistance of NVivo (QSR International, Inc), a software package that is designed to assist in managing, structuring, and analyzing qualitative data such as interview text through functions that support the classification, sorting, and comparing of text units. Because sections of text can be coded with multiple codes and cross-referenced, and intersections and unions of codes can be easily classified for analysis, the analyst can conduct queries that analyze patterns and associations within individual interviews and across interviews. Functions that allow for research notes to be attached to sections of text in an ongoing fashion aid with the identification of and analysis of emergent themes and analytic ideas.

A4. Efforts to Identify Duplication

The data requirements for this study have been carefully reviewed to determine whether the needed information is already available. Efforts to identify duplication included a review of OHS reporting requirements, protocols used in monitoring visits, and information collected by regional OHS staff. Although monitoring review teams and OHS regional staff have asked some questions about school readiness goals, these questions do not address the scope of issues addressed in this study and are not asked of as many different populations of stakeholders (e.g., teaching staff, community partners, parents). It was concluded that no existing data sources can provide data needed to answer the study’s research questions.

A5. Involvement of Small Organizations

Information being requested or required has been held to the minimum required for the intended use. Most of the 90 organizations surveyed will be small organizations, including community-based organizations (Community Action Agencies), other non-profit organizations, school districts, government agencies, and for-profit organizations.

Burden will be minimized for respondents by restricting the interview length to the minimum required, by conducting surveys and on-site interviews, at times convenient for the respondent, and by requiring no record-keeping or written responses.

A6. Consequences of Less Frequent Data Collection

This is a one-time data collection.

A7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register Notice and Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on January 16, 2013, Volume 78, Number 11, page 3,431 and provided a sixty-day period for public comment. A copy of this notice is attached as Appendix E. During the notice and comment period, no comments were received.

Consultation with Experts Outside of the Study

The conceptual model, study design, and data collection protocols were reviewed by two academic researchers with expertise in Head Start and Early Head Start:

Rachel Chazan-Cohen, Associate Professor, George Mason University, and

Katherine Magnuson, Associate Professor, University of Wisconsin-Madison.

In addition, the team consulted with several Head Start and Early Head Start experts, including program directors and technical assistance providers:

Jennifer Boss, Director of Early Head Start National Research Center at Zero to Three

Michele Conklin, Assistant Program Director, UCCAC, Inc., Children’s Learning Center

Stacy Dimino, Head Start National Center on Program Management and Fiscal Operations

Pat Fahey, Head Start National Center on Program Management and Fiscal Operations

Teresa Keller-Amaya, Senior Research Analyst, Early Head Start National Research Center

Maureen McDonald, Early Childhood Education Specialist, Massachusetts Head Start Training and Technical Assistance Office

Sarah Merrill, Training Specialist, Early Head Start National Research Center

Aimee Mitchell, Director of Head Start, Children’s Friend

Karen Pucciarelli, Head Start National Center on Program Management and Fiscal Operations Susan Sandall, Director, National Center on Quality Teaching & Learning, University of Washington

The team also consulted with several officials in the Office of Head Start, including:

Amanda Bryans, Office of Head Start, Director of Education and Comprehensive Services,

Angie Godfrey, Office of Head Start, Infant/Toddler Program Specialist,

Sarah Merrill, Office of Head Start

Larissa Zoot, Office of Head Start, Project Officer, Center for Program Management and Fiscal Operations

Jim O’Brien, Office of Head Start, Project Officer, National Center on Quality Teaching & Learning

Finally, the study design was reviewed by two Urban Institute senior researchers outside the immediate project team: Gina Adams, who has expertise in early childhood programs and qualitative research, and Timothy Triplett, a senior methodologist with over 25 years in survey research experience.

A9. Incentives for Respondents

As a token of appreciation, $25 will be offered to each survey participant. This approach expresses gratitude for the respondents’ participation but will not be an excessive amount that could be coercive to program directors targeted in this study. Respondents who withdraw from the study during the survey will still be offered the $25. Respondents may decide whether to have the $25 check written to them personally or to their organization.

In addition, as a token of appreciation, $25 will be offered to each parent participating in an interview. Again this gift is intended to express gratitude for the respondents’ participation but will not be an excessive amount that could be coercive to the targeted population of low-income parents.

These amounts were determined based on the estimated burden to participants and are in line with those offered in prior studies using similar methodologies and data collection instruments (“Determinants of Subsidy Stability and Continuity of Child Care in Illinois and New York” and “Child Care Providers and the Child Care Subsidy System”).

A10. Privacy of Respondents

Prior to the start of each interview, the researchers will assure the respondents that the information provided will be kept private to the extent permitted by law. Specifically, none of the information obtained during the course of the study will be disclosed in such a way that individuals or organizations can be identified by anyone outside the research team, and the respondents will not be quoted by name. If ACF were to request a copy of the telephone survey information, personal identifying information would be stripped before sharing. Other information will not be shared with anyone other than the research staff assigned to the study, all of whom will be required to sign the Urban Institute’s Staff Confidentiality Pledge. See Appendix G.

Verbal consent will be requested from participants in the telephone survey (see instrument (A-3) for the verbal informed consent language that will be used in the interviews). Participants in site visit interviews will be provided with and asked to sign informed consent forms (B-8 and B9)

This study is also under the purview of the Urban Institute’s Institutional Review Board (IRB), which is registered under Federalwide Assurance number 00000189, indicating it adheres to the requirements in the HHS Protection of Human Subjects regulations at 45 CFR part 46. All data collection and security procedures described in this package have been approved by the IRB. See Appendix F for a copy of the IRB Notice of Approval. To receive IRB approval for this study, the data collection effort must adhere to the following principles:

Subjects are informed of the nature of the research and how it will be used, and their consent either obtained or explicitly waived, where risks to them are determined to be minimal.
Adequate provision is made to protect the privacy of subjects and to maintain privacy of data, where promised and as appropriate.
Risks to subjects are minimized to the extent possible within research designs.
Risks to subjects (from the research) are reasonable in relation to anticipated benefits (from the research).
The selection of subjects is as equitable as possible (the burdens and benefits of the research are fairly distributed) and particular attention is paid to research involving vulnerable populations and protected health information.

A11. Sensitive Questions

There are no sensitive questions in this data collection.

A12. Estimation of Information Collection Burden

Exhibit 3 shows estimated burden of the information collection, which will take place within a one year period. The estimates of annualized hour burden include telephone surveys, site visits and the qualitative telephone interviews with AIAN directors.

The telephone survey burden estimates are based on the following assumptions:

Telephone Survey Recruitment Script: Recruitment and scheduling calls will be made to approximately 112 program directors (in order to recruit 90 programs) and 45 program managers (in programs where the program director delegates survey response to another manger) at an estimated average of 10 minutes per call. That is, the 157 expected respondents include:

22 program directors who refuse to participate (assuming participation rate of approximately 80 percent);
90 program directors who agree to participate, including 45 who schedule a time to complete the telephone survey themselves and 45 who delegate the telephone interview to a program manager; and
45 program managers to whom the telephone survey has been delegated.

Telephone Survey Protocol: The survey will include 90 respondents and is estimated to last an average of approximately 45 minutes.

For the site visits, the specific respondents will vary by site since each program is unique in structure and staffing, but our estimate of burden includes the following respondents, spread across the 12 sites. Note that some categories of respondents are found in all 12 sites, but the Early Head Start respondents are limited to the 6 sites that operate both Head Start and Early Head Start programs, resulting in estimates as follows:

Interview Guide for Program Directors and Managers: Interviews with 24 program directors and other key staff involved in the goal-setting process (2 per each of 12 sites visited). These 24 respondents will include 12 respondents who have already participated in the telephone interviews;
Interview Guide for Other Mangers, Coordinators and Specialists: Interviews with 60 other services managers, coordinators, or specialists (approximately 4 from the 6 single-program sites and 6 from the 6 combined program sites);
Interview Guide for Staff: Interviews with 54 front line staff, including teachers and home visitors (one small group interview of 3 in the single-program sites and two small-group interviews of 3 each in the combined program sites);
Interview Guide for Governing Body or Policy Council Members: Interviews with 48 members of governing bodies and policy councils ( 1 member of the governing body and 3 members of policy councils in each of the 12 sites);
Interview Guide for Local Education Agency Representative: Interviews with 12 liaisons from local education agencies (LEA) with whom grantees have an established MOU or partnership (1 in each of 12 sites); and
- Interview Guide for Parents: Interviews with 36 Head Start and Early Head Start parents (2 Head Start parents from each of 12 sites, and 2 Early Head Start parents in the 6 sites with EHS programs).

Finally, the burden estimate for the telephone interviews with AIAN grantees assumes hour-long qualitative interviews are scheduled with four program directors of tribal Head Start programs.

Exhibit 3: Estimated Burden in Annualized Hours and Costs

Instrument (and appendix number).	Total Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Total Burden Hours	Annual Burden Hours	Average Hourly Wage ($)	Total Annual Cost ($)
Telephone Interview Recruitment Script (A-2)	157	1	0.17	27	27	$24.66	$658
Telephone Interview (A-3)	90	1	0.75	68	68	$24.66	$1,665
Interview Guide for Program Directors and Managers (B-2)	24	1	1.5	36	36	$24.66	$888
Interview Guide for Other Managers, Coordinators and Specialists (B-3)	60	1	1	60	60	$24.66	$1,480
Interview Guide with Staff (B-4)	54	1	1	54	54	$14.50	$783
Interview Guide for Governing Body or Policy Council Representatives (B-5)	48	1	0.75	36	36	$24.66	$888
Interview Guide with Local Education Agency Representative (B-6)	12	1	1	12	12	$25.17	$302
Interview Guide for Parents (B-7)	36	1	0.75	27	27	$9.17	$248
Interview Guide for AIAN Program Directors (C)	4	1	1	4	4	$24.66	$99
Estimated Annual Burden Sub-total					323		$7,009

Total Annual Cost

The estimated total annualized cost burden to respondents is based on the burden hours and estimated hourly wage rates for each data collection instrument, as shown in the two right-most columns of Exhibit A-3. These estimates are based on:

an assumed hourly wages of $24.66 for Head Start directors and program managers/coordinators, based on mean hourly wage for “Education Administrators, Preschool and Child Care Centers/Programs”, as reported in the May 2011 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates, http://www.bls.gov/oes/current/oes119031.htm). Similar wages were assumed for members of the governing body and policy council and AIAN Head Start directors.
an assumed hourly wages of $14.50 for Head Start teachers and other front-line staff, based on mean hourly wage for “Preschool Teachers, Except Special Education,” as reported in the May 2011 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates) http://www.bls.gov/oes/current/oes252011.htm. Note that an analysis of the 2011 PIR data suggests annual Head Start teacher salaries are similar to those reported by the BLS. (Center for Law and Social Policy, November 2011, Head Start Participants, Programs, Families and Staff in 2011).
an assumed hourly rate of $9.17 for Head Start and Early Head Start parents This equates to annual earnings of $19,090 for a worker employed full-time year-round. This earnings amount corresponds to 100 percent of the 2012 federal poverty income guidelines for a household three persons, the income limit for Head Start eligibility for a family of this size. (For reference, this assumed wage rate is more than the federal minimum wage of $7.25 per hour.)
an assumed hourly rate of $25.17 for local educational agency liaisons, based on mean annual salaries for “Kindergarten Teachers, Except Special Education,” as reported in the May 2011 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates, http://www.bls.gov/oes/current/oes252012.htm.

A13. Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents or record keepers.

A14. Estimate of Cost to the Federal Government

The estimated annual cost to the federal government for the proposed data collection and analysis is $254,933. This figure includes labor hours, and other direct costs (travel, photocopying, mailing, etc.).

A15. Change in Burden

This is a new data collection.

A16. Plan and Time Schedule for Information Collection, Tabulation and Publication

Site Recruitment and Data Collection, August 2013-February 2014: Recruitment of grantees for the telephone survey and training of telephone interviewers is scheduled to occur in August and September of 2013, upon OMB approval. The telephone survey is scheduled to be conducted in the fall of 2013 (October-November).

Recruitment of 4 AIAN grantees for the qualitative telephone interviews is scheduled to occur in November 2013, and those telephone interviews are scheduled to be conducted in December 2013.

After the telephone survey data are compiled, interviewers will clean the data file and run preliminary descriptive analyses using an analytic software program such as SAS or STATA. During the month of December 2013, the contractor will examine these preliminary data and select 12 programs for site visits , including six Head Start-only grantees and six combined Head Start EHS-grantees. The goal in selecting programs will be to ensure that they have some range in their level of sophistication in setting school readiness goals and using school readiness data. To implement this, the research team will analyze response to selected survey items, such as those regarding the programs’ overall experiences setting school readiness goals, the level of involvement of various stakeholders, and the technological and analytical capabilities of programs, including the presence of a data analyst or data manager. In addition, the selection process will consider balance in terms of grantee organization type, program size, and geographic diversity. Letters will be sent to those programs to invite them to participate, and follow-up phone calls will be made to further recruit programs and schedule the site visits for the months of January and February 2014.

Following site visits, site visitors will clean their notes to create targeted transcripts, meet as a team to debrief on experiences, and decide on a predefined coding scheme that will be used to code the qualitative interview data. Potential categories in the coding scheme include the key factors programs consider when setting their school readiness goals, challenges in setting goals, and communication between staff and parents regarding progress towards goals.

Data Analysis and Reporting, March-September, 2014: During the months of March and April 2014, there will be further analysis of the telephone survey data by a group of quantitative researchers simultaneously with analysis of the site visit data by a separate group of qualitative researchers. In addition, the qualitative telephone interviews with the AIAN grantees will be reviewed and analyzed by the co-Principal Investigator who conducted those interviews.

The telephone interview data for this study is primarily quantitative with several open-ended questions that will be captured as close to verbatim as possible. The data file generated from the interviewing tool will be imported into an analytic software program such as SAS or STATA. Analysts will run various quantitative analyses, including descriptive statistics on each item or computed scale, and crosstabs or means comparison tests by program characteristics (e.g., large versus small). The results will display the frequency with which responses were reported and any differences in responses based on characteristics of the program (e.g., grantee type, size, region, service options, etc.). For example, comparisons by grantee type will look at Early Head Start and Head Start, and grantees with combined programs as compared with Early Head Start only and Head Start only.

Open-ended survey responses will be copied into Microsoft Word and converted into a clean set of notes. Depending on their nature, these items will either be coded into categorical response categories and analyzed quantitatively or coded according to the procedures described below for the interview data collected through site visits.

For the site visits, a small group of qualitative analysts will code each interview transcript and subsequently analyze the coded themes to identify patterns across sites and differences in patterns based on site characteristics (e.g., Head Start only sites versus combined Head Start and Early Head Start sites). Coding and analysis will be done with the assistance of NVivo (QSR International, Inc), a software package that is designed to assist in managing, structuring, and analyzing qualitative data such as interview text through functions that support the classification, sorting, and comparing of text units.

Analysts will code each interview/focus group transcript independently following the predefined coding scheme and revise the scheme adding new categories and subcategories as the data informs the coder. Coding will be aggregated by respondent type to examine common themes across program directors, teachers, parents, etc. Coding will also be compared within each site to look for patterns in responses within a given program and whether there were consistencies or inconsistencies in the information provided or differences in perspectives. Analysts will also compare the six sites offering only Head Start to the six sites offering both Head Start and Early Head Start to examine similarities and differences, as well as examine only those with Early Head Start to learn what unique experiences those programs reported.

Two draft research briefs will be written and submitted in May 2014, subsequently revised and resubmitted in final form in July 2014. A draft research report will be written during the months of May through July for submission in late July 2014. The final report will be submitted in September 2014, at which point the contractor will present findings to staff from OPRE, Office of Head Start, and other invited guests.

A17. Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	OPRE OMB Clearance Manual
Author	DHHS
File Modified	0000-00-00
File Created	2021-01-29