Interview Guide for Program Directors and Managers

OMB Control No.: 0970-xxxx

Expiration Date: xx/xx/20xx

13

School Readiness Goals and Head Start Program Functioning

Site Visit Protocol

Interview Guide for Program Directors and Managers

Public reporting burden for this collection of information is estimated to average 90 minutes per response, including the time for reviewing the instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. This information collection is voluntary. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: Reports Clearance Officer (Attn: OMB/PRA 0970-XXXX), Administration for Children and Families, Department of Health and Human Services, 370 L'Enfant Promenade, S.W., Washington, D.C. 20447.

INTRODUCTION

Thank you for agreeing to meet with us today. I’m [INTERVIEWER NAME] and this is my colleague [ASSISTANT NAME], and we’re researchers from the Urban Institute, a non-profit policy research organization in Washington, DC. Our discussion today builds on the telephone survey that [you/your program] completed for us a couple months ago.

As we may have mentioned, as part of this study, we are visiting 12 Head Start and Early Head Start grantees from across the country.

Our goal is to learn about grantees’ experiences with setting school readiness goals and their opinions towards the school readiness goals requirement, but we are not evaluating whether programs are meeting the requirement or not. The purpose of the study is to inform the Office of Head Start about how Head Start programs are implementing this requirement and what their strengths and needs are.

During our visit to your program, we will be meeting with program leaders, staff, parents, and other key stakeholders to get different perspectives on your program and its goals.

Our meeting with you today will last about 90 minutes. The structure will be rather open-ended, meaning we have a list of specific questions to cover but we welcome any responses you may have.

INFORMED CONSENT

Before I begin my questions, I’d like to give you a copy of a consent form that describes our study procedures and your rights as a participant. If you agree to the study procedures, I’ll ask you to sign and date your copy.

[N0TES TO FACILITATORS: Give copy of consent form to participant. Participant must sign and return one copy and may keep the second copy.]

• I’ll point out that your participation in this study is completely voluntary. You may choose to not answer any question and may stop the interview at any time.

• Everyone who works on this study has signed a Staff Confidentiality Pledge prohibiting disclosure of anything you say during the interview that would allow someone outside the research team, including government staff and officials, to identify you. The only exception is a researcher may be required by law to report suspicion of immediate harm to yourself, to children, or to others.

• Your name and other identifying information, such as the program’s name and specific location, will be removed from the data to protect your privacy.

• We value the information you will share with us today and want to make sure we accurately capture all the details. With your permission, we will audio record the session and take notes (written and/or on a laptop computer). Those notes will not include your name. The recording will serve as a back-up tool to ensure we capture all your comments in as close to your words as possible. Once the project is complete, all recordings will be destroyed. During the discussion, if you would like to stop the recording while you make a particular comment, please let us know and we will do so.

Do you have any questions about the study procedures?

[If anyone objects to recording the discussion, the researcher who is not leading the interview will need to take thorough notes.]

PROTOCOL

[CONSTRUCT: Program context. Leadership background and experience.]

1. To start, would you please tell us your job title, how long you’ve worked in this position, any past positions you’ve held, and a brief description of your primary role and responsibilities in the program.

2. (If respondent was not surveyed) What is your educational background?

[CONSTRUCT: Program context. Child and family characteristics/needs.]

3. Tell us a little about your program’s history and structure.

   1. How many years have you been offering Head Start (and Early Head Start)?

   2. Do you offer any other early childhood programs or services? Tell us about how those programs operate and any funding you receive.

   3. Please describe any community partnerships you have.

4. Tell us more about this community and the children and families that you serve.

   1. What are some of the greatest strengths of this community?

   2. What are some of the greatest needs that you see in this community?

   3. IF HS/EHS: Are the needs and strengths similar in the Head Start and Early Head Start components of your program? How, if at all, are they different?

[CONSTRUCT: Perspectives towards school readiness goals mandate.]

5. During the telephone survey, we asked some questions about your program’s process of developing and setting school readiness goals. I’d like to ask you some more questions about that process. But to start, can you tell me how you first heard about the new requirement of setting school readiness goals?

1. What did you think of the requirement when you first heard about it?

2. Did your program already have school readiness goals or something similar, or was this idea new to you? (Probe for whether the requirement aligns with what they were already doing or if it was new or different.)

[CONSTRUCT: Steps in goal setting process.]

6. Once you learned of the school readiness goals requirement, what steps did you take to establish your program’s goals?

1. How did you or others decide who should lead the goal-setting effort? Why?

2. What role did you have?

(Probe for who wrote the actual goals, who researched what was required, who communicated with staff and parents, etc.)

3. Who else was involved among the staff?

4. How were others involved? (Probe for role of and communication with parents, policy council, governing body, local education agency, local partners, consultants, etc.)

5. Was input from others sought early in the process, or as final decisions were being made, or when?

6. What kind of input was sought/given?

7. From your perspective, was everyone involved in the process who needed to be to make it successful? Or was someone missing or not as involved as you would have liked?

8. Can you tell me about how you went from setting the larger goals to deciding the objectives and measures you’d use for each goal? (Reference copy of goals documents)

9. (If program has not yet finalized goals) When do you anticipate finalizing your goals?

   • What steps do you need to take before you can finalize the goals?

10. IF HS/EHS GRANTEE: How was the process the same or different for setting goals (and objectives) for preschool children and for infants and toddlers?

• What made you decide to have a [single set of goals for 0-5 / different sets of goals for Head Start and Early Head Start]?

(Probe depending on if goals are similar or different for HS and EHS to understand both the process and the outcome, i.e., the goals selected.)

[CONSTRUCT: Key factors in goal setting process.]

7. We want to learn more about the things that shape the actual goals and objectives selected by Head Start programs. As you think about your goals, what kinds of things did you consider when choosing your goals and objectives?

Did you think about [FILL IN] when you were setting your goals? Why or why not?

(Probe for factors listed in our conceptual model:

• Head Start’s Child Development and Early Learning Framework, Performance Standards, guidance, & materials from the Head Start National TA Centers

• Program Characteristics: size, auspice, funding, governance, service delivery model, leadership, staff, values, theory of change, curricula

• Child & Family Characteristics: ages, needs, languages, culture & beliefs

• Community Context: State early learning guidelines, local education agencies and kindergarten requirements, community stakeholders, partners, resources)

• (IF HS/EHS: Probe for differences in factors by age group)

8. How do the school readiness goals you set fit into other planning and improvement efforts going on in your program? (e.g., state school readiness assessment framework, QRIS initiatives, accreditation process)

(Probe for whether or not the requirement aligns with what program was already doing or if this was a novel task.)

[CONSTRUCT: Guidance and TA from Head Start]

9. Did you use any information provided by the Office of Head Start, the national Head Start technical assistance centers, or your State TA providers during the process of setting your school readiness goals? Tell us about the information you used and how you used it.

1. How did you get the information: through the web, printed materials, conference calls, workshops or conferences?

2. What information was most useful? Why?

3. What information was not very useful? Why?

4. How timely was the information you received?

5. What other information or support would have been useful to you?

[CONSTRUCT: Use of non-Head Start resources in goal-setting]

10. What other resources did you use as you were setting your goals?

(Examples: materials from professional organizations, the local school system, and curriculum publisher.)

1. How did you get the information: through the web, printed materials, conference calls, workshops or conferences?

2. What information was most useful? Why?

[CONSTRUCT: Facilitators in goal setting process.]

11. Looking back, what would you say helped move along the whole process of setting school readiness goals?

(Probe: What things made the process easier for you? For example, your past experiences setting program goals, parental engagement, good communication with LEA, good T/TA)

[CONSTRUCT: Barriers in goal setting process.]

12. Were there particular challenges or barriers when setting goals?

1. What concerns, if any, did people have during this process?

2. How were these issues resolved?

(Probe: For example, getting the right mix of people to provide input, lack of understanding of new requirements, lack of time and resources to plan, disagreement regarding goals, lack of T/TA)

[CONSTRUCT: Infusing goals into practice and daily operations]

13. So we’ve talked a lot about how you set the goals. What did you do with the goals after you set them?

1. Did you tell people about your goals? Who did you tell?

(Probe if and how any information was shared with program managers and coordinators, teachers, family service workers, home visitors, parents, policy council, governing body, community partners, and local education agency.)

2. Did you provide information to teachers and other direct service staff to help them understand what the goals mean for their work? How did you communicate this information?

3. Are there any other steps you plan to take with these goals? (Probe if goals have been or will be turned into an action plan.)

4. [If program has or is developing an action plan] What does your action plan include?

1. How did/will you prioritize what should go in the plan?

2. Where are you in the process of implementing the plan?

3. Did you use any information provided by the Office of Head Start, the national Head Start technical assistance centers, or your State TA providers on how to turn the goals into an action plan? Tell us about the information you used and how you used it.

• How did you get the information: through the web, printed materials, conference calls, workshops or conferences?

• What information was most useful? Why?

• What information was not very useful? Why?

• What other information would be useful to you?

14. Thinking back to before you had school readiness goals and now, how, if at all, has your program changed?

[CONSTRUCT: Measuring progress towards goals]

15. Next, I’d like to discuss the process you follow to collect data to measure progress towards goals.

1. What kinds of measures or assessment tools do staff use to assess school readiness? Were the measures selected before or after you set your goals?

2. Why were these measures selected in particular, versus others that might be available to Head Start programs? (Probe measures for DLLs)

3. What kind of data system do you use to store this information?

16. Did your staff receive training in how to use these measures and collect the data?

    1. Who received training?

    2. What did the training involve? (Probe for whether training was a one-time event or repeated to reinforce skills; any plans to reinforce training.)

    3. Do you feel the training was sufficient? Why or why not?

17. How have these measures been working out for you? Are there any goals (or developmental domains) for which you feel you don’t have the right measure or enough measures to really understand whether you’re reaching the goal (or that children are developing)?

18. Are there populations or groups of children for whom you feel that the measures are not working well? For which populations or groups?

[CONSTRUCT: Data use]

16. Once these data are collected, what do you do with the data?

(Probe: Do you submit it to someone else, do you analyze it yourself, do you save it in a file for safekeeping and possible analysis later, or something else?)

1. Who runs the analyses?

   1. What kind of training and skills does that person have? From your perspective, does that person have all the training and skills needed to do that job?

   2. Does this person have other pressing responsibilities that limit his/her ability to produce the data frequently enough or in a fashion that would be useful?

2. What types of data analyses or reports do you see/run?

   1. Do you aggregate the data at the classroom/group level? At the center level? At the program level?

   2. Do you aggregate the data to compare different groups of children? What groups? (Probe for child characteristics, such age, race/ethnicity, language, or gender.)

   3. Do you aggregate the data by teacher or home visitor for the purpose of training and supervision?

   4. Do you compare school readiness data across different years?

   5. What types of analyses do you run for each child?

3. What kinds of things do you look for as you review the information?

4. How do you determine whether children are making progress toward the goals you’ve set? How do you know if there has been “enough” growth?

(Probes: look at school readiness growth in individual children over time, look at growth toward specific skills, compare scores to norms, look at aggregated scores over time within the program)

19. Have you ever had difficulty interpreting the data (or the information in the data reports you received)?

1. What was challenging about it?

2. What would have made it easier for you to understand?

3. Is there any information you would like to get from the data but can’t get? What information?

20. Who else has access to the school readiness data? (Probe who has access to data that they can analyze versus data reports generated by others)

[CONSTRUCT: Communication regarding progress]

21. What, if any, information do you share with [FILL IN FROM LIST] regarding progress towards school readiness goals? (Probe for how, when, and the type of information shared. For staff, whether data are broken down into groups that individual staff members work with, e.g., classes, caseloads.)

1. Staff who work directly with children or families

2. the staff who you supervise (if different from a)

3. the policy council

4. parents

5. the governing body/Board of Directors

6. the local education agency

7. other program partners

[CONSTRUCT: Data-driven decision-making]

22. Have the results you’ve seen from school readiness assessments ever led you to make changes to the program and to the work you (or your staff) do?

1. Can you provide a few examples of changes you’ve made?

(Probe for changes to professional development, curriculum & instruction, assessments, services delivered, program philosophy, etc.)

2. Have these changes been helpful?

3. Are there other changes you would like to see?

23. Have you ever made changes to your school readiness goals based on the results of assessments? What changes did you make? When did that occur?

[CONSTRUCT: Prioritization of goals]

24. Looking back, has the program emphasized certain school readiness goals more than others?

1. Why do you think that’s the case?

2. IF HS/EHS: Are priorities similar for infants and toddlers as for preschoolers?

[CONSTRUCT: Challenges to meeting mandate]

25. What’s been the most challenging for you and your program during this whole process of setting goals, measuring progress towards goals, and using goals to inform your planning?

(Probe: For example, getting staff buy-in, lack of understanding of new requirements, lack of time and resources to plan, disagreement regarding goals, lack of T/TA)

[CONSTRUCT: Supports needed to fully meet mandate]

26. What kinds of resources or support would you and your staff need to overcome these challenges? (If no challenges, then what is needed to better measure progress towards goals and use goals for planning?)

(Probe types of T/TA and at what level)

[CONSTRUCT: Usefulness of school readiness goals requirement]

27. Overall, do you think that the school readiness goals requirement has been useful for your program? Why or why not?

28. We’d be interested in seeing copies of example data reports that you have available so we can learn about the ways in which you look at your data. Do you have any examples you might be willing to show me before the end of our visit?

29. Those are all of our questions. Is there anything else you would like to tell us or that you think we should have asked?