Instrument 1_System-level screening protocol

ITTCC System-Level Screening Protocol (Instrument 1) Mathematica

Infant and Toddler Teacher and Caregiver Competencies Study: System-Level Screening Protocol

1. INTRODUCTION AND REACHING THE PROPER PERSON

[Use contact listed from X as the first point of contact.]

Hello, may I speak with [NAME OF CONTACT]?

IF YOU REACH THE GATEKEEPER:

Hello, my name is ___________________ from Mathematica, an independent research organization. We are conducting a study called the Infant and Toddler Teacher and Caregiver Competencies Study for the Administration for Children and Families within the U.S. Department of Health and Human Services. This study aims to learn how competency frameworks for teachers and caregivers of infants and toddlers can be used to help build the capacity of the infant/toddler care and education workforce and support quality improvement. May I speak with [NAME OF CONTACT] about this study?

[Before you are transferred, get the gatekeeper’s name. If the contact is not available, ask to leave a voicemail on the contact’s direct line.]

VOICEMAIL:

Hello, my name is ___________________ from Mathematica, an independent research organization. We are conducting a study called the Infant and Toddler Teacher and Caregiver Competencies Study for the Administration for Children and Families within the U.S. Department of Health and Human Services. [I/my teammate] sent you a letter on [X], which I hope you received, that requested your participation in this study. I’m calling to follow-up on that letter to tell you a little more about the study, ask you a few questions, and find out about your interest and availability in participating. It would be great to hear from you, feel free to give me a call back at [TELEPHONE NUMBER] or to respond by email at [EMAIL] at your earliest convenience. Thank you!

IF YOU REACH THE CONTACT DIRECTLY:

Hello, my name is ___________________. I am with Mathematica, an independent research organization that does work to inform policy and practice in early care and education. We are conducting a study called the Infant and Toddler Teacher and Caregiver Competencies Study for the Administration for Children and Families within the U.S. Department of Health and Human Services. [I/my teammate] sent you a letter on [X], which I hope you received, that requested your participation in this study. This study aims to learn how competency frameworks for teachers and caregivers of infants and toddlers can be used to help build the capacity of the infant/toddler care and education workforce and support quality improvement. I’m calling to tell you a little more about the study, ask you a few questions, and find out about your interest and availability in participating.

We have taken a close look at [NAME THE COMPETENCY FRAMEWORK] and would like to learn more about how it is being used by a variety of stakeholders. Based on what we learned from reading about the framework, our understanding is [DESCRIBE PERSON’S ROLE/RELATIONSHIP WITH THE FRAMEWORK].

[If not the correct person, get the name, title, and telephone number of the correct person. Say thank you and goodbye. Contact the proper person and start again with the above.]

This discussion will take about [INDICATE LENGTH UP TO 35 MINUTES DEPENDING ON THE NUMBER OF QUESTIONS WE EXPECT TO ASK THIS RESPONDENT]. If now is not a good time or you would like to include others on the call, we can set up another time, preferably within the next 2 weeks, to continue this discussion.

May I proceed, or would you like to schedule another time to continue our discussion?

[IF THEY WANT TO SCHEDULE ANOTHER TIME] When would be the best time to talk in the next 2 weeks? [MAKE A FIRM APPOINTMENT FOR THE FOLLOW-UP CALL.]

[IF WILLING TO PROCEED WITH THE CALL] Great! I’ll provide clarifications on some terms, a brief overview of the study (if needed), ask you a few questions, and discuss next steps.

Before we get started, I just want to confirm that you also received the document I sent shortly before this meeting; this document includes a table that outlines some of what we have learned so far about [NAME THE COMPETENCY FRAMEWORK]. [IF RESPONDENT DOES NOT HAVE THE DOCUMENT, SEND AGAIN BY EMAIL.]

2. BACKGROUND, PURPOSE, AND STUDY OVERVIEW

As I mentioned, we are conducting a study for the Administration for Children and Families within the U.S. Department of Health and Human Services with approval from the Office of Management and Budget. This collection of information is voluntary and will be used to support efforts to improve the quality of care for infants and toddlers. Public reporting burden for this collection of information is estimated to average 35 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-XXXX, Exp: XX/XX/XXXX. As you know, the early care and education field has in recent years embraced the use of competency frameworks as a tool to improve the quality of care provided to infants and toddlers. This study aims to gather lessons about how competency frameworks have been used in early care and education programs and how early care and education systems (such as quality rating and improvement systems, institutes of higher education, credentialing and licensing bodies, training and technical assistance organizations) help to support implementation.

Before we go further, I just want to take a minute to be clear on how we are using certain words in this conversation that can sometimes mean different things to different people. We also describe how we use these words in the frequently asked questions document that we sent by email. By “competency” we mean a piece of knowledge, a skill, or an attribute essential to the practice of teaching and caring for infants and toddlers. By “competency framework” we mean a set of competencies that an organization or entity has compiled to convey the range of knowledge, skills, and attributes that a teacher or caregiver should have. As I mentioned, today we want to speak with you about [SPECIFY COMPETENCY FRAMEWORK] and how it was developed and is used.

We are starting our study by speaking with people like yourself, who are charged with creating a system and structure for and supporting implementation in states, institutions of higher education, and/or professional organizations. After gathering information from people like you who have this big-picture, system-level perspective, we plan to speak with early care and education programs that are using the competency framework. These programs will include center-based settings and family child care homes that care for infants and toddlers.

Do you have any questions so far about the study? [If so, answer to the best of your ability, referring to the FAQ document. If you are not sure of the answer, please say, “That is a good question. I will ask the study team and get back to you as soon as I can.”]

1. CONFIRMING INFORMATION AND IDENTIFYING INTERVIEW RESPONDENTS

As I mentioned, we plan to conduct a series of in-depth telephone interviews to learn more about how [SPECIFY COMPETENCY FRAMEWORK] is used. To prepare for those interviews, we were hoping you could help us confirm the information we found about [SPECIFY COMPETENCY FRAMEWORK] and identify other people we can speak with to learn more about the framework. We will be using the document that we emailed to you shortly before this meeting to guide us through confirming some of the information we found. I would like to record our discussion so I can listen to it later when I write up my notes. No one besides our research team will listen to the recording. If you want to say anything that you don’t want recorded, please let me know and I will be glad to pause the recorder. We will delete all recordings at the end of the study. The information you share today about the characteristics of [SPECIFY COMPETENCY FRAMEWORK] will be summarized in future memos and reports. Please keep in mind that there are no right or wrong answers to any of the questions and you don’t have to answer any questions you don’t want to answer. Do you have any objections to being part of this conversation or to my recording of our discussion?

1. First, I’d like to confirm whether the information that we included in the table about the characteristics of the framework (listed in Panel A) is correct. Are there any changes we should make to the table? [FOR ANY ROW OF THE TABLE THAT SAYS “NOT YET DETERMINED” IN THE DESCRIPTION/RESPONSE COLUMN, MAKE SURE TO ASK THE RESPONDENT TO INDICATE IF THE ANSWER SHOULD BE “YES” OR “NO”; IF THERE ARE ANY ITEMS IN THE NOTES/DETAILS COLUMN, CONFIRM THOSE AS WELL.]

2. Now I’d like to ask you about other people we should speak with (in addition to you) to learn more about these topics. Are you able to answer questions about the development of the competency framework? In particular, we are interested in who was involved and what their roles were; sources of information that were considered in developing competency framework; and how decisions were made about what should be included in the competency framework and how the content should be organized.

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

3. Are you able to answer questions about how various agencies in the early care and education system work together and coordinate to use and promote the competency framework?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

Next, let’s go to Panel B of the template. [FOR ANY ROW OF THE TABLE (ADDRESSED IN ITEMS B THROUGH L BELOW) THAT SAYS “NOT YET DETERMINED” IN THE DESCRIPTION/RESPONSE COLUMN, MAKE SURE TO ASK THE RESPONDENT TO INDICATE IF THE ANSWER SHOULD BE “YES” OR “NO”; IF THERE ARE ANY ITEMS IN THE NOTES/DETAILS COLUMN, CONFIRM THOSE AS WELL.]

4. Can you confirm whether the framework is used for training or technical assistance? [Update table if necessary. If yes, use probe. If no, go to C.]

1. Are you able to answer questions about how the framework is used for training or technical assistance?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

5. Next, can you confirm whether the framework is tied to national or state credentials, certifications, or teaching licenses? [Update table if necessary. If yes, use probe. If no, go to D.]

1. Are you able to answer questions about how the framework is tied to national or state credentials, certifications, or licenses?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

6. Can you confirm whether the competency framework is tied to existing career lattice or career pathways? [Update table if necessary. If yes, use probe. If no, go to E.]

1. Are you able to answer questions about how the framework is tied to existing career lattice or career pathways?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

7. Can you confirm whether the competency framework is tied to other workforce development initiatives? [Update table if necessary. If yes, use probe. If no, go to F.]

   1. Are you able to answer questions about how the framework is tied to other workforce development initiatives?

1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

8. Can you confirm whether the competency framework is used by institutes of higher education? [Update table if necessary. If yes, use probe. If no, go to G.]

1. Are you able to answer questions about how the framework is used by Institutes of Higher Education?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

9. Can you confirm whether the competency framework is used for licensing? [Update table if necessary. If yes, use probe. If no, go to H.]

1. Are you able to answer questions about how the framework is used for licensing centers or family child care homes?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

10. Can you confirm whether the competency framework is used in an existing quality rating and improvement system? [Update table if necessary. If yes, use probe. If no, go to I.]

1. Are you able to answer questions about how the framework is used in an existing quality rating and improvement system?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

11. Can you confirm whether the competency framework is used in any other existing quality improvement initiatives (such as state infant/toddler specialist networks)? [Update table if necessary. If yes, use probe. If no, go to J.]

1. Are you able to answer questions about how the framework is used in any other existing quality improvement initiatives?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

12. Can you confirm whether there are any existing assessments tied the competency framework? [Update table if necessary. If yes, use probe. If no, go to K.]

1. Are you able to answer questions about any existing assessments tied the competency framework (for example, to talk about how the assessment was developed and how it is used)?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

13. Can you confirm whether there are any data currently collected about the competency framework (example, data to monitor the different ways the framework is used or by whom, or data about outcomes associated with the framework)? [Update table if necessary. If yes, use probe. If no, go to L.]

1. Are you able to answer questions about how data are currently collected about the framework?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

14. Can you confirm whether there are current active efforts to share or disseminate information about the competency framework to potential users? [Update table if necessary. If yes, use probe. If no, go to Section II.]

1. Are you able to answer questions about how information about the framework is disseminated or shared?

   1. Who are other people we could speak with about these topics? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL. IF MULTIPLE PEOPLE ARE IDENTIFIED, ASK WHO WOULD BE MOST IMPORTANT TO SPEAK WITH.]

2. IDENTIFY DOCUMENTS
   
   
   

So far, we have located the following documents about how [SPECIFY COMPETENCY FRAMEWORK] is used:

• [LIST DOCUMENTS HERE.]

1. Are there other documents that I have not listed or that are not publicly available that you recommend I review that are related to the topic areas identified in the table we prepared?

   1. What is the best way to obtain these documents?

• [If some documents are available online, ask for the URLs.]

• [If some documents are not publicly available online, ask if they can send copies within the next 2 weeks.]

3. NOMINATIONS FOR PROGRAMS THAT USE COMPETENCY FRAMEWORKS

Thanks for all the information you’ve provided so far! Now that we’ve covered information on the system-level, in this last question, we’d like to get your input on potential early care and education programs—including both Early Head Start and community-based centers, and family child care homes—we can speak with who use competency frameworks. As a reminder, in this study we also aim to provide lessons about how competency frameworks have been used in early care and education programs by those who work with infant/toddler teachers and caregivers (for example, directors or professional development managers) and by infant/toddler teachers and caregivers themselves.

1. Do you know of any programs that serve infants and toddlers that currently use [SPECIFY COMPETENCY FRAMEWORK]? [If yes, use probes. If no, go to B.]

[IF MORE THAN ONE PROGRAM IS RECOMMENDED, REPEAT PROBES/FOLLOW-UP QUESTIONS FOR EACH PROGRAM IDENTIFIED.]

1. What is the name of the program and do you have any information on who we can contact at that program? [FOR EACH PROGRAM IDENTIFIED, ASK FOR: CONTACT NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, AND EMAIL, AND CONFIRM IF WE CAN USE THE RESPONDENT’S NAME WHEN WE REACH OUT TO THE PROGRAM.]

2. Can you confirm, is this a single, independent setting (meaning, a specific center or family child care home), or is this a larger program that includes multiple settings?

   1. [IF MULTIPLE SETTINGS:] Are the settings in this program all centers, all family child care homes, or a mix of centers and family child care homes?

   2. [IF SINGLE SETTING:] Is this a center or family child care home?

   3. [IF SINGLE SETTING:] Is this setting independent, or part of a larger organization with multiple settings?

   4. [IF MULTIPLE SETTINGS BASED ON QUESTION (1) OR (1)(C):] To confirm, does the contact you gave me work at the larger organization or at a specific setting?

3. Is this program supported by a Head Start grant to provide Early Head Start services, and/or involved in an Early Head Start-Child Care Partnership?

1. Do you have any other thoughts about how we might identify programs that are using competency frameworks, including ideas about other people we should contact who might be able to identify programs that are using [SPECIFY COMPETENCY FRAMEWORK]?

   1. If not you, who is the best person to speak with about this topic? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, EMAIL, AND TOPICS. CONFIRM IF WE CAN USE THE RESPONDENT’S NAME WHEN WE REACH OUT TO THE PERSON.]

CLOSING SCRIPT

Thank you for this extremely helpful information.

1. Here are the resources and documents you identified for us during our call [LIST]. Aside from those listed, are there other resources and documents that you think would be helpful for us to look at?

2. And, here are the people you recommended [LIST]. Are there other people that you think would be helpful for us to speak with to learn more about the topics we discussed today? [FOR EACH PERSON IDENTIFIED, ASK FOR: NAME, ORGANIZATION, TITLE, TELEPHONE NUMBER, EMAIL, AND TOPICS. CONFIRM IF WE CAN USE THE CURRENT RESPONDENT’S NAME WHEN WE REACH OUT TO THE PERSON/PEOPLE.]

3. Before we end the call, can you confirm your contact information? [CONTACT INFO]

[NEXT STEPS FOR RESPONDENTS]

Do you have any questions for me?

Thank you so much for your time.

Infant and Toddler Teacher and Caregiver Competencies Study:

[INSERT NAME OF COMPTENCY FRAMEWORK] Overview

Topic	Description/response	Notes/details
Characteristics of the competency framework
Title of competency framework:
Source Definition: source or owner of the framework (for example, a particular state or professional organization)
Types of teachers/caregivers focused on in framework Definition: types of teachers and caregivers that are the focus of the competencies in the framework (for example, infant and toddler teachers, preschool teachers, family child care providers)
Targeted users of framework Definition: all identified users of the framework (for example, teachers/caregivers, ECE program administrators, technical assistance providers)
Governing authority Definition: entity responsible for monitoring and/or enforcing competencies
Funding sources Definition: entity or source of funds responsible for financing development or implementation of competencies
Uses of the competency framework
Is the competency framework used for training or technical assistance?	OPTIONS INCLUDE: yes/no/not yet determined
Is the competency framework tied to national or state credentials, certifications, or teaching licenses?	OPTIONS INCLUDE: yes/no/not yet determined
Is the competency framework tied to existing career lattice or career pathways?	OPTIONS INCLUDE: yes/no/not yet determined
Is the competency framework tied to other workforce development initiatives?	OPTIONS INCLUDE: yes/no/not yet determined
Is the competency framework currently used by institutes of higher education?	OPTIONS INCLUDE: yes/no/not yet determined
Is the competency framework currently used in licensing?	OPTIONS INCLUDE: yes/no/not yet determined
Is the competency framework used in an existing quality rating and improvement system?	OPTIONS INCLUDE: yes/no/not yet determined
Is the competency framework used in any other existing quality improvement initiatives?	OPTIONS INCLUDE: yes/no/not yet determined
Are any existing assessments tied to the framework?	OPTIONS INCLUDE: yes/no/not yet determined
Are data currently collected about the framework?	OPTIONS INCLUDE: yes/no/not yet determined
Are their current active efforts to disseminate information about the framework to potential users?	OPTIONS INCLUDE: yes/no/not yet determined

1