Q-CCIIT PDT SSA_REVISED_7.9.18_clean

Professional Development Tools to Improve the Quality of Infant and Toddler Care
(Q-CCIIT PD Tools) Project

OMB Information Collection Request

New Collection

Supporting Statement

Part A

February 2018

Updated July 2018

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Ann Rivera, Ph.D.

Amy Madigan, Ph.D.

CONTENTS

A1. Necessity for the Data Collection 1

A2. Purpose of survey and data collection procedures 3

A3. Improved information technology to reduce burden 9

A4. Efforts to identify duplication 9

A5. Involvement of small organizations 10

A6. Consequences of less frequent data collection 10

A7. Special circumstances 10

A8. Federal Register notice and consultation 10

A9. Incentives for respondents 11

A10. Privacy of respondents 11

A11. Sensitive questions 12

A12. Estimation of information collection burden 12

A13. Cost burden to respondents or record keepers 14

A14. Estimate of cost to the federal government 14

A15. Change in burden 14

A16. Plan and time schedule for information collection, tabulation and publication 14

A17. Reasons not to display OMB expiration date 15

A18. Exceptions to certification for paperwork reduction act submissions 15

REFERENCES 16

TABLES

A.1 Research questions 5

A.2 Data sources, modes, length and timing 7

A.3 We Grow Together technical expert panel membership 11

A.4 We Grow Together field test: honoraria 11

A.5 Total burden requested under this information collection 13

A.6 We Grow Together 2018-2019 schedule for data collection 15

FIGURES

A.1 We Grow Together conceptual framework 4

Appendices

Appendix A. Research questions and sources

Appendix B. 60-Day Federal Register Notice

Appendix C. Mathematica Confidentiality Pledge

Appendix D. Study recruitment materials

Appendix E. Study participant consent form

Attachments (study instruments)

Attachment 1. ECE setting eligibility screener

Attachment 2. PD provider training survey

Attachment 3a. Background survey – Caregiver

Attachment 3b. Background survey – PD Provider

Attachment 4a. Feedback survey – Caregiver

Attachment 4b. Feedback survey – PD Provider

Attachment 5. Website user pop-up questions

Attachment 6. Classroom information roster

A1. Necessity for the Data Collection

The Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS) seeks approval to conduct a field test of We Grow Together: The Q-CCIIT Professional Development System. These professional development (PD) tools and the interactive website featuring the materials were developed to promote high-quality caregiver-child interactions in settings serving infants and toddlers. We Grow Together is based on the principles and practices emphasized by the Quality of Caregiver-Child Interactions for Infants and Toddlers (Q-CCIIT) observational measure. This field test aims to: (1) examine changes in caregiver practice that we expect to be associated with use of the We Grow Together system, and (2) examine implementation of We Grow Together. As a secondary goal, we will further evaluate the psychometrics of the Q-CCIIT measure. Ultimately, findings from this field test will inform refining We Grow Together to make evidence-based practices as user-friendly as possible for caregivers of infants and toddlers.

This request is for collecting data from field test participants from summer 2018 through spring 2019. First, we will screen ECE settings for eligibility in summer 2018. Second, we plan to collect a background survey with caregivers and PD providers in fall 2018.¹ Third, in fall 2018 and spring 2019 we plan two rounds of classroom observations using the Q-CCIIT measure and classroom rosters.² Fourth, we plan to collect data on the use and implementation of We Grow Together through the interactive website from fall 2018 through spring 2019. Finally, we plan to conduct a feedback survey with caregivers and PD providers in spring 2019.

Study Background

Interactions with caregivers are the mechanism through which relationships form and children’s early communication, learning, and skills unfold (Shonkoff and Phillips 2000; IOM and NRC 2015). Children’s developmental outcomes depend not only on their interactions with parents, but also on the care and interactions that take place with caregivers in early childhood education (ECE) settings (NICHD Early Child Care Research Network 2002).

In 2010, ACF funded Mathematica Policy Research to create a reliable and valid research-based observational tool to measure the quality of interactions between infants and toddlers and their caregivers in ECE settings.³ This instrument, the Q-CCIIT, measures the support that caregivers provide for the social-emotional, language and literacy, and cognitive development of infants and toddlers, and also identifies areas of concern (Atkins-Burnett et al. 2014a). ACF designed the Q-CCIIT measure for use in multiple types of settings and for multiple purposes, including PD. Results of a psychometric field test in 400 classrooms across the United States support the reliability and validity of the Q-CCIIT measure (Atkins Burnett et al. 2014b).The principles and practices emphasized in the Q-CCIIT measure serve as the basis of We Grow Together: The Q-CCIIT Professional Development System.

In 2015, ACF funded a contract to develop and evaluate a PD system to support the quality of caregiving for infants and toddlers. Very few studies have tested the effectiveness of PD strategies in improving the quality of interactions between non-parental caregivers and infants and toddlers (Zaslow et al. 2010). The We Grow Together intervention is designed to help caregivers understand how their interactions with children support the children’s development. We Grow Together is available on a secure interactive website, includes training videos, materials, and exercises to be used by caregivers, with the help of local PD providers, to support high quality practices with infants and toddlers.

Drawing on a literature review⁴ (Aikens et al. 2016) and the principles of high quality caregiving reflected in the Q-CCIIT measure (support for the social-emotional, language and literacy, and cognitive development of infants and toddlers), the team developed a conceptual framework that captures the learning process between a local PD provider and an infant or toddler caregiver. The We Grow Together conceptual framework recognizes the roles played by the characteristics of the PD provider, the caregiver, and the caregiver’s setting in the implementation and success of PD. The framework also situates the PD process within a trusting PD provider-caregiver relationship. Grounded in research, the framework includes recognition of the “who” (infant-toddler caregivers from diverse settings), “what” (Q-CCIIT principles and practices), and “how” (process of changing practice) of PD.

We Grow Together is designed to strengthen high quality caregiver interactions with infants and toddlers. The starting point for the We Grow Together process is assessment of a caregiver’s interactions with infants and toddlers in their ECE setting, using the Q-CCIIT observational measure. Data from the Q CCIIT observations highlight areas for growth in the caregiver’s practices and inform a PD goal-setting process, with the help of a PD Provider. We have developed an array of We Grow Together materials targeting principles and practices assessed by the Q-CCIIT measure, organized into modules, and delivered on an interactive website. Over the course of a program year, each caregiver in the study will work with a PD provider who is trained in guiding caregivers through the use of We Grow Together materials.

As part of the We Grow Together intervention, all PD providers will receive training in using We Grow Together to support caregivers’ understanding and practices. PD providers’ training will focus on understanding the caregiving principles, relationship skills, adult learning principles, and coaching strategies that facilitate use of We Grow Together and promote growth of caregivers’ knowledge and skills. PD providers will also have access to their own set of materials to support implementation of We Grow Together. These materials support the ongoing relationship-based PD between the PD Provider and caregiver.

Legal or administrative requirements that necessitate the collection

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose of survey and data collection procedures

Overview of purpose and approach

The purpose of information collected under the current request is to examine the implementation and the outcomes associated with use of We Grow Together. The goals of the field test will be to: (1) implement We Grow Together for five months in a variety of settings, (2) collect information on the implementation of We Grow Together, and (3) collect information on care practices we hypothesize are associated with use of We Grow Together. We propose the following data collection activities, to be carried out in summer 2018 through spring 2019:

• An ECE setting eligibility screening (Attachment 1) will determine if a setting is eligible and if center directors and individual personnel are willing to participate in the field test.

• A PD provider training survey (Attachment 2).

• A background survey, completed by caregivers (Attachment 3a) and PD providers (Attachment 3b).

• A feedback survey, completed by caregivers (Attachment 4a) and PD providers (Attachment 4b).

• Web-user pop-up questions for caregivers and PD providers, collected monthly at website login (Attachment 5).

• A classroom information roster, completed by caregivers at the time of the observation, about the composition and activities within care settings (Attachment 6).

• Observations of settings using the Q-CCIIT measure, completed by study team staff with no burden to participants, to assess the quality of care provided by participating caregivers.

Research questions

Table A.1 outlines the We Grow Together field test’s primary and secondary research questions. In consultation with the project’s technical expert panel, and drawing from the literature review findings (Aikens et al. 2016), the team developed these research questions to address gaps in knowledge and measurement. Appendix A provides a detailed list of the constructs, methods, and respondents we propose to address specific research questions.

Our research questions and the specific constructs we intend to measure are guided by a conceptual framework that summarizes findings from the literature review and consultation with experts. The conceptual framework, shown in Figure A.1, illustrates how caregiver and PD provider characteristics interact in the PD process. The We Grow Together system is designed to support high quality, responsive caregiving that promotes better outcomes for children. The system is adaptable to a variety of caregivers and care settings, including center-based and family child care (FCC) settings.

Figure A.1. We Grow Together conceptual framework

Table A.1. Research questions

Research questions (RQs)

1. Can We Grow Together be used effectively by early childhood professionals to support improvement in caregivers’ interactions with infants and toddlers? [Primary/Secondary]

a. Is five months’ implementation of We Grow Together associated with change in the quality of caregiver-child interactions, as measured by Q-CCIIT instrument scores (fall to spring)? [Primary]

b. Is five months’ implementation of We Grow Together associated with change in caregivers’ and PD providers’ beliefs and knowledge about child development and caregiving (fall to spring)? [Primary]

c. Does the PD provider perceive change in his or her own practice after PD providers’ training and five months’ implementation of We Grow Together? [Primary]

d. Do answers to these primary questions differ by subgroups (caregivers and PD providers associated with FCCs versus center-based settings)? [Primary] How are answers to these primary questions associated with characteristics of caregivers and providers (for example, demographics, FCC/classroom characteristics, caregiver mental health, PD provider experience and beliefs)? [Secondary]

e. Does the caregiver perceive change in his or her own practice after five months’ implementation of We Grow Together? [Secondary]

f. Does the caregiver report other changes after participating in We Grow Together? [Secondary-exploratory]

2. What tools and support are required for early childhood professionals to use the responsive caregiving principles covered by We Grow Together to improve caregiver-child interactions? [Primary/Secondary]

a. How frequently do caregivers and PD providers make use of We Grow Together over the implementation period, and which We Grow Together tools did they access? [Primary]

b. How do caregivers and PD providers engage with the technological components of We Grow Together (that is, usability of the website, accessing the website and We Grow Together within it, using the tablets)? [Primary]

c. Are participants satisfied with We Grow Together (tool types, content)? [Primary]

d. Are participants satisfied with the We Grow Together process (goal setting, action planning, practice and observation, reflection, feedback, trusting relationship)? [Primary]

e. What are challenges and barriers to We Grow Together implementation in infant/toddler settings? [Primary]

f. Do answers to any of these primary questions differ by subgroups (caregivers and PD providers associated with FCC versus center-based settings) [Primary]. How are answers to these primary questions associated with characteristics of caregivers and providers (for example, demographics, FCC/classroom characteristics, caregiver mental health, PD provider experience and beliefs?) [Secondary]

3. Does the Q-CCIIT reliably assess improvements in caregiver-child interactions in different types of care settings serving infants and toddlers? [Secondary]

a. What is the inter-rater reliability of the Q-CCIIT measure? [Secondary]

b. Does the Q-CCIIT measure’s factor structure hold with a sample that is involved in an intervention? [Secondary]

c. Is there any evidence of the Q-CCIIT measure’s sensitivity to intervention, for example, correlations with caregiver and provider reports of change, or differences in change for those in the top and bottom quartiles of implementation? [Secondary]

Study Design

The research team will use multiple data sources, methods, and analyses to examine change from baseline to follow-up for a purposive sample of caregivers and PD providers who participate in the We Grow Together field test.

We aim to have a diverse study sample that includes caregivers and PD providers, working in a variety of ECE settings, with a range of qualifications and professional experiences. Diversity in care settings and experiences of study participants will allow us to ensure that the We Grow Together system is helpful for a wide range of caregivers and PD providers. The study team plans to sample care settings in 10 geographical areas across the country. The field test will include a purposive sample of 175 center-based classrooms and 125 FCCs, operating both under Early Head Start (EHS) and in community-based settings. Purposive sampling will allow us to include diverse criteria and still ensure efficient recruitment. A site contact at each ECE setting will help us arrange our data collection activities. We will collect baseline information about caregivers and PD providers in fall 2018. We will then train PD providers in use of the We Grow Together system. Caregivers and PD providers will use We Grow Together in the course of their work for approximately five months. Caregivers will be provided with an iPad mini and stand to facilitate consistent access to the website and to use for video recording their own teaching practice for discussion with the PD provider. The video recordings will not be collected or viewed by the study team. Provision of the iPad mini is necessary to ensure that caregivers have the technology they need to participate in the study. During that time, we will monitor implementation. We will collect follow-up information about caregivers and PD providers in spring 2019.

With this design, we will look for change in participants from baseline to follow-up in key areas targeted by the We Grow Together system, and we will also gather information about caregivers’ and PD providers’ use of the We Grow Together system. Although the proposed design cannot determine causality, it allows us to evaluate whether use of the We Grow Together system supports growth in early childhood professionals’ knowledge and caregiving. The findings will drive improvements in the We Grow Together system.

Universe of data collection efforts

Table A.2 provides the data source, mode, length, timing, and respondents for the data collection activities. For the web-based background and feedback surveys, the study team drew, to the extent possible, on questions used in prior studies with early childhood professionals, including the Teachers’ Attitudes About Professional Development (Torff et al. 2005), the Center for Epidemiologic Studies Depression Scale (CES-D; 1.Radloff 1977), the Knowledge of Infant Development Inventory (MacPhee 1981), the Perceived Website Usability Items (Wang and Senecal 2007), the Computer System Usability Questionnaire (CSUQ, Lewis 1995), the National Center for Research on Early Childhood Education Teacher Interview (Pianta et al. 2016),the Administrator Technology Survey (Burris 2014), the Teacher Opinion Survey (Geller and Lynch 2000), the Stages of Change Self-Report Form (Children’s Institute 2009), the Kessler 6 Self-Report Measure (Kessler et al. 2003; Kessler et al. 2010), and the Psychologically Healthy Workplace Program Survey (American Psychological Association 2011). Appendix A specifies data sources, constructs, measures and instruments that align with each research question. In addition, the Background (Attachments 2a and 2b) and Feedback (Attachments 4a and 4b) survey instruments are annotated to identify sources of questions from existing studies, as well as original questions we developed for this study. Below, we briefly describe each of the sources and instruments we plan to use in the study.

Table A.2. Data sources, modes, length and timing

Data source	Respondent(s) group	Mode	Length	Timing
ECE setting eligibility screener	ECE program administrators, FCC providers. PD providers	Telephone interview	15 minutes	Summer 2018
PD provider training survey	PD providers	Web-based survey	10 minutes	Fall 2018
Background survey	Caregivers	Web-based survey, with paper-and-pencil option if respondent requests it	45 minutes	Fall 2018
Background survey	PD providers	Web-based survey, with paper-and-pencil option if respondent requests it	30 minutes	Fall 2018
Feedback survey	Caregivers	Web-based survey, with paper-and-pencil option if respondent requests it	60 minutes	Spring 2019
Feedback survey	PD providers	Web-based survey, with paper-and-pencil option if respondent requests it	45 minutes	Spring 2019
Pop-up web survey	Caregivers	Web-based questions asked every 30 days on the We Grow Together website	10 minutes	Six collections from fall 2018 through spring 2019
Pop-up web survey	PD providers	Web-based questions asked every 30 days on the We Grow Together website	6 minutes	Six collections from fall 2018 through spring 2019
Classroom information roster	Caregivers	Paper-and-pencil form	5 minutes	Fall 2018 and spring 2019
Q-CCIIT measure	Study team	Classroom observation conducted on paper-and-pencil form	n.a. There is no burden associated with this data collection activity	Fall 2018 and spring 2019

n.a. = not applicable.

We will use the Head Start Program Information Report (PIR), state and local lists, and child care resource and referral (CCR&R) agency directories, where available, as administrative data sources for site selection and background characteristics for analysis. For example, the PIR provides information on EHS programs at the program level only. We will use the PIR data for sampling and to obtain basic descriptive information about programs’ structural characteristics and enrollment. We will use web analytics to determine which practices and tools are viewed by the PD providers and caregivers. The study team will log and categorize any questions received from caregivers and PD providers.

Telephone and web survey data collection activities:

ECE setting eligibility screener (Attachment 1). In summer 2018, the study team will use a screener to determine if a setting is eligible and willing to participate. The study team will conduct a 15-minute telephone interview with center-based administrators and FCC providers to gather and/or confirm information about the ECE setting, such as the number of classrooms and the number of infants and toddlers in them. The study team will also confirm if ongoing professional development is taking place in that setting. We will contact up to 745 ECE settings to reach a minimum of 88 center-based administrators⁵ and 125 FCC providers who agree to participate and who meet the study requirements.

PD provider training survey (Attachment 2) – At the completion of the We Grow Together training for PD Providers, the 175 PD providers will complete a 10-minute PD provider training survey online. This survey allows PD providers to provide feedback on training materials and experiences as well as PD provider tools. This will also allow PD providers to share if they feel prepared to support caregivers in the use of We Grow Together.

Background survey – Caregiver (Attachment 3a) and PD Provider (Attachment 3b). At the outset of the field test in fall 2018, 300 caregivers will complete a 45-minute web survey and 175 PD providers will complete a 30-minute web survey. Both surveys ask about their background characteristics (such as demographics, education, experience), depressive symptoms, knowledge and beliefs about early child development, PD provider’s cultural responsiveness, knowledge and beliefs about caregiving, philosophy and curriculum of caregiving, technological literacy, experiences and time spent with PD outside of the We Grow Together field test, awareness of and access to PD resources, administrative and collegial support, and willingness to change their practices. Participants will have the option to complete a paper-and-pencil survey, if they prefer.

Feedback survey – Caregiver (Attachment 4a) and PD provider (Attachment 4b). At the completion of the PD period in spring 2019, the study team will ask 300 caregivers to complete a 60-minute web survey and 175 PD providers to complete a 45-minute web survey. All participants answer a subset of the same questions that were in the background survey, to allow us to examine whether there are changes from fall to spring in areas such as knowledge and beliefs about child development, attitudes and beliefs about caregiving, and administrative and collegial support. The survey will ask participants about their experiences with the We Grow Together system, including questions about the materials, website, process, and content. Some questions assess awareness of and access to PD resources and ask about the challenges that caregivers and PD providers might face in implementing PD activities. The caregivers will also be asked about their perceptions of change in caregiving practices, their sense of self-efficacy, and their relationship with the PD provider. The PD providers will also be asked their beliefs and practices about PD. Participants will have the option to complete a paper-and-pencil survey, if they prefer.

Pop-up web survey (Attachment 5). The We Grow Together website will incorporate pop-up questions to 300 caregivers and 175 PD providers participating in the field test. When they first access the secure website, caregivers will be asked to complete a one-time, ten-minute web survey on their learning preferences. This information will be used to recommend specific materials for the caregiver and results will be available to the PD provider and caregiver to help inform the PD process. In addition, over the five-months of the field test, caregivers and PD providers will be asked about how they are using the PD materials and exercises outside of time they spend on the website and how they are working together (for example, how frequently they attended meetings with the PD provider in the last month, and what their methods of communication were). The pop-up questions will take approximately ten minutes to complete for caregivers and six minutes for PD providers. The survey will be asked every 30 days during the 5 months of PD implementation from fall 2018 through spring 2019.

Site visit data collection activities:

Classroom information roster (Attachment 6). In fall 2018 and spring 2019, at the time of the observation visits, 300 caregivers will be asked to complete a 5-minute paper-and-pencil roster to collect information about their classroom. The study team will use the classroom roster to collect information about the ages and genders of children, languages spoken, and the number of caregivers in the classroom.

Q-CCIIT measure. We will conduct observations in each of the 300 classrooms using the Q-CCIIT (Atkins-Burnett et al. 2014b) at two time points―fall 2018 and spring 2019. During the observation, the observer will observe a short (fewer than 10 minutes) small-group book-sharing activity. There is no burden for participants associated with classroom observations.

A3. Improved information technology to reduce burden

Our data collection approach aims to obtain information efficiently while minimizing respondent burden. The study team will ask caregivers and PD providers to complete web-based background and feedback surveys. The pop-up web surveys will be completed when the participant logs in to the We Grow Together website. The We Grow Together website will be available through a user-friendly, learning management site. Participants can access the PD website via the Internet using a range of devices (smartphones, tablets, and computers).

A4. Efforts to identify duplication

There is no other current or planned effort to conduct a field test to examine factors associated with use of We Grow Together that support caregiver-child interactions in settings serving infants and toddlers, and to examine implementation of the We Grow Together system. None of the study instruments asks for information that can be obtained from alternative data sources, including administrative data such as the Head Start PIR, state child care resource and referral agencies, or state and local FCC network lists. The design of the study instruments ensures minimal duplication of data collected across instruments; duplication exists only in cases where the study team needs the perspective of more than one type of respondent to answer specific research questions or where we need to control for baseline information. For example, in the spring feedback survey we will ask the same questions that were in the fall background survey about child development, attitudes and beliefs about caregiving in order to examine any change.

A5. Involvement of small organizations

Some of the child care centers and most of the FCC settings included in the study will be small organizations, including community-based organizations and other nonprofits. The study team will minimize burden for participants by (1) offering web surveys that participants can complete at their convenience and (2) restricting the length of the web surveys. Participants will also have the option of completing the surveys with paper and pencil and giving them to the classroom observer or mailing them back if that is more convenient.

A6. Consequences of less frequent data collection

This is a one-time data collection activity. These data collection activities are necessary for ACF to examine how the Q-CCIIT instrument and related We Grow Together system can be used by a diverse group of early childhood professionals to support improvement in caregivers’ interactions with infants and toddlers.

A7. Special circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register notice and consultation

Federal Register notice and comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on January 3, 2018, Volume 83, Number 2, page 382, and provided a 60-day period for public comment. A copy of this notice is attached as Appendix B. During the notice and comment period, no comments were received.

Consultation with experts outside of the study

The study team consulted with experts to complement the knowledge and experience of the team (Table A.3). Consultants included researchers and PD developers with expertise in professional development services in ECE programs and in the broader realm of child care practices and policies. Throughout the study, the study team will continue to work with selected members of the expert panel.

Table A.3. We Grow Together technical expert panel membership

Name	Affiliation
Camille Catlett	FPG Child Development Institute, UNC-Chapel Hill
Rachel Chazan Cohen	College of Education and Human Development, University of Massachusetts-Boston
James Elicker	College of Health and Human Services, Purdue University
Rena Hallam	University of Delaware
Ursula Johnson	The University of Texas-Houston
Joanne Knapp-Philo	Early childhood education consultant
Sharan Merriam	University of Georgia
Toni Porter	Early childhood education consultant
Heath Ouellette	HHS/ACF/OHS Region IX T/TA Network, a member of the OHS T/TA System
Kathy Thornburg	Professor emerita, University of Missouri Applied Engineering Management Corporation
Kathryn Tout	Child Trends
Claire Vallotton	College of Social Science, Michigan State University
Dale Walker	Juniper Gardens Children's Project, University of Kansas

A9. Incentives for respondents

No incentives are proposed for this data collection.

A10. Privacy of respondents

Respondents will be informed of all planned uses of data, told their participation is voluntary, and assured their information will be kept private to the extent permitted by law.

As specified in the contract signed by ACF and Mathematica (referred to as the Contractor in this section), the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements. All Mathematica employees sign a Mathematica Confidentiality Pledge (Appendix C) that emphasizes the importance of confidentiality and describes employees’ obligations to maintain it.

As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system and establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor will submit a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access. Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individual’s personal identifier.

A11. Sensitive questions

In the background survey, we will ask caregivers about their current symptoms of depression. A long history of research indicates that problems with mental health such as depression, anxiety, and persistent high stress negatively affect parent’s interactions and relationship with children, and the developmental outcomes for children (Cummings and Davies 1994; Liu et al., 2016; Smith 2004; Zeanah 1993, 2000). In addition, persistent high stress and depression affect cognitive functioning making it difficult to learn new behaviors (Joëls et al. 2006). FACES and Baby FACES have included surveys of teachers in Head Start and Early Head Start found that many of the teachers report depressive symptoms (Vogel et al. 2015; Moiduddin et al. 2012). Understanding the challenges to learning new caregiving behaviors as well as strategies and resources that appear to support learning despite the challenges, will be important for understanding how to best support these caregivers of infants and toddlers. This information may help inform revisions to the materials and resources or lead to recommendations for the coaches. In the invitation to participate in the survey, we will inform caregivers that they do not have to answer questions they do not wish to answer, and that none of the responses they provide will be reported back to program staff.

A12. Estimation of information collection burden

Burden hours

Table A.6 provides an estimate of time burden for the data collections, broken down by instrument and respondent. These estimates are based on our experience collecting administrative data from states and administering surveys to program and center directors, PD providers, and caregivers.

Table A.5. Total burden requested under this information collection

Instrument	Total/Annual number of respondents	Number of responses per respondent	Average burden hours per response	Annual burden hours	Average hourly wage	Total annual cost
ECE setting eligibility screener	745	1	.25	186	$22.83	$4,246.38
PD provider training survey	175	1	.17	30	$20.97	$629.10
Caregiver background survey	300	1	.75	225	$11.02	$2,479.50
PD provider background survey	175	1	.50	88	$20.97	$1,845.36
Caregiver feedback survey	300	1	1.0	300	$11.02	$3,306.00
PD provider feedback survey	175	1	.75	131	$20.97	$2,747.07
Caregiver We Grow Together website: user pop-up questions	300	6	.17	306	$11.02	$3,372.12
PD provider We Grow Together website: user pop-up questions	175	5	.10	88	$20.97	$1,845.36
Classroom information roster	300	2	.08	48	$11.02	$528.96
Estimated Annual Burden Total				1,402		$20,999.85

^a The average hourly wage for the study participant consent form is the average of the combined hourly wage for caregivers and PD providers.

Total annual cost

The study team expects the total annual burden to be 1,402 hours, and the annual costs to be $20,999.85, for all of the instruments in the current information collection request. The study team based average hourly wage estimates for deriving total annual costs on data from the Bureau of Labor Statistics, Occupational Employment Statistics (2016). For each instrument included in Table A.6, the team calculated the total annual cost by multiplying the annual burden hours by the average hourly wage.

The mean hourly wage of $22.83 for education administrators of preschool and child care centers or programs (occupational code 11-9031) is used for ECE program administrators. The mean hourly wage for childcare workers (occupational code 39-9011) of $11.02 is used for caregivers. The mean hourly wage of $20.97 for instructional coordinators (occupational code 25-9031) is used for PD providers.

A13. Cost burden to respondents or record keepers

ECE program administrators will use approximately one hour of their time in both fall 2018 and spring 2019. Administrative tasks associated with a site’s participating in the study include assisting with scheduling classroom observations and following up with caregivers who haven’t completed their baseline or follow up surveys. To acknowledge and offset the time commitment of a site’s participation on the program administrator, we propose to offer site contacts honoraria of $25 in the fall, at the program’s enrollment in the study, and $25 in the spring, at the study’s completion.

Table A.5. We Grow Together field test: honoraria

Project activity	Respondent	honoraria	Mode	Length
Data collection coordination (setting level)	Site contact	$25 in fall $25 in spring	In-person coordination	60 minutes in fall and 60 minutes for spring

A14. Estimate of cost to the federal government

The total cost to the federal government for the data collection activities under this current request will be $4,207,867. These costs include respondent recruitment, data collection, data processing and data coding.

A15. Change in burden

This is a new data collection.

A16. Plan and time schedule for information collection, tabulation and publication

Analysis plan

The analysis plan is described in detail in supporting statement B, section B4. Tests of Procedures or Methods to Be Undertaken.

Time schedule and publication

Table A.7 contains the timeline for the data collection and reporting activities. Recruiting of ECE settings will begin in summer 2018, pending OMB approval. Data collection will follow and is expected to take place between September 2018 and June 2019. The study team will produce several publications based on analysis of data from the first round.

• The study team will prepare a set of tables describing findings from the field test data collection activities. The intention is to quickly produce findings that can be used by the government.

• The study team will prepare a final report on the data that includes the information from the descriptive tables along with narrative explanation of the findings. The format of the report will be accessible to a broad audience, and we will use graphics and figures to communicate key findings.

• The study team will produce briefs of varying lengths on specific topics of interest to the government. These briefs will be accessible to a broad audience.

Table A.6. We Grow Together 2018-2019 schedule for data collection

Activity	Timing a
Recruitment
Recruitment of PD providers and caregivers	Summer 2018
Data collection
PD provider training survey Background survey: caregivers and PD providers	Fall 2018
Q-CCIIT measure observations	Fall 2018 and spring 2019
Classroom information roster	Fall 2018 and spring 2019
Website analytics and user inquiries	Fall 2018 through spring 2019
Pop-up web survey	Fall 2018 through spring 2019
Feedback survey: caregivers and PD providers	Spring 2019
Analysis
Data processing Background survey data analysis Website analytics/metrics Help inquiries Feedback survey data analysis	Ongoing, as each data collection is completed Late Fall 2018 Ongoing (Fall 2018 – Spring 2019) Ongoing (Fall 2018 – Spring 2019) Summer 2019
Reporting
Descriptive tables	Fall 2018 and summer 2019
Final report	Winter 2019
Briefs/topical products	To be determined

^a After obtaining OMB approval.

A17. Reasons not to display OMB expiration date

All instruments will display the expiration date for OMB approval.

A18. Exceptions to certification for paperwork reduction act submissions

No exceptions are necessary for this information collection.

REFERENCES

Aikens, N., L. Akers, and S. Atkins-Burnett. “Professional Development Tools to Improve the Quality of Infant and Toddler Care: A Review of the Literature.” OPRE Report 2016-96. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation, 2016.

American Psychological Association. “Stress in the Workplace Survey Summary.” Washington, DC: American Psychological Association, 2011. Retrieved 11/06/2017 from http://www.apa.org/news/press/releases/phwa-survey-summary.pdf.

Atkins-Burnett, Sally, Shannon Monahan, Louisa Tarullo, Yange Xue, Elizabeth Cavadel, Lizabeth Malone, and Lauren Akers. “Measuring the Quality of Caregiver-Child Interactions for Infants and Toddlers (Q-CCIIT).” OPRE report 2014-14. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation, 2014b.

Atkins-Burnett, Sally, Shannon Monahan, Louisa Tarullo, Elizabeth Cavadel, Ashley Kopack Klein, and Lizabeth Malone. “The Q-CCIIT Instrument: Measuring the Quality of Caregiver-Child Interactions for Infants and Toddlers.” Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation, 2014a.

Bureau of Labor Statistics. “Usual Weekly Earnings of Wage and Salary Workers: Third Quarter 2016.” USDL-16-2025. Washington, DC: Bureau of Labor Statistics, October 2016.

Burris, Jade. “Leveraging Technology to Improve Professional Development Opportunities for Early Childhood Administrators Leading Quality Improvement: Identifying Current Practices in Delaware.” Dissertation, University of Delaware, 2014.

Cummings, E. Mark, and Patrick T. Davies. “Maternal depression and child development.” Journal of Child Psychology and Psychiatry, vol. 35, no. 1, 1994, pp. 73-122.

Geller, S., and K. B. Lynch. “Teacher Opinion Survey.” Virginia Commonwealth University Intellectual Property Foundation and Wingspan, LLC, 2000.

Children’s Institute. “The Stages of Change Self-Report Form.” Rochester, NY: Children’s Institute, Inc.2009.

Institute of Medicine (IOM) and National Research Council (NRC). “Transforming the Workforce for Children Birth Through Age 8: A Unifying Foundation.” Washington, DC: The National Academies Press, 2015.

Joëls, Marian, Zhenwei Pu, Olof Wiegert, Melly S. Oitzl, and Harm J. Krugers. "Learning under stress: how does it work?." Trends in cognitive sciences, vol. 10, no. 4, 2006, pp.152-158.

Kessler, R. C., P. R. Barker, L. J. Colpe, J.F. Epstein, J.C. Gfroerer, E. Hiripi, M.J. Howes, S.L. Normand, R.W. Manderscheid, E.E. Walters, and A.M. Zaslavsky. “Screening for Serious Mental Illness in the General Population.” Archives of General Psychiatry, vol. 60, no. 2, 2003, pp.184-189.

Kessler, R.C., J.G. Green, M.J. Gruber, N.A. Sampson, E. Bromet, M. Cuitan, T.A. Furukawa, O. Gureje, H. Hinkov, C.Y. Hu, C. Lara, S. Lee, Z. Mneimneh, L. Myer, M. Oakley-Browne, J. Posada-Villa, R. Sagar, M.C. Viana, and A.M. Zaslavsky. “Screening for Serious Mental Illness in the General Population with the K6 Screening Scale: Results from the WHO World Mental Health (WMH) Survey Initiative.” International Journal of Methods in Psychiatric Research, vol. 19, no. S1, 2010, pp. 4–22; Erratum in International Journal of Methods in Psychiatric Research, vol. 20, no. 1, March 2011, p.62.

Lewis, J. R. “IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use.” International Journal of Human-Computer Interaction, vol. 7, no.1, 1995, pp. 57–78.

Liu, Cindy H., Miwa Yasui, Rebecca Giallo, Ed Tronick, and Larry J. Seidman. “U. S. Caregivers with Mental Health Problems: Parenting experiences and children’s functioning.” Archives of Psychiatric Nursing, Vol. 30, no. 6 , 2016, pp. 753-760.

MacPhee, D. “Manual: Knowledge of Infant Development Inventory.” University of North Carolina, unpublished manuscript, 1981.

Moiduddin, E., Aikens, N., Tarullo, L., West, J., Xue, Y. (2012). Child Outcomes and Classroom Quality in FACES 2009. OPRE Report 2012-37a. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

NICHD Early Child Care Research Network. “Early Child Care and Children’s Development Prior to School Entry: Results from the NICHD Study of Early Child Care.” American Educational Research Journal, vol. 39, no. 1, 2002, pp. 133–164.

Pianta, Robert, and Margaret Burchinal. “National Center for Research on Early Childhood Education Teacher Professional Development Study (2007–2011).” ICPSR34848-v2. Ann Arbor, MI: Inter-University Consortium for Political and Social Research [distributor], 2016-04-12.

Radloff, L.S. “The CES-D Scale: A Self-Report Depression Scale for Research in the General Population.” Applied Psychological Measurement, vol. 1, 1977, pp. 385–401.

Shonkoff, Jack P., and Deborah A. Phillips (eds.). From Neurons to Neighborhoods: The Science of Early Childhood Development. Washington, DC: National Academies Press, 2000.

Smith, Marjorie. "Parental mental health: disruptions to parenting and outcomes for children." Child & Family Social Work 9, no. 1, 2004, pp. 3-11.

Torff, B., D. Sessions, and K. Byrnes. “Assessment of Teachers' Attitudes About Professional Development.” Educational and Psychological Measurement, vol. 65, 2005, pp. 914–924.

Vogel, Cheri A., Pia Caronongan, Jaime Thomas, Eileen Bandel, Yange Xue, Juliette Henke, Nikki Aikens, Kimberly Boller, and Lauren Bernstein. “Toddlers in Early Head Start: A Portrait of 2-Year-Olds, Their Families, and the Programs Serving Them.” OPRE Report 2015-10. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families.

Wang, Jianfeng, and Sylvain Senecal. “Measuring Perceived Website Usability.” Journal of Internet Commerce, vol. 6, no. 4, 2007, pp. 97–112.

Zaslow, Martha, Kathryn Tout, Tamara Halle, Jessica Vick Whittaker, and Bridget Lavelle. “Toward the Identification of Features of Effective Professional Development for Early Childhood Educators. Literature Review.” Washington, DC: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, 2010.

Zeanah, Charles H., Jr. (ed.) Handbook of infant mental health –second edition. New York: Guilford Press, 2000.

Zeanah, Charles H., Jr., (ed.) Handbook of infant mental health –second edition. New York: Guilford Press, 1993.

1 Throughout this statement, “caregiver” is used to refer to teachers in infant and toddler classrooms in center-based settings and child care providers in FCC settings. “PD provider” is used to refer to the person providing professional development assistance, such as coaching or mentoring, designed to support or enhance the caregiver’s practice. These can represent a range of ECE staff, both those working within programs and those employed by outside entities, such as ECE setting managers and education directors, mentors, coaches, employees of technical assistance networks or centers, and master teachers.

2 Throughout this statement, “classroom” is used to refer to groups in both centers and family child care (FCC) settings.

3 Throughout this statement, “ECE setting” is used to refer to a classroom or FCC.

4 The literature review summarizes the state of the field, highlighting the most promising methods and approaches for enhancing caregivers’ interactions with young children, particularly caregivers serving infants and toddlers, those with limited education, and those in home-based and FCC settings.

5 This assumes two caregivers/classrooms per center selected to participate.