Part A QCCIIT Revised 8_29-11

Measurement Development: Quality of Caregiver-Child Interactions for Infants and Toddlers

Supporting Statement, Part A

For OMB Approval

February 28, 2011

A. JUSTIFICATION

The Administration for Children and Families (ACF) of the Department of Health and Human Services is requesting Office of Management and Budget (OMB) clearance for data collection associated with the Measurement Development: Quality of Caregiver-Child Interactions for Infants and Toddlers (Q-CCIIT) project, a three-year project that began in 2010. The current submission requests clearance for the two-year period of data collection and analysis (FY 2011–2013). This section provides supporting statements for each of the points outlined in Part A of the OMB guidelines.

A.1. Circumstances Making the Collection of Information Necessary

a. Overview of Request

ACF has contracted with Mathematica Policy Research and its subcontractors, Child Trends, FPG Child Development Institute at the University of North Carolina, and WestEd, under contract number HHSP23320095642WC/HHSP23337016T, to develop a new measure to assess the quality of caregiver-child interaction for infants and toddlers in nonparental care.

The legislative base for the Q-CCIIT project is the Improving Head Start for School Readiness Act of 2007 (P.L. 110-134), outlining requirements on monitoring, research, and standards for Head Start and Early Head Start (see Appendix A). The Q-CCIIT measure to be developed and evaluated will provide the Office of Head Start with a reliable and valid observation tool to document caregiver-child interactions for infants and toddlers in nonparental care. This information will satisfy the accountability and program improvement goals mandated by the Act. Further, the Q-CCIIT project includes both Early Head Start settings and community-based settings to address the crucial need of measuring quality in infant and toddler classrooms across diverse settings. The potential audiences include state and county Quality Rating and Improvement Systems (QRIS) that monitor quality, often providing tiered reimbursement for care settings serving children with subsidies. A new tool may then help support the quality assessment process and possibly inform the quality improvement efforts of providers receiving Child Care Development Funds.

b. Project Context and Rationale

The new Q-CCIIT observational measure will assess the quality of child care settings for children from birth through 36 months, focusing specifically on the quality of caregiver‐child interactions.³ The measure will be appropriate for use across child care settings, center‐based and family child care (FCC), and single- and mixed-age classrooms.⁴ Specifically, the Q-CCIIT project is designed:

• To assess the state of the measurement field related to child-adult interactions and quality of child care settings for infants and toddlers

• To develop a measure for assessing the quality of child-caregiver interactions efficiently, yielding reliable and valid data

• To collect data to demonstrate the psychometric soundness of the new measure

• To outline a plan for the sustainability of the measure beyond its development and psychometric testing

After several decades of research demonstrating the generally low quality of infant/toddler child care (Boller et al. 2010; Helburn, 1995), the field is now focused on how best to provide nurturing, safe child care environments for infants and toddlers. Central to this focus is determining how child care quality is defined and measured. With the rise in high-stakes uses of observational measures of child care in state QRIS and in Head Start performance measurement, the availability of reliable and valid measures of infant/toddler care is critical for a range of stakeholders, including policymakers, program developers, child care providers, parents, and researchers. There is a dearth of measures appropriate for assessing the quality of infant/toddler care settings; those that do exist have limitations, ranging from being too specific to a certain type of setting to lacking reliability and both concurrent and predictive validity (Zaslow et al., 2011). Many measures currently available were developed as downward extensions of observations for older preschool children’s settings, which differ in structure and content from classrooms for infants and toddlers.

ACF has proposed that a new measure to assess the quality of infant/toddler care should focus on caregiver-child interactions, because relationships with primary caregivers are the “active ingredients” by which children’s early communication, learning, and competence unfold. The significant role of early relational influences on children’s emerging abilities extends beyond those encountered in the immediate home setting; interactions between young children and their nonparental caregivers have likewise been shown to be important contributors to children’s well-being (Shonkoff and Phillips, 2000). As illustrated in the Q-CCIIT research-based conceptual model (Figure A.1), indicators of process quality, such as sensitive and stimulating caregiver-child interactions, directly influence children’s cognitive, language, and social-emotional development. Structural features of care, such as caregiver professional development and child-to-adult ratio, likewise affect children’s development, albeit indirectly, through influences on process quality (NICHD ECCRN, 2002; National Research Council, 2003; Phillipsen et al., 1997; Vandell & Wolfe, 2000). The quality of early care is multifaceted, and encompasses child-peer relations and effective communication between caregivers and parents (for example, consistency in childrearing philosophy and practices). In recognition of the transactional nature of caregiver-child, child-peer, and caregiver-parent relationships, individuals’ own behavioral styles also contribute to and shape the quality of interactions with others (Sameroff, 2009; Sameroff & Chandler 1975).

Figure A.1.
Q-CCIIT Research-Based Conceptual Model for Infant/Toddler Quality of Care

Several activities will address the goals of the Q-CCIIT project. A literature review has been conducted (1) to provide a summary of the extant measures of quality appropriate for use in nonparental care environments that serve infants and toddlers, and (2) to evaluate the degree to which these measures adequately gauge the features of quality that are important to capture. Drawing on recommendations from the literature review and the relationship-focused conceptual model (Figure A.1), we have outlined a measurement framework for the Q-CCIIT observational measure (Appendix B). Through an iterative development process, the Q-CCIIT measure will be created, tested, and refined to observe child care settings with levels of reliability and validity that meet or exceed the standards in the field. Item development will include further consultation of the literature as well as video review. As part of the iterative process, pretesting of observation components will be conducted and the observers and measure development team will debrief on items and procedures. Once an almost-final measure is prepared, we will conduct a pilot test, to be followed by a psychometric field test with the final measure. With the completion of the data collection activities, the Q-CCIIT project will develop a detailed plan to ensure the sustainability of the new infant/toddler child care quality measure.

A.2. Purpose and Use of the Information Collection

The Q-CCIIT project will address a crucial need to identify and measure quality in infant and toddler nonparental care settings. Building on a strong, research-informed conceptual framework, the Q-CCIIT measure will have the capacity to provide information about the quality of caregiver-child interaction across diverse settings, caregivers, and children and will measure constructs that relate to child development. The information collected will be used to understand the psychometric soundness of the new measure—its reliability and validity to measure what it aims to measure across people and time. We will examine the Q-CCIIT measure’s characteristics by subgroups of setting type (center-based versus FCC) and by age category (infant, toddler, and potentially mixed-age groups) to ensure its utility across diverse settings and ages. With a literature review of the existing and emerging research base as a foundation, the Q-CCIIT project will begin developing and testing a child-caregiver measure for infants and toddlers. Once the development of the Q-CCIIT measure is complete (see section A.1), data collection to refine and test the psychometric properties of the measure will involve multiple phases, which include focus groups with parents, caregivers, and technical assistance providers; a pilot test; and a psychometric field test (Table A.1).

Focus groups. Before we conduct observations for the pilot test, we will hold focus groups to capture feedback from stakeholders about the new measure. These semistructured discussions will also provide evidence for investigating the face validity of the Q-CCIIT measure. To determine that the Q-CCIIT observational assessment measures what it intends to measure, we will ask the group to discuss key aspects of caregiver-child interactions (for example, support for emotional development, self regulation, or cognitive development) from the perspective of what they see in classrooms or consider best practice. We will conduct four focus groups consisting of 10 participants each: two with parents and one each for caregivers and training and technical assistance (T/TA) providers. There will be separate focus groups for English- and Spanish-speaking parents of infants and toddlers. Each of the four semistructured group discussions will last about 1 hour, 55 minutes. So we can document the diversity of the group of people sharing their views, the session will conclude with a 5-minute demographic questionnaire on topics such as type of setting they use for care or provide for children (center-based or FCC), work experience, and race/ethnicity.

Pilot test. A pilot test in early 2012, involving 120 classrooms⁵ and about 560 children from four geographic locations, will allow us to operationalize the almost-final measure and examine the psychometric properties of the measure (including its reliability and validity). As part of conducting the 3-hour observation, we will request to observe a short (fewer than 10 minutes) small-group activity, such as shared book time, during the full observation (for caregivers already do such activities). It will conclude with brief follow-up questions (less than 5 minutes) on how typical the day was and about certain key events if they were not observed (for example, departure interactions with children and families). After the observation, caregivers who spend more than 4 hours a day in the classroom will complete a 15-minute background questionnaire (on topics such as education, experience, child care practices, and depressive symptoms) that could account for variation in observed interactions. We will gather parent-report questionnaires to refine our data collection procedures and to assess concurrent validity, examining the association of the Q-CCIIT measure with child competence. To do this, we will assess the correlation between the Q-CCIIT and the parent-report child competence questionnaire. It contains items about the child’s cognitive, language, and social-emotional development, as well as the demographic characteristics of the family. Child competence items come from published measures: Ages and Stages Questionnaires, 3rd Edition (Squires et al., 2009), MacArthur-Bates Communicative Development Inventories—Short Forms (Fenson et al., 2000), and Brief Infant Toddler Social Emotional Assessment (BITSEA; Carter & Briggs-Gowan, 2005).

We note that the Ages and Stages Questionnaire has been found in preliminary studies with low income samples to have only a weak association with other observational measures of classroom quality, including the CLASS-T and the ITERS-R. However, we will retain the instrument as a placeholder during the pilot phase, and pending further analysis. It is possible that we will replace it prior to the psychometric field test with an instrument that is similar in terms of constructs and burden.

Psychometric field test. The psychometric field test, to be conducted in fall 2012, aims to document the psychometric properties of the final Q-CCIIT measure. It will involve 400 classrooms and about 1,600 children from 10 geographic locations. Its components include the Q-CCIIT observations, validation observations with a second measure of quality (Appendix C), and collection of caregiver background questionnaires and parent-report child competence questionnaires. We will collect reliability evidence for both test-retest reliability (with repeat observations at two time points for 60 classrooms) and inter-rater reliability (across paired observations). We will collect validity evidence through concurrent validation observations (in 100 classrooms) as well as examine the ability of the measure to relate to child competence at two points in time (at the beginning of the field test and in a 6-month followup).

Setting recruitment activities. For both the pilot test and the psychometric field test, Mathematica site coordinators will recruit settings into the Q-CCIIT project. We expect to obtain the names and contact information for a group of potentially eligible settings from public sources such as state and local child care resource and referral agencies. Site coordinators will also attempt to identify the name of the proprietor (that is, the center director or FCC owner) by searching databases and internet sites. Advance letters will be sent to settings, followed by a call to answer any questions the proprietors might have, to determine their eligibility to participate,⁶ and to enroll their setting into the project. Specifically, the Mathematica site coordinator will follow the basic topics discussed in the advance letter, including reviewing the data collection activities, giving tokens of appreciation, and responding to questions. As part of this initial call, the Mathematica site coordinator will collect information on the number of classrooms and the number of infants and toddlers in them, as well identify a setting point person (SPP) who will assist with recruiting families and scheduling the observation visits.

About a month before the start of data collection, Mathematica site coordinators will contact the designated SPPs at each setting to collect more specific information on eligible classrooms affiliated with the center or FCC setting. SPPs will gather classroom child rosters⁷ and update classroom information to assist in completion of a child care setting recruitment form. Site coordinators will collect the names and birthdates of the children from birth to 30 months of age in each of the eligible classrooms, along with the parent’s primary language. We will use this information to determine which age-specific self-administered questionnaire (SAQ) to send the parent for the child and whether to send the materials in English or Spanish. We estimate that the gathering of this information prior to the visit will take 30 minutes per setting.

Table A.1
Q-CCIIT Data Collection Plan

	How Instrument Is Collected	Pilot Test (Early 2012)	Psychometric Field Test (Fall 2012)
Child care setting recruitment form	Paper/Phone	80	300
Q-CCIIT observation	Paper	120	400
Infant classrooms		40	100
Toddler classrooms		40	100
Family child care		40	200
Test-retest		n.a.	60
Caregiver background questionnairea	Paper SAQ	240	800
Validation observation	Paper	n.a.	100
Parent-report child competence questionnaireb	Paper SAQ	560	1,600
Focus groups participantsc	Semistructured discussion	40	n.a.
Parent focus group demographic questionnaire	Paper SAQ	20	n.a.
Caregiver focus group demographic questionnaire	Paper SAQ	10	n.a.
Training and technical assistance provider focus group demographic questionnaire	Paper SAQ	10	n.a.

^a All caregivers in an observed classroom will complete a questionnaire. On average, we expect two caregivers per classroom.

^b Number of parents was based on national organizational standards on group size and child-to-staff ratios and assumes a 60 percent response rate.

^c Focus groups will take place prior to the pilot observation data collection.

n.a. = not applicable; SAQ = self-administered questionnaire.

A.3. Use of Improved Information Technology and Burden Reduction

Overall, the proposed data collection does not lend itself to methods involving information technology. Focus groups consist of semistructured discussions not conducive to information technology such as computerized interviewing. We will use paper-and-pencil SAQs to collect caregiver background characteristics (a 15-minute, one-time questionnaire). The parent-report child competence questionnaire contains standardized instruments designed to be administered with paper and pencil. Access to the internet can be limited in child care settings; given the time available and the anticipated technology resources of respondents, online or web versions are not feasible.

A.4. Efforts to Identify Duplication and Use of Similar Information

From our review of the measures available in the field and expert consultation, we found no evidence of any other assessments that offer information as comprehensive as the Q-CCIIT measure. Since the main goal of this project is measurement development, existing administrative data sets are also not appropriate.

No existing measures or data provide the in-depth assessment of child-caregiver interactions that are the focus of the new Q-CCIIT measure. Existing observational measures of infant and toddler interactions have focused primarily on parent and home settings rather than nonparental child care settings. There is no evidence of any existing measures that fulfill the same role and have the same psychometric validation as the planned Q-CCIIT measure.

A.5. Impact on Small Businesses or Other Small Entities

Small businesses might be affected, since they will likely make up a portion of the FCC and community-based child care centers. Data collection for all settings includes a child care setting recruitment form and for all caregivers includes a background questionnaire (on topics such as education and experience). So that burden on setting staff is minimized, the recruitment form will take no more than 30 minutes and the caregiver instrument will take no more than 15 minutes.

A.6. Consequences of Collecting Information Less Frequently

The data collection includes a pilot test and a psychometric field test. We will recruit different child care settings for the two phases. In each phase, information is collected at one time point, except for parent-report child competence questionnaires in the field test, which have a followup 6 months after the initial data collection period. The data collection cannot occur less frequently, as the two phases support the development and refinement of the Q-CCIIT measure, and the second parent questionnaire provides a measure of growth needed for validation of the Q-CCIIT measure by examining predictive associations with child development.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances requiring deviation from these guidelines.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with the Paperwork Reduction Act of 1995 (P.L. 104-13) and OMB regulations at 5 CFR Part 1320 (FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection. The first notice for the Q-CCIIT data collection was published in the Federal Register, volume 46, no. 76, page 12966 on March 9, 2011. No public comments were received during the 60 days following that announcement.

Members of the Q-CCIIT expert panel have been contacted for advice on various aspects of the study design and the data collection instruments. Their feedback was obtained through in-person meetings, telephone conversations, and written comments. Members of the Q-CCIIT expert panel are listed in Table A.2.

TABLE A.2

Q-CCIIT MeaSUREMENT DEVELOPMENT Expert Panel Membership

Name	Affiliation
Robert Bradley	Arizona State University
Judith Carta	University of Kansas
Martha Edwards	Ackerman Institute for the Family
Karen Heying	Zero to Three
Judith Jerald	Save the Children
Ronald Lally	WestEd
Tammy Mann	United Negro College Fund
Lori Roggman	Utah State University
Susan Sandall	University of Washington
Kathy Thornburg	University of Missouri, Center for Family Policy and Research
Deborah Lowe Vandell	University of California, Irvine

A.9. Explanation of Any Payment or Gift to Respondents

We recognize that participation in the Q-CCIIT project will place some burden on the child care settings, caregivers, and parents. We have attempted to minimize this burden through our data collection procedures and our use of carefully constructed instruments and assessments. Nevertheless, at each phase of the project we should acknowledge the burden that participation entails. Our plan to provide tokens of appreciation is based on those used effectively in previous projects and attempts to acknowledge respondents’ efforts in a respectful way. As described below, ACF has structured the incentives to be provided at each level of data collection—the setting level, the caregiver level, and the family level:

• As a token of appreciation for participation in the focus groups, we will give each participant a $25 gift card.

• For his/her assistance in organizing data collection, we will provide each SPP a $25 gift card to purchase materials to use with the children at the center or FCC home.

• For each classroom observation coupled with completion of the caregiver background questionnaire, we will provide each caregiver with a $25 gift card.

• During the psychometric field test, we will also offer an additional $25 gift card for classrooms selected for a second observation to assess test-retest reliability.

• Finally, parents will receive a $25 gift card for their time completing each parent-report child competence questionnaire.

A.10. Assurance of Privacy Provided to Respondents

This project is being conducted in accordance with all relevant regulations and requirements, including the Privacy Act of 1974 (5USC 552a), the Privacy Act Regulations (34 CFR Part 5b), and the Freedom of Information Act (5 CFR 552) and related regulations (41 CFR Part 1-1, 45 CFR Part 5b, and 40 CFR 44502). All data collectors will be knowledgeable about privacy procedures and prepared to describe them in detail or to answer any related questions raised by respondents.

If classrooms are to be video-recorded prior to the observation, we will obtain written consent from all caregivers and parents of children in the classroom. We have crafted consent forms and advance materials (Appendix E) that explain in simple, direct language the steps we will take to protect the privacy of the information about each participant. During the Q-CCIIT observations, caregivers and children will engage in typical classroom activities. When not video-recording classroom observations, explicit consent will not be sought for the observations; however, all caregivers will receive documents explaining the project, how they were selected to participate, what their participation entails, and how the data will be used. They will also receive information about privacy protections, which will be repeated in the introductory comments of observation visits. For focus groups, Mathematica moderators will read a script that covers the basic tenets of informed consent, including confidentiality, voluntary participation, purpose of the focus group, and intended use of the findings. Staying to participate in the focus group constitutes the respondent’s consent. For self-administered questionnaires, caregivers and parents will read an introduction that explains their right to refuse to participate altogether, and tells them they may choose not to answer any individual question or set of questions if they do participate. Completion of the questionnaire constitutes implicit consent. In all cases, assurances of privacy will be given to each participant as he or she is recruited for the project. Individuals will be assured (1) that their responses will not be shared with the child care setting, (2) that all responses will be reported in aggregate form only, and (3) that the data collected are for research and education purposes only and will never be used for monitoring or accountability purposes. The Q-CCIIT fact sheet makes it clear that participation is voluntary (Appendix E). We are in the process of obtaining IRB clearance to assure privacy procedures and protection of participants’ rights.

As a further guarantee of privacy, identifying information will be maintained in separate tables in the database, which will be linked to the data entry screens only by a sample identification number. Personal identifiers that could be used to link individuals with their responses will be removed from all completed questionnaires and stored under lock and key at the research team offices. Access to the file linking sample identification numbers with identifying information will be limited to a small number of people with a “need to know.” In addition, all Mathematica staff are required to sign a confidentiality statement (see Appendix F).

Access to hard-copy documents will be strictly limited. Completed paper instruments are stored in secure facilities. Mathematica’s data systems and their associated databases are secured behind a firewall between the local area network and any external internet connection; all data stored on Mathematica’s servers are encrypted with a FIPS 140-2 complaint encryption program validated under the Crypotgraphic Module Validation Program, and data are available only to staff associated with the project through password protection and encryption keys.

A.11. Justification for Sensitive Questions

To assess the psychometric properties of the new Q-CCIIT measure in distinguishing levels of quality, we will be asking some sensitive questions as part of the caregiver background questionnaire in the pilot and psychometric field tests, including some aimed at assessing feelings of depression. Caregivers’ mental health can affect their ability to provide supportive and enriching care, which would be assessed in an observation of caregiver-child interactions. The questions are from standardized measures or have been used extensively with nonparental caregivers or teachers in prior studies with no evidence of harm (for example, in the Early Head Start Family and Child Experiences Survey and the Head Start Family and Child Experiences Survey). All respondents will also be assured that their identity will be kept private. Data from these questionnaires will be reported in aggregate. None of the responses will be shared at the individual level, so there will be no risk of disclosure. Each caregiver will receive an envelope to conceal the completed SAQ for privacy.

A.12. Estimates of Annualized Burden Hours and Costs

The proposed data collection does not impose a financial burden on respondents, nor will respondents incur any expense other than the time spent participating. The estimated annual burden for project respondents—caregivers, parents, and T/TA providers—is listed in Table A.3. Response times are derived from previous studies using the same instruments with a similar population. For copyrighted measures, published estimates of the administration times were also used. The total annual burden is expected to be 1,659 hours for all the instruments.

Estimates of annualized costs

To compute the total estimated annual cost, we multiplied total burden hours by the average hourly wage for each adult participant, based on median weekly wages from the Bureau of Labor Statistics, Current Population Survey estimates (fourth quarter of 2010 provided for full-time employees over the age of 25 by education and 2009 annual estimates by occupation). The results are in Table A.3 below. For caregivers, we used the median salary for child care providers ($9.18 per hour). For parents, we used the median salary for full-time employees over the age of 25 with some college or an associate’s degree ($18.18 per hour), as we will sample sites for diversity to include Early Head Start programs, which often serve families with parents who have a high school diploma, as well as universities, which may serve families with parents who have a bachelor’s degree or higher. For T/TA providers, we used the median salary for full-time employees over age 25 with a bachelor’s degree ($26.23 per hour). For the focus groups, respondents will include parents, caregivers, and T/TA providers, given their diverse education and employment backgrounds, we used the median salary for full-time employees over the age of 25 with some college or an associate’s degree ($18.18 per hour) to reflect a mid-point of those backgrounds.

TABLE A.3
ESTIMATED ANNUAL RESPONSE BURDEN AND ANNUAL COST

Instrument	Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours	Average Hourly Wage	Total Annual Cost
Child care setting recruitment form	190	1	0.5	95	$9.18	$871.63
Q-CCIIT measure-group activity and follow-up	290	1	0.25	73	$9.18	$669.78
Caregiver background questionnaire	520	1	0.25	130	$9.18	$1,192.75
Focus group interview guide	20	1	1.90	38	$18.18	$690.65
Parent focus group demographic questionnaire	10	1	0.10	1	$18.18	$18.18
Caregiver focus group demographic questionnaire	5	1	0.10	1	$9.18	$9.18
Training and technical assistance provider focus group demographic questionnaire	5	1	0.10	1	$26.23	$26.23
Parent-report child competence questionnaire	880	2	0.75	1,320	$18.18	$23,991.00
Estimated Annual Total				1,659		$27,469.38

A.13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

Not applicable.

A.14. Annualized Cost to Federal Government

The total cost to the federal government for Q-CCIIT under the terms of the contract to Mathematica is $4,061,923.00. The cost for the data collection elements is $2,574,614.00, or $1,287,307.00 per year. These costs include the sampling, data collection, data processing, and data coding. Respondent tokens of appreciation are also included in the costs.

A.15. Explanations for Program Changes or Adjustments

None; this is a new collection.

A.16. Plans for Tabulation and Publication and Project Time Schedule

Analysis plans

The focus groups and pilot phase of Q-CCIIT (as described in A.2) serve to inform the psychometric field test for the final Q-CCIIT measure. We will conduct conceptual analyses of the responses to the semistructured questions. We will look for participant agreement with how we have ordered the anchors on the Q-CCIIT draft measure and look for patterns in any disagreements (for example, cultural differences, age of child). We will revise the measure based on the results of these analyses.

Quantitative analyses will use the data collected in the pilot test to conduct preliminary psychometric analyses and make any adjustments needed to the measure. We will use the data from the psychometric field test to demonstrate the new measure’s reliability and validity. The analyses will be completed in the following ways:

Reliability estimates. To ensure that the Q-CCIIT observation provides reliable measurement of the quality of caregiver‐child interaction for infants and toddlers, we will calculate both item- and scale-level descriptive statistics, estimates of reliability as measured by the coefficient alpha (internal consistency), and item-total correlations. We will also estimate (1) test-retest reliability by calculating Pearson correlations and (2) inter-observer reliability by calculating Pearson correlation, percent agreement, and Kappa coefficient.

IRT analysis. To examine item functioning within subscales and evaluate the reliability and validity, we will conduct one-parameter IRT analysis (Rasch model) to generate interval-level scores. The reliability estimates, ordering of item difficulties, factor analysis of residuals, and fit statistics produced by Rasch models can provide evidence for evaluating the validity and reliability of the quality measure and identifying optimal rating scale categories. We will conduct this analysis in the pilot test and make any needed revisions before the psychometric field test. With data from the psychometric field test, we will examine the model’s replicability for specified subgroups of interest (for example, infant or toddler classrooms, center or family child care).

Confirmatory factor analysis (CFA). We will perform CFA to test hypotheses corresponding to the theoretical notions of the quality of caregiver‐child interaction for infants and toddlers. We will use Mplus (Muthén and Muthén, 2007) to examine the relationships between observed indicators and primary latent factors and the correlations between latent factors. In addition to CFA for the overall sample of settings, we will perform subgroup analysis to test whether the factor structure of the new measure is the same in specified subgroups such as setting (center-based or FCC) or by age (infant or toddler classrooms).

Convergent validity. We will examine evidence of convergent validity of the Q-CCIIT measure with the validation observation tool and with children’s developmental outcomes. To examine convergent validity with child outcomes, we will use two-level hierarchical linear modeling (Raudenbush & Bryk, 2002) that nests children within classrooms/settings (child characteristics at level 1 and classroom/setting at level 2). First, we will examine the association between the Q-CCIIT measure and child outcomes controlling for dosage (amount of time with the caregiver) and time one scores. Next, we will add child/family characteristics (such as child age, race/ethnicity, maternal education, and dual language learner status) to level 1 of the model to see if the associations change, thereby controlling for possible selection effects.

Time schedule and publications

A sustainability plan (slated for summer of 2013) will detail guidelines on training and materials and processes for producing future editions of the measure and conducting ongoing, appropriate psychometric testing to ensure distribution of the measure for widespread and appropriate use. The psychometric analysis report (planned for fall of 2013) will present analyses and results from the psychometric field test.

A.17. Reason(s) Display of OMB Expiration Date Is Inappropriate

Approval not to display the expiration date for OMB approval is not requested.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this data collection.

REFERENCES

American Academy of Pediatrics, American Public Health Association, and National Resource Center for Health and Safety in Child Care and Early Education. (2002). Caring for Our Children: National Health and Safety Performance Standards: Guidelines for Out-of-Home Child Care Programs, 2nd edition. Elk Grove Village, IL: American Academy of Pediatrics, and Washington, DC: American Public Health Association.

Boller, K., Del Grosso, P., Blair, R., Jolly, Y., Fortson, K., Paulsell, D., Lundquist, E., Hallgren, K., & Kovac, M. (2010). The seeds to success modified field test: Findings from the impact and implementation studies. Princeton, NJ: Mathematica Policy Research.

Carter, A. S., & M. Briggs-Gowan. (2005). ITSEA BITSEA: The Infant-Toddler and Brief Infant Toddler Social Emotional Assessment. San Antonio, TX: PsychCorp.

Fenson, L., S. Pethick, C. Renda, J. L. Cox, P.S. Dale, & J. S. Reznick. (1995). Short-form versions of the MacArthur Communicative Development Inventories. Applied Psycholinguistics, vol. 21, 2000, pp. 95–115.

Muthén, L.K., & B.O. Muthén. (2007). Mplus User’s Guide, 5th ed. Los Angeles: Muthén & Muthén.

National Research Council, Committee on Family and Work Policies. (2003). Working Families and Growing Kids: Caring for Children and Adolescents, Eds. Eugene Smolensky and Jennifer Appleton Gootman. Washington, DC: National Academies Press.

NICHD Early Child Care Research Network. (2002). Child-Care Structure  Process  Outcome: Direct and indirect effects of child-care quality on young children’s development. Psychological Science, vol. 13, no. 3, pp. 199-206.

Phillipsen, L. C., M. R. Burchinal, C. Howes, & D. Cryer. (1997). The prediction of process quality from structural features of child care. Early Childhood Research Quarterly, 12, pp. 281-303.

Raudenbush, S. W., & A. S. Bryk. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods, 2nd ed. Newbury Park, CA: Sage.

Sameroff, A. (2009). The Transactional Model of Development: How Children and Contexts Shape Each Other. Washington, DC: American Psychological Association.

Sameroff, A. J., & M. J. Chandler. (1975). Reproductive risk and the continuum of caretaking casualty. In Review of Child Development Research, vol. 4, Eds., F.D. Horowitz, M. Hetherington, S. Scarr-Salapatek, and G. Sigel. Chicago, IL: University of Chicago Press.

Shonkoff, J. P., & D. A. Phillips. (2000). From Neurons to Neighborhoods. Washington, DC: National Academies Press.

Squires, J., Bricker, D., Twombly, E., Nickel, R., Clifford, J., Murphy, K., Hoselton, R., Potter, L., Mounts, L., & Farrell, J., (2009). Ages & Stages Questionnaires, Third Edition (ASQ-3): A Parent-Completed, Child-Monitoring System. Baltimore: Paul H. Brookes Publishing Co.

Vandell, D. L., & Wolfe, B. (2000). Child care quality: Does it matter and does it need to be improved? Madison, WI: University of Wisconsin-Madison, Institute for Research on Poverty.

Zaslow, M., I. Martinez-Beck, K. Tout, & T. Halle. (2011). Differing purposes for the measurement of quality: Implications for selection of measures, for carrying out the measurement process, and communicating results. In Quality Measurement in Early Childhood Settings, Eds., M. Zaslow, I. Martinez-Beck, K. Tout, and T. Halle. Baltimore, MD: Brookes Publishing.

3 We use the term caregiver throughout this document to signify teachers or providers in center-based and family child care settings.

4 In this document, the term classrooms signifies groupings in center-based and family child care settings.

5 In up to 21 classrooms (7 infant classrooms, 7 toddler classrooms, and 7 family child care), pairs of observers will conduct the observations to video-record sessions. We will use the videos for item development (in creating, refining, and applying items), future training (as exemplars to facilitate discussion), and future certification (in establishing and monitoring inter-rater reliability).

6 Eligible settings must have a classroom with at least two children from birth to 30 months of age at the time of data collection and must have been in operation for at least one year.

7 Because some settings could have concerns about providing personal identifying information, Mathematica site coordinators will be prepared to work with SPPs to use alternate means of creating identifiers, such as recording children’s initials rather than their names on rosters.