Attachment B - Pretesting Generic Usage Report

Attachment b: use of pretesting generic CLEARANCE

(0970-0355) From 2012-2015

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

7^th Floor, West Aerospace Building

370 L’Enfant Promenade, SW

Washington, D.C. 20447

In January 2012, OMB approved ACF’s request to renew the generic information collection for pre-testing of evaluation surveys (0970-0355). This report describes the use of the generic IC over the three years of approval, including the number of hours used, as well as the nature and results of the activities completed under this generic clearance.

The renewal of the generic IC was approved for three years, during which time ACF requested nine generic ICs for pretesting. The use of the pre-testing generic IC has been beneficial to the development of instruments for these research studies. Over the three years, ACF was approved to collect information from 3,395 respondents, requiring 1,742 hours of burden. During the previous three years of clearance, information was collected from 3,605 respondents requiring 1,336 hours of burden. The significant continued use of the generic from 2012-2015 is indicative of the usefulness of pretesting our surveys and procedures.

Date	Project	Respondents	Burden Hours
3/30/12	Quality of Family-Provider Relationships in Early Care and Education	162	235
10/10/12	Permanency Innovations Initiative - LA RISE	500	300
1/30/13	Permanency Innovations Initiative - AZ, CAPP, IL	954	521
Total Year 1		1616	1056
3/29/13	Head Start Family Voices Pilot Study	380	341
11/14/13	Measurement Development: Quality of Family-Provider Relationships in Early Care and Education	45	37
12/27/13	Redesign of the Head Start Family and Child Experiences Survey (FACES 2012)	958	185
01/09/14	Q-DOT Pilot Study	60	20
Total Year 2		1443	583
9/25/14	Measurement Development: Healthy Marriage and Relationship Education Models and Measures	36	38
11/20/14	Measurement Development: Quality Relationship of Family and Family Services Staff in Head Start/Early Head Start	300	65
Total Year 3		336	103
Total over 3 Years		3395	1742

Example uses of the Pre-testing Generic Clearance

Family-Teacher/Provider Relationship Quality measure development (FTPRQ) project

The Family-Teacher/Provider Relationship Quality measure development (FTPRQ) project created a set of new measures to assess the quality of the relationship between families of young children (ages 0-5 years) and the child’s teacher or child care provider. These relationships are an important component of family engagement in early childhood programs, but no appropriate measure existed.

Both the formative generic clearance (0970-0356) and the pre-testing generic clearance (0970-0355) were integral in developing these measures. Under the formative generic clearance, ACF/OPRE conducted focus groups with families and teachers/providers. These focus groups were essential for ensuring that our measures were based on a realistic conceptual model and that we were using terminology and language that families and teachers/providers would understand. After developing initial drafts of measures, which were based in part on the results of the focus groups as well as an extensive review of the literature and pre-existing related measures, ACF received permission conduct cognitive interviews under the pre-testing generic clearance. Three rounds of cognitive interviews were conducted, which further enabled us to test and refine our measures, so that they were ready for testing in pilot and field tests (conducted under OMB clearance 0970-0420). The thoroughly tested and final measures were released in November 2014 and have been very well-received by the field.

Head Start Family and Child Experiences Survey

An expert panel convened as part of the Head Start Family and Child Experiences Survey (FACES) Redesign recommended that ACF consider optimal approaches to assessing DLL children in future rounds of FACES with the goal of expanding and improving measurement of dual language learner (DLL) children’s development. The panel recommended that ACF pilot additional measures suitable for assessing the vocabulary skills of DLL children more efficiently as well as measures that offer a broader picture of children’s language development.

To accomplish these goals, ACF submitted a generic Information Collection Request (ICR) to pretest measures and procedures that would (1) be reliable and valid for assessing DLL children and (2) reduce the repetitive vocabulary assessment burden on DLL children in the study. Findings from the pretesting were used to inform the measurement battery for the FACES 2014-2018 child assessment (0970-0151).