FACES_OMB_Part B_11-13-13

Redesign of the Head Start Family and Child Experiences Survey (FACES 2012)

Supporting Statement Part B: Statistical Methods (Used for Collection of Information Employing Statistical Methods)

November 13, 2013

CONTENTS

B. STATISTICAL METHODS (USED FOR COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS) 1

B1. Respondent Universe and Sampling Methods 1

B2. Procedures for Collection of Information 1

1. Child Assessments 2

2. Parent Survey and Cognitive Interviews 2

B3. Methods to Maximize Response Rates and Deal with Nonresponse 2

1. Expected Response Rates 2

2. Dealing with Nonresponse 3

3. Maximizing Response Rates 3

B4. Tests of Procedures or Methods to Be Undertaken 3

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 3

REFERENCES 5

ATTACHMENTS

b.1 Program Recruitment aDVANCE LETTER

b.2 pARENT LETTER AND CONSENT FORM

b.3 eMAIL INVITATION

b.4 cati ADVANCE LETTER

b.5 SAMPLE SCREENS

B. STATISTICAL METHODS (USED FOR COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS)

B1. Respondent Universe and Sampling Methods

The sample for the FACES Pilot Study will include a purposive sample of Head Start programs, centers, children and their parents.

We will draw the program sample from a group of Head Start programs with which Mathematica has existing relationships, due to their participation in prior cohorts of FACES. From that universe, we will select eight programs located in three geographical areas (New York/New Jersey, southern Texas, and southern California). These areas were chosen because of the need to have programs with high concentrations of Spanish-speaking children and minimize travel to the extent possible. Mathematica already has experienced bilingual assessors in these areas. Our goal is to have approximately five programs with high concentrations of dual language learner (DLL) children (DLL enrollment of 50 percent or higher) and three programs with smaller concentrations (DLL enrollment of less than 50 percent).

Next, we will select two centers from each program, again being mindful of the distance between centers, to control travel costs while also looking for centers that serve a higher concentration of DLL children (DLL enrollment of 50 percent or higher), for a total of 16 Head Start centers. In each of the centers, we will select two classrooms, for a total of 32 Head Start classrooms. We will invite all children in each of the selected classrooms to participate in the assessment data collection. Assuming that classrooms will have 15 children on average, we expect to sample a total of 480 children, with 450 participating in the study. The parents of all 450 consented children will be invited to complete the parent survey (Survey of Well-Being of Young Children: SWYC). Our goal is to have 450 completed parent surveys.

B2. Procedures for Collection of Information

Recruitment and data collection activities will commence upon OMB approval and are expected to last approximately three months. We will begin by sending program directors in selected programs a Program Recruitment Advance Letter (Attachment B.1). We then will contact programs by phone, using the Program Recruitment Script (Attachment A.1). We will work with the participating programs to identify and recruit a sample of children and their parents. As part of initial conversations with programs, we will query directors about the characteristics of families served by each program, including the percentage of children and families who are from Spanish-speaking households and the ages of children served. We will use this information, captured with the Program Recruitment Screener (Attachment A.1), to identify the range of children and families available to participate at each program and update the information we have from the most recent Program Information Report (PIR). Once programs have been successfully recruited, we will gather information about the centers in each program and select two per program, following the criteria listed in B.1. Once centers are selected and programs designate an On-Site Coordinator (OSC) who will serve as the key contact and study liaison for coordinating the collection of consent forms and scheduling the assessment data collection visits, recruiters will work with the OSC to obtain information about the classrooms in each center, including the total number of children; number of 3-, 4-, and 5-year-old children; and percentage of DLL children (excluding children who are not speakers of English or Spanish). We will then randomly select two or three classrooms and distribute the Parent Letter and Consent Form (Attachment B.2) to all children in the selected classrooms. OSCs will keep track of outstanding consent forms and use prepaid Federal Express materials to return completed forms to Mathematica’s Survey Operations Center for receipt.

1. Child Assessments

In conjunction with the OSC, Mathematica’s recruiters will schedule the child assessments at the Head Start center. A team of three to four assessors will visit each program for four days. When the team arrives on site, assessors will be taken by the OSC to a prearranged assessment area. The assessors will arrange their space to remove potential distractions to the extent possible and establish a comfortable environment for conducting the assessment. They will set up the assessment materials and log into the child assessment program on laptops they carry with them. The OSC will introduce the assessor to the teacher whose children will be assessed. Assessors then will escort the selected children to the assessment areas one by one and conduct each 60-minute assessment. Each child then will be returned to the classroom and the next selected child assessed.

2. Parent Survey and Cognitive Interviews

We will randomly assign parents of half of the Spanish-speaking and half of the English-speaking children to complete the SWYC via the web and the other half to complete it via Computer-Assisted Telephone Interview (CATI). The survey will be available in both Spanish and English, but we will first offer it in the parent’s preferred language, as identified on the consent form. We will conduct 20 cognitive interviews by telephone with parents who complete the SWYC via the web and 20 who complete it via CATI. Cognitive interviews will take place within two weeks of survey completion. We will conduct approximately half of the cognitive interviews with Spanish-speaking parents and use the interviews to identify any problems parents experienced in completing the survey (for example, difficulty with language, response format). The interviews will afford us an opportunity to compare the feasibility of administering the SWYC via the web and CATI, and help us determine which mode is more feasible for use with low-income parents. The interviews also will help us ensure that the final format is easy to follow and comprehensible to parents of English- and Spanish-speaking children, and that the instrument can be completed within 10 minutes.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

1. Expected Response Rates

Mathematica has an established, successful record of gaining program cooperation and obtaining high response rates from staff and families in research studies of Head Start and other preschool programs. Based on our experience in working with the programs in the past, we do not anticipate any problems in contacting and gathering information from program directors and OSCs. Mathematica staff will call program directors and OSCs during business hours and conduct calls at times determined by the schedules of directors and coordinators.

In addition, building on our experience and existing relationships with programs as a basis for recruitment will serve to minimize nonresponse by allowing us to gain access smoothly, ease recruitment of families, and make arrangements for data collection. The assessment team will be prepared to speak to parents and obtain consent for any children who have not yet returned consent forms at the start of the assessment visit. Based on our experience in prior similar studies, we expect to achieve an assessment response rate of 95 percent and a parent response rate of 85 percent.

2. Dealing with Nonresponse

Given that we have modeled our approach to the recruitment of programs, centers, parents, and their children on those successfully used on previous rounds of the FACES study, we do not anticipate issues associated with nonresponse. However, should our sample sizes fall short of what is needed we will identify and recruit replacements.

3. Maximizing Response Rates

For the assessments, we will present children with a book that costs under $5. For the SWYC questionnaire, we will offer parents a $15 gift card as a token of appreciation to improve cooperation. We then will offer cognitive interviewing participants a $20 gift card for participating in that activity. Additionally, we will offer each program a $200 gift card to thank them for their participation.

B4. Tests of Procedures or Methods to Be Undertaken

All four of the assessments are standardized tests, and two have been used with prior cohorts of FACES children (PPVT-4 and preLAS 2000). Prior versions of the PLS-5 and the ROWPVT-4 have been used on previous Mathematica studies [the PLS-4 has been used on Baby FACES, and the ROWPVT has been used on the Universal Preschool Child Outcomes Study (UPCOS)]. The SWYC is a surveillance instrument developed to provide a screening instrument that is free of cost and easily accessed by parents as well as professionals involved in child care and education. Unlike prior screeners that assessed development in children under age 5, the SWYC is a comprehensive screener that integrates behavior and development, and assesses cognitive, language, motor, and social-emotional development as well as family risk factors. It is currently in the process of being validated and, as of 2013, two of its three components (Developmental Milestones and Preschool Pediatric Symptoms Checklist) that we will use for this study have been statistically compared to validated screeners (Sheldrick et al. 2012a, 2012b) and parents’ reports of developmental-behavioral diagnoses. Items in the third component (Family Questions) were assembled from previously validated tests. The format of the questions on the SWYC is similar to that of many other parent surveys of young children, and it has been used successfully as a screener in well-child visits. However, it has never been used in an electronic format to interview parents. The goal of this study is to compare web and phone administration of the SWYC to interview parents and assess the feasibility of its use on FACES.

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Maria Woolverton, Federal Contracting Officer’s Representative (COR), and Dr. Jerry West, project director, lead the team. Additional staff contributed to the design of the FACES Pilot Study, including Dr. Emily Moiduddin, Dr. Nikki Aikens, and Dr. Sally Atkins-Burnett. In addition to these staff, Ms. Cassandra Meagher, Dr. Harshini Shah, and Ms. Sara Skidmore will lead the data collection efforts. A team of 10 trained assessors will administer the child assessments; 10 telephone interviewers will conduct SWYC with parents over the telephone; and a team of 4 Mathematica project staff will conduct the cognitive interviews with parents.

Inquiries regarding the statistical aspects of the study’s planned analysis should be directed to:

Dr. Jerry West, Project Director

REFERENCES

Brownell, R. (2012). Receptive One-Word Picture Vocabulary Test (Spanish-Bilingual edition, manual). Novato, CA: Academic Therapy Publications.

Duncan, Sharon, and Edward DeAvila. “Pre-Language Assessment Scales [PreLAS2000].” Monterey, CA: CTB-McGraw Hill, 2002.

Dunn, Lloyd, and Doug Dunn. Peabody Picture Vocabulary Test. Fourth Edition. Circle Pines, MS: American Guidance Service, 2006.

Dunn, L.M., E.R. Padilla, D.E. Lugo, and L.M. Dunn. Test de Vocabulario en Imagenes Peabody. Circle Pines, MN: American Guidance Service, 1986.

Hernandez, D. “Young Hispanic Children in the U.S.: A Demographic Portrait Based on Census 2000.” Tempe, AZ: Arizona State University, June 26, 2006.

Hulsey, L. K., Aikens, N., Kopack, A., West, J., Moiduddin, E., and Tarullo, L. (2011). Head Start Children, Families, and Programs: Present and Past Data from FACES. OPRE Report 2011-33a. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Martin N, A., Brownell R (2011) ROWPVT-4: Receptive One-Word Picture Vocabulary Test – Fourth Edition. Novato, CA: Academic Therapy Publications.

Sheldrick, R.C., B.S. Henson, S. Merchant, E.N. Neger, J.M. Murphy, and E.C. Perrin. “The Preschool Pediatric Symptom Checklist (PPSC): Development and Initial Validation of a New Social-Emotional Screening Instrument.” Academic Pediatrics, vol. 12, no. 5, September-October 2012a, pp. 456−67. PMID: 22921494.

Sheldrick, R.C., E. Neger, and E.C Perrin. “Concerns about Development, Behavior and Learning Among Parents Seeking Pediatric Care.” Journal of Developmental and Behavioral Pediatrics, vol. 33, no. 2, February 2012b, pp. 156−160.

West, J., and L. Hulsey. “Who is Served by Head Start? Changes in the 3-Year-Old Population.” Presentation at the Society for Research in Child Development Biennial Meeting, Denver, April 3, 2009.

Zimmerman, Irla Lee, Violette G. Steiner, and Roberta Evatt Pond. Preschool Language Scale, Fifth Edition (PLS-5). San Antonio, TX: Pearson Assessments, 2011