Pretesting Data Collection (OMB 0970-0355)
Supporting Statement Part B for OMB Approval
Measurement Development: Quality of Family-Provider Relationships in Early Care and Education
A total of two iterative rounds of cognitive interviews will be conducted with 9 parents who have children attending a Head Start program and 36 Head Start Family Service Workers (FSW) from Head Start programs. Currently, there is no consensus in the literature on sample size requirements for cognitive interviews. However, it is clear that as sample size increases so does the likelihood of detecting new problems in questions (see Blair & Conrad, 2011; Blair, Conrad, Ackermann & Claxton, 2006). Blair and colleagues (2006) found that the mean number of problems found in survey items significantly increased when the sample size was increased from 5 respondents to 50 respondents, but that the rate for identifying unique problems tapered off at a sample size of 20. Based on our experience with diverse samples, saturation (the point in which the addition of interviews yields redundant data and, thus, no new information is learned) is reached at a sample size of at least 12 respondents per round. Given this experience and that we want to stratify the samples on key variables (e.g., FSW tenure, caseload), for the present study we plan to conduct 36 interviews with FSWs. We plan to conduct 9 interviews with parents to supplement data collected during previous cognitive interviews completed under this generic clearance (OMB Control #0970-0355).
For the cognitive interviews, Child Trends will recruit participants, determine participants’ study eligibility, and schedule cognitive interviews with eligible parents and FSWs. Participants will be recruited from different communities and programs in the Washington, D.C., metropolitan area as well as other large metropolitan areas and rural areas in the United States such as Chicago, IL, Detroit, MI, Seattle, WA, and Los Angeles, CA.
Parent Recruitment
Parents will be recruited from Head Start programs, from study fliers posted in the community (Appendix F), from website (e.g., Craigslist) and newspaper ads (Appendix G), and, in DC, from on-the-ground recruitment (handing out brochures and describing the study to potential participants) (Appendix H). The study flier will briefly describe the project and provide study contact information. If potential participants express interest during a recruitment announcement at a program or in the community, they will be informed that they may also indicate their interest by noting their names and phone numbers on individual sign-up sheets (Appendix I) to obtain more information about the study. By using individual sign-up sheets, potential participants will not be able to see the names and phone numbers of the others who have expressed interest.
Sign-up sheets will be transported back to Child Trends’ office in a sealed envelope and placed in a locked file cabinet. Child Trends study staff will then call interested individuals to establish eligibility using screeners (see Part A Appendices A, B) (see below for screening process). Contact sheets for those who are ineligible or not interested will be shredded on a weekly basis. Additional information from the sign-up sheet, such as e-mail addresses, will only be used to contact participants to give them our contact information if we are unable to reach them by phone.
Family Service Worker Recruitment
FSWs will be recruited in several ways including: (1) from website (e.g., Craigslist) and newspaper ads (Appendix G); (2) from study fliers posted in strategic places in the community (Appendix G); and (3) from Head Start programs identified from lists compiled by localities and child care referral programs. Programs that are identified from provider lists will be contacted via phone to ascertain their interest in assisting with the study. The study will be briefly described and programs’ will be asked for permission to recruit FSWs from their program (Appendix J). If permission is given, study fliers (Appendix F) will be posted to inform potential participants about the study and how to contact Child Trends to obtain more information and to establish study eligibility.
Screening and Interview Procedures
A brief screener interview will be conducted with those who call Child Trends or “sign up” using individual sign-up sheets to determine their eligibility for the study. The parent and FSW screeners (Appendices A and B) will first make sure that those who call in are either a Head Start FSW or a family with children in a Head Start program. The parent screener instruments also ask questions about race/ethnicity, education, and income to ensure we are recruiting a diverse sample. Likewise, on the FSW screener instrument, we are including questions such as length of tenure and case load, to ensure we are capturing those traits that may impact the FSW-family relationship.
If the potential participant qualifies for the study, a cognitive interview will be scheduled at a time that is convenient for him or her. The cognitive interviews (Appendix D and E) will be conducted in-person and over-the-phone. Recruited participants will receive an email or letter (via air mail) thanking them for agreeing to be part of the study along with the date, time and location of the interview (Appendices K and L). We will also include directions and a map of the area for in-person interviews. Recruited participants will receive a reminder call and/or text the day before the cognitive interview (Appendix C). These procedures have been found to minimize the number of cancellations and no-shows.
Each cognitive interview will be approximately 2 hours long; the screener interviews will average 5 minutes for ineligible participants and 15 minutes for eligible participants. Reminder calls for eligible, scheduled participants will average 3 minutes (see Supporting Statement Part A). In person, cognitive interviews will be conducted at Child Trends when possible or at meeting rooms in community centers; interviews will also be conducted via the telephone.
To maximize response rates, Child Trends project staff will conduct cognitive interviews during hours most convenient for parents and early care and education providers (e.g., evenings, weekends). Additionally, we will conduct phone interviews when possible in order to decrease travel burden.
The proposed procedures for the cognitive interviews have been successfully applied in other similar studies conducted by Child Trends (including this study), and there are no plans to test the procedures.
The cognitive interview team is led by Drs. Lina Guzman and Nikki Forry, co-principal investigators; Dr. Toni Porter, senior substantive consultant; Dr. Manica Ramos, project manager; and Eliza Brown, research assistant. Additional staff consulted at Westat (prime contractor) include Dr. Christine Nord, project director.
References
Blair, J., Conrad, F. (2011). Sample Size for Cognitive Interview Pretesting. Public Opinion
Quarterly, 75 (4), 636-658.
Blair, J., Conrad, F., Ackerman, A.C. & Claxton, G. (May, 2006). The Effect of Sample Size on
Cognitive Interview Findings. Proceedings of the American Statistical Association, Section on Survey Research Methods. Alexandria, VA: American Statistical Association.
Appendices
Appendix A: Cognitive Interview Instruments for Ineligible Parents
Appendix B: Cognitive Interview Instruments for Ineligible Family Service Workers
Appendix C: FPRQ Cognitive Interviews - Email and Text Message Reminder of Interview
Appendix D: Cognitive Interview Instruments for Eligible Parents
Appendix E: Cognitive Interview Instruments for Eligible Family Service Workers
Appendix F: FPRQ Cognitive Interviews – Study Fliers
Appendix G: FPRQ Cognitive Interviews – Website and Newspaper Advertisements
Appendix H: FPRQ Study Brochure
Appendix I: FPRQ Cognitive Interviews – Individual Sign-Up Sheet
Appendix J: FPRQ Cognitive Interviews – Script for Program Recruitment
Appendix K: FPRQ Cognitive Interviews – Thank you letter for telephone interviews
Appendix L: FPRQ Cognitive Interviews – Thank you letter for in-person interviews
B-
| File Type | application/msword |
| Author | jwest |
| Last Modified By | Molly |
| File Modified | 2013-10-25 |
| File Created | 2013-10-24 |