Pretesting Data Collection (OMB 0970-0355)
Supporting Statement Part B for OMB Approval
Measurement Development: Quality of Family-Provider Relationships in Early Care and Education
A total of three iterative rounds of cognitive interviews will be conducted with 102 English- and Spanish-speaking early care and education providers, center directors, and parents of children ages birth through five years. Currently, there is no consensus in the literature on sample size requirements for cognitive interviews. It is clear that as sample size increases so does the likelihood of detecting problems in questions (see Blair & Conrad, 2011; Blair, Conrad, Ackermann & Claxton, 2006). Blair and colleagues (2006) found that the mean number of problems found in survey items significantly increased when the sample size was increased from 5 respondents to 50 respondents, but that the rate for identifying unique problems tapered off at a sample size of 20. From our experience with diverse samples, saturation (the point in which the addition of interviews yields redundant data and thus, no new information is learned) is reached at a sample size of at least 12 respondents per round. Given our experience with cognitive testing and that we want to stratify the samples on key variables (e.g., teacher assistants/aids for the provider group), for the present study we plan to conduct 36 interviews with parents and 55 with early care and education (ECE) providers. We plan to conduct only 11 interviews with ECE center directors because we are not stratifying them on key variables and they will only be given a small portion of the survey.
Child Trends will recruit participants, determine participants’ study eligibility and schedule a cognitive interview with eligible parents and childcare providers. Participants will be recruited from different communities, programs and care facilities in the Washington, D.C., metropolitan area as well as other large metropolitan areas and rural areas in the United States.
Early care and education providers will be recruited in several ways including: (1) from website (e.g., Craigslist) and newspaper ads (Appendix H); (2) from study fliers posted in strategic places in the community (Appendix I); and (3) from centers, programs, and home-based providers identified from lists compiled by localities and child care referral programs. Programs that are identified from provider lists will be contacted via phone to ascertain their interest in assisting with the study. The study will be briefly described and programs’ will be asked for permission to recruit parents and providers from their center/program (Appendix J). If permission is given, study fliers (Appendix I) will be posted and announcements will be conducted to inform potential participants about the study and how to contact the study center to obtain more information about the study as well as to establish study eligibility.
Parents will be recruited from community-based centers, early education and Head Start programs, from study fliers posted in the community (Appendix I), from website (e.g., Craigslist) and newspaper ads (Appendix H), and, in DC, from on-the-ground recruitment (handing out brochures and describing the study to potential participants) (Appendix N). If potential participants express interest during a recruitment announcement at a center/program or in the community, they will be informed that they may also indicate their interest by noting their names and phone numbers on individual sign-up sheets (Appendix K) to obtain more information about the study. By using individual sign-up sheets, potential participants will not be able to see the names and phone numbers of the others who have expressed interest.
Sign-up sheets will be transported back to Child Trends offices in a sealed envelope and placed in a locked file cabinet. Child Trends study staff will then call interested individuals to establish eligibility using screeners (see Part A Appendices A, C and E) (see below for screening process). Contact sheets for those who are ineligible or not interested will be shredded on a weekly basis. Additional information from the sign-up sheet, such as e-mail addresses will only be used to contact participants to give them our contact information, if we are unable to reach them by phone. Child Trends has successfully used sign-up sheets in past recruitment efforts of hard-to-reach populations, such as Spanish speaking participants.
A brief screener interview will be conducted with those who express an interest in participating in the cognitive interviews to determine their eligibility for the study. The parent screener (see Part A Appendix A) will identify those who have at least one child aged birth to five years old in a non-parental early care and education arrangement. The early care and education (ECE) provider screener (see Part A Appendices C and E) will identify those who provide early care and education services in a home-based, center-based, Head Start, or pre-K program to children ages birth to five years. Additionally, other questions are included in the ECE screener instrument, such as job title and length of tenure, in order to ensure that center directors, teachers, aides and assistants are included as well as those with differing levels of experience. Questions about the level and frequency of communication between parents and providers are also included to ensure that the study includes parents and providers with varying levels of engagement.
If the potential participant qualifies for the study, a cognitive interview will be scheduled at a time that is convenient for him or her. The cognitive interviews (see Part A Appendices B, D, F, O, P, Q, and R) will be conducted in-person and over-the-phone. Recruited participants will receive an email or letter (via air mail) thanking them for agreeing to be part of the study along with the date, time and location of the interview (Appendices L and M). We will also include directions and a map of the area for in-person interviews. Recruited participants will also receive a reminder call and/or text the day before the cognitive interview (see Part A Appendix G). These procedures have been found to minimize the number of cancellations and no-shows.
Each cognitive interview will be approximately 2 hours long; the screener interviews will average 5 minutes for ineligible participants and 12 minutes for eligible participants. Reminder calls for eligible, scheduled participants will average 3 minutes (see Supporting Statement Part A). In person, cognitive interviews will be conducted at Child Trends when possible or at meeting rooms in community centers; interviews will also be conducted via the telephone.
To maximize response rates, Child Trends project staff will conduct cognitive interviews during hours most convenient for parents, early care and education providers, and center directors (e.g., evenings, weekends). Additionally, we will conduct phone interviews when possible in order to decrease travel burden.
The proposed procedures have been successfully applied in other similar studies conducted by Child Trends, and there are no plans to test the procedures.
The cognitive interview team is led by Drs. Lina Guzman and Nikki Forry, co-principal investigators; Drs. Toni Porter and Juliet Bromer, senior substantive consultants; Dr. Selma Caal, research scientist; Manica Ramos, qualitative analyst; and Katherine Rodrigues, research assistant, Kristen Peterson, research assistant, and Ali Chrisler, project manager. Additional staff consulted at Westat (prime contractor) include Dr. Christine Nord, project director.
References
Blair, J., Conrad, F. (2011). Sample Size for Cognitive Interview Pretesting. Public Opinion
Quarterly, 75 (4), 636-658.
Blair, J., Conrad, F., Ackerman, A.C. & Claxton, G. (May, 2006). The Effect of Sample Size on
Cognitive Interview Findings. Poceedings of the American Statistical Association, Section on Survey Research Methods. Alexandria, VA: American Statistical Association.
Appendices
Appendix H: FPRQ Cognitive Interviews – Website and Newpaper Advertisements
Appendix I: FPRQ Cognitive Interviews – Study Fliers
Appendix J: FPRQ Cognitive Interviews – Script for Program Recruitment
Appendix K: FPRQ Cognitive Interviews – Individual Sign-Up Sheet
Appendix L: FPRQ Cognitive Interviews – Thank you letter for telephone interviews
Appendix M: FPRQ Cognitive Interviews – Thank you letter for in-person interviews
Appendix N: FPRQ Study Brochure
B-
| File Type | application/msword |
| Author | jwest |
| Last Modified By | DHHS |
| File Modified | 2013-05-16 |
| File Created | 2012-03-22 |