Measurement Development: Healthy Relationship and Marriage Education Models and Measures
OMB Information Collection Request
0970-0355
Supporting Statement
Part B
August 2014
Submitted By:
Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
7th Floor, West Aerospace Building
370 L’Enfant Promenade, SW
Washington, D.C. 20447
Project Officer: Lauren Supplee
B.1. Respondent Universe and Sampling Methods 2
B.2. Procedures for Collection of Information 2
B.3. Methods to Maximize Response Rates and Deal With Nonresponse 4
B.4. Test of Procedures or Methods to be Undertaken 4
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Attachments
Attachment A: HMRE Cognitive Interview Instrument for Parents in Complex Families
Attachment B: HMRE Cognitive Interview Guide for Complex Families
Attachment C: HMRE Cognitive Interview Instrument for High School Age Youth
Attachment D: HMRE Cognitive Interview Guide for High School Age Youth
Attachment E: HMRE E-mail and Text Reminder
Attachment F: HMRE Consent Form for Complex Families
Attachment G: HMRE Consent for Minor Participation Form for Parents of High School Age Youth
Attachment H: HMRE Assent Form for High School Age Youth
Attachment I: HMRE Recruitment Advertisement for Complex Families
Attachment J: HMRE Recruitment Advertisement for High School Age Youth
Attachment K: HMRE Recruitment Flyer for Complex Families
Attachment L: HMRE Recruitment Flyer for High School Age Youth
Attachment M: HMRE Pledge of Confidentiality
Attachment N: HMRE Individual Sign-Up Sheet
Attachment O: HMRE Cognitive Interview Screener for Complex Families
Attachment P: HMRE Cognitive Interview Screener for High School Age Youth
Attachment Q: HMRE Script for Study Recruitment for Complex Families
Attachment R: HMRE Script for Study Recruitment for High School Age Youth
Attachment S: HMRE Participant Eligibility Matrix for Complex Families
Attachment T: HMRE Participant Eligibility Matrix for High School Age Youth
Attachment U: HMRE Introduction to Study Letter for Complex Families
Attachment V: HMRE Introduction to Study Letter for High School Age Youth
Attachment W: HMRE Thank-you Letter for Complex Families
Attachment X: HMRE Thank-you Letter for High School Age Youth
A total of three iterative rounds of cognitive interviews will be conducted with up to 27 high school age youth and 27 parents in complex families. Currently, there is no consensus in the literature on sample size requirements for cognitive interviews. It is clear that as sample size increases, so does the likelihood of detecting problems in questions (Blair, 2011, 2006). Blair and colleagues (2006) found that the mean number of problems found in survey items significantly increased when the sample size was increased from 5 respondents to 50 respondents, but that the rate for identifying unique problems tapered off at a sample size of 20. From our experience with diverse samples, saturation (the point in which the addition of interviews yields redundant data and thus, no new information is learned) is reached at a sample size of around12 respondents per round. Although we may not reach complete saturation per round, conducting multiple rounds of interviews will yield more complete results. Given our experience with cognitive testing and because we want to stratify the samples on key variables (e.g., age and relationship status for the high school age group), for the present study we plan to conduct up to 54 interviews with 27 adolescents and 27 individuals in complex families.
The respondent universe includes adolescents ages 14-17 and parents of children under the age of five in committed relationships in which one partner has a child from a previous relationship in the United States. Individuals in the respondent universe are those who hear about our study through on-the-ground recruitment and individuals who have access to the internet (or who hear about our study through word of mouth from someone with access to the internet) and a phone connection.
We will track adolescent participant characteristics by geographic region, sex, whether in a current relationship, race/ethnicity and age. We will track complex families’ characteristics such as geographic region, sex, race/ethnicity, age, and whether the participant has children with previous partners. Our sample size is not big enough to stratify by all of these characteristics, but we will track these categories to make sure we get input from a broad range of participants. We may modify our eligibility criteria in later rounds of the cognitive interviews based on the characteristics of participants in earlier rounds in order to get the most diverse sample possible. For example, if we recruit mostly 17 year old teens that have never been in a relationship in the first round of interviews, we will prioritize younger teens and teens in relationships in later rounds. Tracking these types of characteristics will allow the study team to understand the context of our findings as well as diversify our sample.
Child Trends will recruit participants, determine participants’ study eligibility and schedule a cognitive interview with eligible high school age adolescents and parents in complex families. Participants will be recruited from different communities in the Washington, D.C., metropolitan area as well as other large metropolitan areas and rural areas in the United States. The same procedures will be used for all three rounds of interviews.
Respondents will be recruited in several ways including: (1) from websites (e.g., Craigslist) (Attachments I and J) and (2) from study fliers posted in strategic places in the local Washington DC community (Attachments K and L). If potential participants express interest during a recruitment announcement they will be informed that they may participate on the spot or indicate their interest by noting their names and phone numbers on individual sign-up sheets (Attachment N) to obtain more information about the study. By using individual sign-up sheets, potential participants will not be able to see the names and phone numbers of the others who have expressed interest.
Child Trends study staff will call interested individuals to establish eligibility using screeners (see Attachments O and P). Child Trends staff will be trained on a standardized recruitment script to use when calling back interested participants and when recruiting participants on-the-ground (see Attachments Q and R).The complex family screener (see Attachment O) will identify low-income individuals in committed relationships who are parenting a child under the age of five together. The respondent participating in the interview does not have to have a child with a previous partner, but at least one of the individuals in the relationship will be required to also have a child from a previous relationship. We hope to interview low-income individuals below at least 200%, of the Federal Poverty Level, which will be determined in accordance with family size and self-reported household income. We will aim to find participants who meet all of the characteristics of our definition of a complex family but if recruitment is not successful we will adjust some of the specifications of the sample as needed. We will use a screening matrix to track demographic qualities of this sample (Attachment S). The adolescent screener (see Attachment P) will identify young people ages 14-17. Teens who are in relationships or not in relationships will be eligible. We will use screening matrices to track these and other demographic characteristics of the sample (see Attachment T).
If the potential participant qualifies for the study based on the screener questions, a cognitive interview will be scheduled at a time that is convenient for him or her. The majority of the cognitive interviews (Attachments A-D) will be conducted via telephone. However, when we are recruiting participants on-the-ground locally in Washington, DC, we may conduct in-person interviews if it is easiest for the participant to be screened and interviewed on the spot. This option will only be available to adolescents if a parent is available to provide consent. Recruited participants will receive an email (or fax or ground letter if email is not possible) thanking them for agreeing to be part of the study along with the date and time of the interview (Attachments U and V). Parents of adolescents will also receive a letter introducing them to the study (Attachment V), a consent form (Attachment G), and a copy of the child’s assent form (Attachment H). Parents will have to provide consent before the time of their child’s interview. Recruited participants will also receive a reminder call and/or text the day before the cognitive interview (Attachment E). These procedures have been found to minimize the number of cancellations and no-shows. Participants will be asked to provide consent or assent at the time of their scheduled interview (Attachment F and H). Participants will receive thank-you letters via e-mail or post (depending on which method of communication they have previously indicated that they prefer) after completing their interview (Attachments W and X).
Each cognitive interview will be approximately 1.5 hours long; all other screening activities that will take place prior to the cognitive interview will average 0.5 hours. Interviews will be conducted via telephone or in-person if convenient for the participant at the time that they are recruited.
We expect that the maximum of up to 54 participants can be successfully recruited based on experience with similar studies in the past. Based on prior experience, we anticipate that we will need to contact 135 high school age youth (5x more than the maximum amount) and 108 parents in complex families (4x more than the maximum amount) to achieve the maximum of nine of each of these groups per three round of interviews.
If an eligible participant declines to participate, the interviewer will discuss the individual’s concerns and attempt to address the concerns. If the participant cannot or will not participate in the research, we will select another individual with similar characteristics. Nonresponse bias may occur if the participants provide responses that may differ from teens and parents in complex families that don’t participate. For example, given that many teen relationships are short-term, we may not be as successful in recruiting teens that have been in relationships for longer than three months. Parents in lower quality relationships may not be interested in participating in this study. The responses of these two subgroups within our target populations may differ from the teens and parents in complex families that we do recruit. However, our screening and tracking procedures should help to alleviate some of this potential bias by carefully tracking the characteristics of the participants so that we may adjust our enrollment decisions to ensure a more balanced sample across a number of specific characteristics.
To maximize response rates, Child Trends project staff will conduct cognitive interviews during hours most convenient for adolescents and individuals in complex families. We will conduct phone interviews in order to decrease travel burden, unless in-person interviews can be conducted at the time of recruitment for local participants. The day before the scheduled interview, we will call and/or text to remind the participant of the interview. To further ensure a high rate of participation, we will offer participants $50 as a token of appreciation (see Supporting Statement A, section A.9, for additional information).
There was no pre-testing conducted for this study, but the proposed procedures have been successfully applied in other similar studies conducted by Child Trends, and there are no plans to test the procedures.
The cognitive interview team is led by Dr. Scott, principal investigator, Shelby Hickman, qualitative analyst; and Eliza Brown, research assistant.
References
Blair, J., Conrad, F. (2011). Sample size for cognitive interview pretesting. Public Opinion Quarterly, 75(4), 636-658.
Blair, J., Conrad, F., Ackermann, A., Claxton, G. (2006). The effect of sample size of cognitive interview findings. Paper presented at the American Statistical Association.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | jwest |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |