0990-TPP CDC OMB Comment Responses 1-30_12

Responses to OMB Comments on “Outcome Evaluation of ‘Teenage Pregnancy Prevention: Integrating Services, Programs, and Strategies through Community-Wide Initiatives”

This document summarizes the CDC’s response to OMB concerns and questions about “Outcome Evaluation of ‘Teenage Pregnancy Prevention: Integrating Services, Programs, and Strategies through Community-Wide Initiatives.” In addition, since the original submission, the number of participating sites has been set at four (two target and two comparison). Therefore the overall burden estimate has been reduced accordingly from “up to three target and three comparison” communities.

Part A:

1. Comment: Consider describing respondents in the consent forms and questionnaire cover pages as “young adult,” rather than “child.” This may resonate better with the audience.

Response: Consent forms have been updated to refer to the respondent as ‘teen’ or ‘teenager’. Please see Attachment D – Parental Consent. Questionnaire cover pages have been updated to refer to “respondent” rather than “child.” These forms are programmed into the ACASI survey and will only be seen by the interviewers.

2. Comment: Additionally, consider describing the purpose and nature of the study somewhat differently to respondents and their parents. A study described as regarding teenage pregnancy may not resonate with youth who are not sexually active, or do not have heterosexual sex, and even, potentially, males.  Nonetheless, responses from these groups would be important for the purposes of the study as a whole. Parents also may be more reluctant to consent if the study purposes are presented as focusing on teenage pregnancy. Consider describing the study as teenage health, dating behavior, and sex (or, something like that).

Response: The consents have been updated to refer to “teen health and behavior. Questions will ask about feelings and behaviors including sex and drug use. Some questions will ask about family and friends” (see Attachment D – Parental Consent and Attachment E-Young Adult Consent). The youth assent has been updated to refer to ‘teen behaviors’ (see Attachment E – Youth Assent).

3. Comment: Please ensure that questionnaires and consent form language is consistent with a fourth grade reading ability.  (We believe Flesch-Kincaid is available on Word for this purpose.)

Response: The youth assent has been edited and is now at a Flesch-Kincaid reading level of 4.9 (see Attachment E – Youth Assent). The young adult and parent consents are now at a Flesch-Kincaid reading level of 6.5 (see Attachment F – Youth Consent and see Attachment D – Parental Consent). Furthermore, as an additional measure to address any concerns with reading levels, the youth assent/young adult consent will be programmed into the A-CASI to be read aloud to the respondent prior to administering the survey. We believe the combination of written and audio presentation will make the information accessible to low and non-literate respondents.

The questionnaire is based on the Pregnancy Prevention Approaches (PPA) household survey. Variations of this survey have been approved for other similar data collection efforts with a youth population.

OMB Control No	Date Approved	Link to Approved Instrument(s)
0970-0360	7/26/2010	http://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201003-0970-003&icID=186736
0970-0360	8/17/2011	http://www.reginfo.gov/public/do/PRAICList?ref_nbr=201107-0970-003
0970-0360	9/27/2011	http://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201108-0970-006&icID=186736
0970-0360	9/27/2011	http://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201108-0970-006&icID=192281

The questionnaire for this data collection has been modified to include some additional items. The additional (not previously approved) questions are written at a 3^rd grade reading level. Furthermore, all questionnaires will be programmed into the A-CASI to be read aloud to participants. Technical language (e.g. contraceptives, sexual intercourse, etc.) has been defined throughout the survey. Additionally, respondents will be provided with a flipbook of show cards with survey definitions for terms that may be unfamiliar.

4. Comment: Please describe clearly on consent forms and questionnaire cover pages whether the individual-level information gathered in the questionnaire will be shared with individual parents.

Response: The youth assent and the young adult consent have been updated to state “Your parents will not see your answers." (See Attachment E – Youth Assent and Attachment F – Young Adult Consent). None of the items within the questionnaire address abuse or violence towards self or others, so there will be no reason to share a youth’s responses with a parent.

5. Comment: Please describe in the supporting statement the visit choreography, and in particular, whether parents will be present during the administration of the questionnaire and how youth responses will be kept private during collection. Please describe any other procedures that will be put in place to reduce social desirability bias.

Response: Information about how the interview will be conducted, including measures used to ensure privacy in order to reduce potential social desirability bias, have been added to the summary statement (see page 4 of Supporting Statement Part A: Justification under the heading ‘Overview of Data Collection Procedures’).

6. Comment: Please insert language into consent forms and questionnaire cover pages that clearly outlines who the agency will be reporting to in the event that a respondent demonstrates that he/she is a danger to themselves or others (state authority, parent, both, etc.). We understand that reporting requirements vary by state and age of majority; perhaps youth will want clarification on this point from the data collector.

Response: The youth assent and young adult consent have been updated to indicate that a police officer or parent will be notified if youth are at risk of harming themselves or others. Additionally, data collectors will carry crisis brochures (in both English and Spanish) with community-specific resources available for substance abuse or other crisis support, child abuse reporting, and sexual/reproductive health services.

7. Comment: Please use the supporting documentation to elaborate on the training that survey administrators will be given and the use of help screens, show cards, and similar to answer respondent questions on technical language and context (ex. “What is celibacy?” “What is gonorrhea?”).

Response: The summary statement has been updated with information about how data collectors respond to questions about technical language (see pages 4 to 6 of Supporting Statement Part A: Justification under the heading ‘Overview of Data Collection Procedures’).

Survey administrators will be instructed to provide respondents with a show card flipbook of terms from the survey, question numbers where the terms are referenced, and definitions of the terms. Where appropriate, photos and slang terms will be included to further clarify the meaning of technical language. As respondents have questions regarding terms used in the survey, the administrator will refer the respondent to the show card flipbook. Additionally, the survey will include instructions for referring the youth to the show card flipbook for further explanation of the terms.

For example, the text highlighted in red will appear on the screen for Question 2.5.

2.5. Have you ever received any of these services from a medical provider or clinic?

(For more information on terms used here, view page 7 of the show card flipbook.)

CHECK ALL THAT APPLY

□ Counseling or information about birth control

□ Counseling or information about an Intrauterine Device (IUD)

□ Counseling or information about Implanon® (a hormone-release device placed under the skin on your arm)

□ Counseling or information about emergency contraception, also known as “Plan B” or “Preven” or the “morning-after pill”

□ Counseling or information about sexually transmitted diseases (STDs)

□ I have never received any of these services

(Excerpt from show card flipbook)

Intrauterine Device (IUD) –The intrauterine device (IUD) is a small, T-shaped device that is put into and left inside the uterus, or womb. Two types are available in the United States: the hormonal IUD (Mirena) and the copper IUD (ParaGard). The copper IUD works to stop sperm from reaching an egg. It also stops an egg from attaching to the uterus. The hormonal IUD (Mirena) does the same things. It also prevents the ovaries from releasing an egg.

8. Comment: Please ensure FISMA compliance prior to web-based data collection methods are used.

Response: ICF will implement an information systems security plan adhering to all relevant security-related management, operational, and technical safeguards which ensure proper functioning of the data system and protect against breaches in confidentiality and integrity of the data. Our security planning will be in compliance with relevant HHS automated information system security policies.  In addition, ICF will ensure that all NIST security controls are implemented commensurate with the overall security categorization standards defined in FIPS 199, “Standards for Security Categorization of Federal Information and Information Systems”.  Prior to system deployment, a system security scan will be performed to assess potential security vulnerabilities.

9. Comment: Please cite the privacy statute your agency will be invoking for this study in the informed consent documents and in the supporting statements.

Response: No personally identifiable information will be transferred to the CDC therefore the agency will not be invoking a privacy statute. Consents include information about efforts to maintain privacy and cases when privacy cannot be assured.

Part B:

1. Comment: On the face of it, a response rate of 76% seems a bit ambitious, given that households will be enumerated, and screened and that parental consent for minors will need to be obtained prior to youth assent and subsequently data collection. We see that you cited NGHS, a clinical trial. Do you mean NSFG? If NSFG is the basis for the estimated response rate, please clarify if this pertains the full NSFG sample (which included adults) or just youth.

Response: The response rate provided is based on the one obtained specifically among teenagers by National Survey of Family Growth (NSFG) 2006-2008. In particular, “the response rate for female teenagers was 77% and for male teenagers, 75%” (Abma et al, 2010, page 29). In the case of NSFG, the response rate among teenagers is very similar to the overall response rates, despite the fact that the data collection has to deal with the same challenges as the survey we are proposing, as for example, the required parental consent. A recently released document (Martinez et al, 2011, page 1) reported that slightly higher response rates were achieved for the NSFG 2006-2010. It must be noted that in determining the number of housing units to screen, in addition to non-respondent households, 15% of the addresses sampled are expected to be unoccupied or not actually housing units following estimation for NSFG (Lepkowski et al, 2010, page 150).

2. Comment: If your response rate is adjusted downward to something closer to 60%, how might this affect your power calculations and, potentially, your sampling units?

Response: In Table 2, Supporting Statement Part B:  Statistical Methods, we provide power estimations for subsamples of different sizes. It is shown that a power above 80% is estimated for a sample of size representing 66% of the total. That scenario would arise if the response rate were as low as 50% (i.e. 34% lower than expected) and no corrective action were taken. Table 2 is reproduced below with an additional column showing hypothetical response rates that would originate the different sample sizes.

However, we intend to reach a minimum sample size of 1,200 youths per community and administration wave. We based our expectations on the response rates among teenagers for a very similar survey, as referenced before. Nevertheless, data collection will be continuously monitored and if, due to unexpected response rates or other factors, the sample size is projected to be lower than the targeted minimum, two corrective actions will be taken: a) revisit non respondent households or housing units whose status was not determined using the best performing data collectors; and b) sample additional addresses.

Estimated power at 5% significance level to detect a change in the proportion of youth 15-19 who either did not have sexual intercourse or used contraceptive methods consistently during the 12 months previous to the interview

Sample size	Response rates	Power at 5% significance level*
Complete sample	76%	91%
66% of the sample	50%	83%
50% of the sample	38%	74%
33% of the sample	25%	58%

*Power estimations are based on 1,000 stochastic simulations using a beta-binomial distribution with a correlation parameter of 0.1. Correlation between beta distributions was introduced using a Gaussian copula. The estimation of the difference in the change across administrations between the two communities was computed for each simulated sample using a logistic regression fitted by generalized estimating equation.

3. Comment: Thank you for including your power analysis. Your analysis indicates that you anticipate a 10% difference in sexual intercourse, and a 10% difference in consistent use of contraception among teens that are sexually active. Is this expected decrease something documented in other literature, or based on reflection? If the latter (assuming that nothing is available in the literature), please indicate. 

Response: The hypotheses of change are based on program goals. Exact measures to compare from previous evaluations are not available. Results from a Cochrane meta-analysis of randomized control trials evaluating pregnancy prevention programs that combine multiple interventions are inconclusive regarding outcomes such as consistent condom use or initiation of sexual intercourse (Oringanje et al., 2009, page 16). Nevertheless, estimated effect sizes for these outcomes suggest that the hypothesis of change regarding consistent use of contraception is well within plausible results, while the hypothesis regarding abstinence may be rather ambitious (see table below). It is important to note, however, that the ultimate outcome evaluated in the power analysis is an increase in the percentage of teens that were not exposed to risk in the last year (specifically, a change from 89% to 94%). Such ultimate outcome could arise from different combinations of ‘abstinence’ and ‘consistent contraceptive use’, which will likely vary across communities depending on the specific emphasis of the intervention implemented.

Hypothesis	Implied Risk Ratio	Estimate effect size from meta analysis*1
The proportion of teens who had sexual intercourse during the 12 months prior to the interview decreases from 38% to 28% in the exposed community while it remains constant in the control community	.74	0.86 [0.77, 0.96] *2,4 0.90 [0.77, 1.06] *3,4
The proportion of teens who used contraception consistently during the 12 months prior to the interview increases from 70% to 80% in the exposed community while it remains constant in the control community	1.14	2.78 [0.98, 7.84] *2,5 1.10 [0.74, 1.64] *3,5

^*1 Oringanje et al., 2009, page 65.

^*2 Based on individually randomized clinical trials.

^*3 Based on cluster randomized clinical trials.

^*4 Result refers to initiation of sexual intercourse.

^*5 Result refers to consistent condom use.

4. Comment: We see that your power analysis references estimates of teenage compliance with contraception at 70% based on NSFG rates of using condoms consistently within this population. Could you walk us through the figuring? We see 61% of teens using condoms consistently * 75% of teens reporting using a contraceptive at last sex * 10% to reflect decreasing compliance over 12 months. In particular, please confirm your hypotheses are not focused solely on consistent condom use among teens sexually active but in any consistent contraceptive use. 

Response: The hypothesis focuses on consistent use of any contraception in the last 12 months (not solely condom use). There is no available information on the exact indicator selected, though there is information on several related indicators. Consistent condom use in the last month is used as a reference (it was 51.6% among female teens and 71.3% among males, or 61% on average, Abma et al., 2010, Table 16, page 23). We are certain that consistent contraceptive use is larger than consistent condom use: condom is the most popular but not the only type of contraceptive (for example, at last sex encounter, 54.7% of female teens used condom while 83.3% used any method, and 78.6% males used condom while 93.3% used any method, Abma et al., 2010, Table 15, page 23). On the other hand, we speculate that a longer reference period (12 months instead of one) will result in higher rates of inconsistency, though the exact extent of this decreasing compliance is unknown.

5. Comment: Where literature is cited, please be specific.

Response: Page numbers have been included in the citations to specify the portions of text referenced.

6. Comment: Please explain “culturally tailored flyers.”

Response: ICF International has prepared flyers to publicize data collection in the communities. These flyers have been designed for a low literacy audience and have been reviewed by grantees for appropriateness, suitability, and accuracy. The flyers note the presence of data collectors in the community and provide ICF International’s toll free number for inquiries about the Teen Pregnancy Prevention study. The flyers are intended to be distributed to community leaders by the Field Supervisors and posted in high-traffic areas throughout the community.

7. Comment: Please give some detail on the re-visitation protocol that will be utilized when a parent/guardian cannot be reached to give consent to data collection.

Response: No data will be collected unless required consent and assent have been obtained.  If the parent or guardian is unavailable for the youth aged 15-17 years of age and cannot give consent, the data collector will inform the person to whom they are speaking with that they will revisit the home at a more convenient time to obtain consent.  The data collector will ask for a phone number to reach the respondent in order to secure a future appointment. The data collector will make notation in the log and will attempt the visit at a later date.

8. Comment: Please indicate on all cover letters and relevant documentation, a short paragraph with information required by the Paperwork Reduction Act  (From PRA implementation regulations §1320.8):

• Example:

  • “Public reporting burden of this collection of information is estimated to average X minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information including suggestions for reducing this burden to [Insert contact information].”

Response: Consent forms have been updated to include PRA language. Please see Attachment D – Parental Consent, Attachment E – Youth Assent, and Attachment F – Young Adult Consent.

9. Comment: Who will be the final proprietors of respondent data holding personally identifiable information? CDC? CBO? Contractor?

Response: Administrative data used by contractor for fielding purposes will be destroyed at the end of the contract period. At the conclusion of data collection, the CDC will receive de-identified datasets with baseline survey data.

10. Comment: Will respondents have any access to any web display of information collected?

Response: No, respondents will not be able to view information collected.

11. Comment: Please clarify if you intend to include Spanish language versions of all data collection instruments and consent forms.

Response: The baseline survey, youth consent, youth assent, and parental consent will be available in Spanish. Additionally, the A-CASI and web-based surveys are also available in Spanish.

References included in this response:

Abma JC, Martinez GM, Copen CE (2010). Teenagers in the United States: Sexual activity, contraceptive use, and childbearing, National Survey of Family Growth 2006–2008. National Center for Health Statistics. Vital Health Stat 23(30). Available at http://www.cdc.gov/nchs/data/series/sr_23/sr23_030.pdf

Lepkowski, JM, Mosher, WD, Davis, KE, Grove, RM, and Van Hoewyk, J (2010). The 2006–2010 National Survey of Family Growth: Sample Design and Analysis of a Continuous Survey. National Center for Health Statistics. Vital Health Stat 2 (150). Available at http://www.cdc.gov/nchs/data/series/sr_02/sr02_150.pdf

Martinez G, Copen CE, Abma JC (2011). Teenagers in the United States: Sexual activity, contraceptive use, and childbearing, 2006–2010 National Survey of Family Growth. National Center for Health Statistics. Vital Health Stat 23(31). Available at http://www.cdc.gov/nchs/data/series/sr_23/sr23_031.pdf

Oringanje C, Meremikwu MM, Eko H, Esu E, Meremikwu A, Ehiri JE (2009). Interventions for preventing unintended pregnancies among adolescents. Cochrane Database of Systematic Reviews, Issue 4. Art. No.: CD005215. DOI:10.1002/14651858.CD005215.pub2. Available at http://apps.who.int/rhl/reviews/CD005215.pdf