0990-0382 Supporting Statement A_REVISED

Supporting Justification for the Extension of OMB Clearance of the Evaluation of Adolescent Pregnancy Prevention Approaches

Part A: Justification for the Collection of the Data

January 2015

Submitted by:

Office of Adolescent Health

U.S. Department of Health and Human Services

1101 Wootton Parkway, Suite 700

Rockville, MD 20852

Project Officer: Amy Farb

INTRODUCTION

The Office of Adolescent Health (OAH) is requesting an extension through September 14, 2016 of the OMB No. 0990-0382, Evaluation of Adolescent Pregnancy Prevention Approaches (PPA) Follow-up Data Collection, currently approved through May 31, 2015. The PPA study is an eight-year multi-site random assignment evaluation designed to study the effectiveness of promising approaches to teen pregnancy prevention. OAH is requesting this extension in order to accommodate the ongoing follow-up data collection, which will extend past the current OMB expiration date. This extension request does not involve changes to the data collection instruments previously approved by OMB, which are included in this submission for reference. The previously approved burden estimates have been updated to reflect the current data collection progress. These updated estimates are presented in this submission.

A1. Circumstances Making the Collection of Information Necessary

a. Background

High rates of teen pregnancy, sexually transmitted infections (STIs), and associated sexual risk behaviors remain a troubling issue in the United States. Nationwide, 24 percent of high school students report having had four or more partners by graduation, and nearly 40 percent of sexually active students had not used a condom during their last sexual intercourse. These behaviors increase the risks of pregnancy and STIs, including HIV. Preliminary national data for 2012 indicate there were approximately 29.4 births per 1,000 females 15 to 19 years of age, a rate higher than in most other industrialized countries. In addition, estimates suggest that adolescents and young adults account for half of all new STI cases in the United States every year.

Policymakers and practitioners have had a long-standing interest in identifying programs effective in reducing these risks. To date, researchers have identified more than two dozen teen pregnancy prevention programs with evidence of effectiveness of reducing teen pregnancy, STIs, and associated sexual behaviors. However, there is still need to identify a broader range of effective programs and approaches, especially those targeting high-risk populations. As a result, researchers and practitioners continue to develop and test innovative new approaches to teen pregnancy prevention.

b. Legal or Administrative Requirements that Necessitate the Collection

Public Law 110-161, which set fiscal year (FY) 2008 appropriations levels, included the following language: “$4,500,000 shall be available from amounts available under section 241 of the Public Health Service Act to carry out evaluations (including longitudinal evaluations) of adolescent pregnancy prevention approaches.” The same language appropriated $4,450,000 in each of FYs 2009, 2010, and 2011 and $8,455,000 in each of FYs 2012, 2013, and 2014. These funds have been used for the PPA evaluation.

c. Study Objectives

The PPA study is designed to expand available evidence on effective ways to prevent and reduce pregnancy and related sexual risk behaviors among teens in the United States. The eight-year (2008-2016) evaluation is documenting and testing promising pregnancy prevention approaches in seven sites across the United States, each of which has been implementing a different program. The PPA evaluation has two components: (1) a rigorously designed impact study of the selected programs and (2) an in-depth implementation analysis of each program. The impact studies use experimental designs and longitudinal survey data to assess the effectiveness of each selected program on its own, compared to a control group in the same site. The implementation analysis, which OAH completed in 2014, examined the context and delivery of each program and provides a basis for interpreting estimates of program impacts.

OMB approved information collection request No. 0990-0382 to support follow-up data collection for the PPA impact study. This follow-up data collection involves longitudinal surveys administered to the individual youth enrolled in each study site. To date, follow-up data collection has been completed in one of the seven PPA study sites, Chicago Public Schools (Table A.1). In a second site (Live the Life Ministries), the planned follow-up data collection was canceled because the site lost funding for the teen pregnancy prevention program it had been implementing. In the five remaining PPA sites, follow-up data collection is expected to continue beyond the current OMB expiration date of May 31, 2015. OAH is requesting an extension to the current expiration date to complete the data collection as planned.

Table A.1. Follow-up Data Collection Progress and Response Rates to Date

			Data Collection			Response Rates For Released Cases
Site	Total Sample Enrolled		Short-Term Follow-Up	Longer-Term Follow-Ups		Short-Term Follow-Up	Longer-Term Follow-Ups
Chicago Public Schools	1535		Completed	Completed		94%	90%
Oklahoma Institute for Child Advocacy (OICA)	1039		Completed	Continuing Through 9/2015		84%	84%
OhioHealth	598		Completed	Continuing Through 9/2016		82%	75%
Children’s Hospital Los Angeles	954		Completed	Continuing Through 12/2015		83%	75%
EngenderHealth	823		Continuing Through 3/2015	Continuing Through 3/2016		91%	85%
Live the Life Ministries	0		Cancelled	Cancelled		NA	NA
Princeton Center for Leadership Training	1934		Continuing Through 2/2015	Continuing Through 2/2016		87%	75%

A2. Purpose and Use of the Information

OAH will use the PPA follow-up data to estimate the impacts of the selected programs on youth outcomes. The follow-up data encompass two types of outcomes. The first are sexual risk outcomes, including the extent and nature of sexual activity, use of contraception (if sexually active), pregnancy, and testing for and diagnoses of STIs. The second are a series of intermediate outcomes that may be associated with the sexual risk outcomes and therefore important to measure as potential pathways of any program effects on sexual risk behavior. Examples of these outcomes include participation in and exposure to pregnancy prevention programs and services, intentions and expectations of sexual activity, relationships with family and friends, knowledge of contraception and sexual risks, dating behavior and alcohol and drug use. OAH plans to assess the impacts of each program on both types of outcomes.

A3. Use of Technology to Reduce Burden

The PPA follow-up data collection methods have been customized to the unique circumstances of each participating site. In one site (Children’s Hospital Los Angeles), OAH is using audio computer-assisted self-administered interviewing (ACASI) to collect follow-up data from a high-risk sample of adolescent mothers. In two other sites (OhioHealth and EngenderHealth), OAH is administering follow-up surveys individually by telephone. In the remaining two sites (Princeton Center for Leadership Training and OICA), OAH is administering follow-up surveys primarily in group-based settings using paper and pencil interviewing (PAPI). OAH is not proposing any changes to follow-up data collection as part of this extension request.

A4. Efforts to Avoid Duplication

The PPA follow-up data collection does not duplicate any other information collection by OAH.

A5. Methods to Minimize Burden on Small Entities

No small entities will be impacted by this information collection.

A6. Consequences of Not Collection Data

Completing the PPA follow-up data collection is necessary to answer the study’s primary research questions and determine the effectiveness of the selected programs.

A7. Special Circumstances

There are no special circumstances associated with this information collection.

A8. Federal Register Announcement and Consultation

The 60-day notice was published in the Federal Register on February 4, 2015 on page 6090, with the document identifier HHS-OS-0990-0382-60D. No comments were received.

A9. Payments or Gifts

The incentive amounts for completing the surveys range from $10 to $50 based on the type of site, mode of survey administration, mobility of the target population, and length of follow-up. For sites serving general school-age populations, the incentive amounts are $10 for group administration and $25 for phone administration. For the OhioHealth site serving low-income teen mothers, the incentive amounts are $25 for the shorter-term follow up conducted 18 months after study enrollment and $50 for the longer-term follow up conducted 30 months after enrollment. For the CHLA site serving low-income mothers, the incentive amounts are $20 for the shorter-term follow-up conducted 12 months after study enrollment and $30 for the longer-term follow-up conducted 24 months after enrollment. For the OICA site serving youth in group foster care homes, the incentive amounts are $25 for the short-term follow-up and $50 for the longer-term follow-up. The incentives amounts have been successful in helping OAH achieve high survey response rates (see Table A.1 above).

A10. Assurance of Confidentiality

OAH has embedded protections for confidentiality in the study design. Data collection occurs only if informed consent was provided by a parent or legal guardian if the respondent was a minor, or by respondents themselves if they were 18 or older. Consent for the duration of the study was collected prior to baseline data collection. Youth without consent were not included in the study sample and no data have been collected. The consent form notes that the evaluation has obtained a Certificate of Confidentiality from the National Institutes of Health (NIH). In addition, youth assent is obtained prior to each survey administration. Copies of the consent and assent forms for each site are found in Attachment B.

For group-based PAPI data collection, the study protocol helps reassure respondents of their privacy. It is made clear to respondents that identifying information will be kept separate from questionnaires. The questionnaire and envelope have a label with a unique ID number; no identifying information appears on the questionnaire or return envelope. Before turning completed questionnaires in to field staff, respondents place them in blank envelopes and seal them.

Telephone surveys are completed by interviewers recording respondent’s answers on a hard copy of the survey. Prior to beginning the survey, a statement ensuring privacy and the youth assent is read aloud and respondents are given a chance to verbally opt out of the survey. As with the hard copy for the group administrations, no identifying information is attached to the questionnaire; only a unique study ID will be included on the questionnaire.

A11. Justification for Sensitive Questions

OAH is not proposing adding any new measures as part of this extension request. Because the programs being evaluated are designed to reduce teen pregnancy and associated sexual risk behaviors, the follow-up surveys must ask for information of a sensitive nature. The surveys include measures of the extent and nature of sexual activity, use of contraception, pregnancy, and testing for and diagnoses of STIs. Although these questions are sensitive, they are commonly and successfully asked of youth similar to those who will be in the study. Many of the sensitive items related to sexual activity will be asked only of sample members who report being sexually active.

Table A.2: Summary of Sensitive Questions and their Justification

Topic	Justification
Intentions regarding sexual activity	Intentions regarding engaging in sex and other risk-taking behaviors are extremely strong predictors of subsequent behavior (Buhi and Goodson, 2007). Intentions are strongly related to behavior and will be an important mediator predicting behavior change.
Drug and alcohol use	There is a substantial body of literature linking various high-risk behaviors of youth, particularly drug and alcohol use, sexual intercourse, and risky sexual behavior. The effectiveness of various program strategies is expected to differ for youth who are and are not experimenting with or using drugs and alcohol (Tapert et al., 2001; Li et al., 2001; Boyer et al., 1999; Fergusson and Lynskey, 1996; Sen, 2002; Dermen et al., 1998; Santelli et al., 2001.)
Sexual activity, incidence of pregnancy and STDs, and contraceptive use	Sexual activity, incidence of pregnancy and STDs, and contraceptive use are all key outcomes for the evaluation. The majority of these questions are asked only of youth who report being sexually active.

A12. Estimates of Annualized Burden Hours and Costs

a. Annualized Burden Estimates

OAH has updated the previously approved burden estimates to reflect the achieved study sample enrollment and current data collection progress and response rates (Table A.3). This update resulted in a reduction in the annualized number of responses, from 5,786 to 3,117. The decrease in the annualized number of responses can be attributed primarily to the completion of follow-up data collection in the Chicago site and the canceling of follow-up data collection in the Live the Life site. This decrease in the annualized number of responses in turn resulted in a decrease in the estimate of annual burden hours, from 3,675 to 1,973.

Table A.3. Updated Estimates of Annualized Burden for the PPA Follow-Up Data Collection

Site	Annualized Number of Respondents	Number of Responses Per Respondent	Average Burden Hours per Response	Annualized Burden Hours
Chicago Public Schools	0	0	0	0
Oklahoma Institute for Child Advocacy (OICA)	294	2	42/60	412
Ohio Health	148	3	42/60	310
Children’s Hospital Los Angeles	254	2	36/60	305
EngenderHealth	240	2	36/60	288
Live the Life Ministries	0	0	0	0
Princeton Center for Leadership Training	548	2	36/60	658
Total				1,973

b. Estimates of Annualized Costs

The PPA follow-up data collection does not impose a financial burden on youth respondents.

A13. Estimates of Other Total Annual Costs

The PPA follow-up data collection does not place any additional cost on youth respondents.

A14. Annualized Cost to the Federal Government

The total estimated cost to the government for continuing to collect data for 16 months is $1,326,986. The estimated annualized cost to the government for follow-up data collection is $995,244 ($1,326,986/16 months = $82,937 per month * 12 months = $995,244).

A15. Explanation for Program Changes or Adjustments

This request is an adjustment reducing the annual burden from 3,675 hours to 1,973 hours. This decline reflects a reduction in the annualized number of responses, due primarily to the completion of follow-up data collection in the Chicago site and the canceling of follow-up data collection in the Live the Life site.

A16. Plans for Tabulation and Publication and Project Time Schedule

1. Analysis Plan

OAH will use the PPA follow-up data to estimate the impacts of each program on youth outcomes. Program impacts will be estimated separately for each site using a regression framework. The empirical specification for the regression model will depend on the unit of random assignment, which will depend on the type of program provided at a specific site. Some sites randomly assign youth to research conditions in intact groups (e.g,. schools), whereas some sites employ random assignment of individuals within the site. With random assignment of individual youth, the regression model can be expressed as:

(1) ,

where y_i is the outcome of interest for respondent i; x_i is a vector of baseline characteristics for respondent i, including baseline measures of the key outcomes; T_i is an indicator equal to one if the respondent is in the treatment group and zero if in the control group; and _i is a random error term for respondent i. The vector of baseline characteristics x_i will include demographic characteristics such as age, gender, race/ethnicity, and baseline measures of key outcomes. The parameter estimate for  is the estimated impact of the program.

In some sites, youth were randomly assigned in intact groups (e.g., schools) and the estimation must account for the correlation of outcomes between respondents in the same group, as they may be exposed to similar influences not otherwise captured in the regression model. Therefore, each respondent cannot be considered statistically independent. For these sites, the regression model will be modified as follows:

(2) .

The general structure of the model is the same, but now y_is is the outcome measure for respondent i in group s (and similarly for the vector of baseline characteristics x_is and the error term _is). The treatment status T_s is now defined by group rather than by individual. Most importantly, the error term in Equation (2) accounts for the clustering of respondents within groups because of the inclusion of the group-level error term _s. If this error term is excluded, the precision of the impact estimates could be overstated. As in Equation (1), the estimated impact of the program is .

The specific maximum-likelihood methods for estimating the parameters of the models will depend on the form of the dependent variable. Logistic regression procedures will be specified for binary outcomes (such as whether the student has an STD) and multinomial regression procedures will be specified for categorical outcomes (such as the number of sexual partners).

2. Time Schedule and Publications

Program implementation reports for all sites and an impact report for the Chicago site were released on a rolling basis beginning in summer 2013. These reports are available on the OAH website at http://www.hhs.gov/ash/oah/oah-initiatives/for-grantees/evaluation/index.html. Site-specific interim and final impact reports for the remaining sites will be released on a rolling basis in 2015 and 2016.

A17. Display of Expiration Date for OMB Approval

All instruments will display the OMB number and the expiration date.

A18. Explanation of Exceptions

No exceptions are necessary for this information collection.

i