Supporting Statement A_TPP20_30 Day_10.13.2022

Supporting Justification for OMB Clearance of Teen Pregnancy Prevention FY20 Tier 1 and Tier 2 Implementation Study

Part A: Justification for the Collection of Implementation Data

June 2022

Department of Health and Human Services

Office of Population Affairs

1101 Wootton Pkwy

Suite 200

Rockville, MD 20852

Alexandra Osberg, 301-348-3537 alexandra.osberg@hhs.gov

Part A Introduction

In July 2020, the Office of Population Affairs (OPA) awarded three-year grants for Teen Pregnancy Prevention (TPP) Tier 1-Optimally Changing the Map for Teen Pregnancy Prevention through Replication of Programs Proven Effective to 49 organizations, referred to as the fiscal year (FY) 2020 Tier 1 grantees. In June 2021, OPA awarded two-year TPP Tier 1 grants to an additional 13 organizations, known as the FY 2021 Tier 1 grantees. These 62 Tier 1 grantee organizations are using a systems-thinking approach to replicate evidence-based, culturally appropriate, and age-appropriate programs and support services to prevent and reduce rates of teen pregnancy and sexually transmitted infections (STIs) and improve adolescent health in communities or populations disproportionately impacted by teen pregnancy and STIs.

In July 2020, OPA also awarded three-year grants for Tier 2-Innovation and Impact Networks to 13 organizations, referred to as the Tier 2 grantees. These 13 grantee organizations are forming multidisciplinary innovation and impact networks that will develop, test, refine, evaluate, and disseminate innovative new teen pregnancy and STI prevention programs to reduce the rates of teen pregnancy and STIs. Each grantee is tailoring their programming for adolescents, programs, and systems affiliated with one of the following priority areas: (1) juvenile justice, (2) foster care/child welfare, (3) caregivers, (4) expectant and parenting youth, (5) youth with disabilities, (6) youth access to and experience with sexual health care, or (7) youth engagement.

The TPP20 Implementation Study will add valuable information to what is known about the implementation of TPP programs. OPA is explicitly interested in understanding and documenting how the 75 Tier 1 and Tier 2 grantees are implementing TPP projects in their communities or selected priority areas, the challenges they have faced in this process, facilitators of successful implementation, and adaptations made in response to the COVID-19 pandemic. This package covers the review and documentation of grantee approaches and experiences.

1. Circumstances Making the Collection of Information Necessary

OPA seeks approval of a new clearance request for two years of data collection activities described herein to support the TPP20 Implementation Study.

Study Background

Although the U.S. has made great progress in reducing teen pregnancy, births to teen mothers are still much more common than in other western industrialized nations. In addition, not all teens are at equal risk; there are disparities in teen pregnancy and STI rates by race and ethnicity, and among youth who are experiencing homelessness, living in foster care, or involved with the juvenile justice system. There is some recent evidence to suggest that more than educational programming is needed to achieve significant improvements in the lives of youth (CDC, 2015). The TPP Tier 1 and Tier 2 grant programs invest in both the implementation of evidence-based programs (Tier 1) and the development and evaluation of new and innovative approaches (Tier 2) to prevent teen pregnancy and STIs among adolescents and promote adolescent health among populations with the greatest need to reduce disparities in teen pregnancy and birth rates. To understand, learn from, and document the experiences of grantees implementing the Tier 1 and Tier 2 grant strategies, we propose to collect implementation information on each grantee project.

This study includes a descriptive assessment and documentation of: (1) how the 62 projects awarded funding under the Tier 1 grant programs are using a systems-thinking approach to replicate effective programs with fidelity in communities with the greatest needs and disparities; and (2) how the 13 Tier 2 projects are working to establish and support multidisciplinary networks of partners to develop, refine, test, and disseminate innovative interventions. In-person or remote discussions with staff from the 75 grantees and some of their partner organizations will provide detailed information about the implementation and experiences of projects that represent the diversity of the grantees, their approaches, experiences, and communities.

Legal or Administrative Requirements that Necessitate the Collection

Under the authority of Division A, Title II of the Further Consolidated Appropriations Act, 2020 (Public Law No. 116-94), the Office of Population Affairs made available approximately $101 million in funds to support the OPA Teen Pregnancy Prevention Program. The goal of the funding is to have a significant impact on improving the health of adolescents and reducing teen pregnancy and STIs. Tier 1 grantees are intended to accomplish this through saturation of communities with the greatest needs and disparities using a systems-thinking approach to replicate effective programs with fidelity. Tier 2 grantees will establish, coordinate, and support a multidisciplinary network of partners to develop, refine, and test innovative interventions and to disseminate those demonstrated to be effective in improving health and preventing teen pregnancy and STIs. The funds support the HHS Strategic Goal to “Protect and Strengthen Equitable Access to High Quality and Affordable Healthcare.” Under section 241 of the Public Health Service Act, $6,800,000 was made available to carry out evaluations of teen pregnancy prevention approaches.¹ The TPP20 Implementation Study is a key piece of OPA’s ongoing effort to comprehensively evaluate its teen pregnancy prevention funding efforts as required by the legislation.

This study, designed to provide a deeper understanding of the TPP program, is authorized under Section 301 of the Public Health Service Act (42 U.S.C.241)².

2. Purpose and Use of the Information Collection

Objectives of the TPP20 Implementation Study

The TPP20 Implementation Study is intended to: (1) document how grantees are implementing the expected components of the grant programs; (2) summarize implementation strategies and key themes for each grant program; and (3) recommend additional evaluation options for future research into the Tier 1 and Tier 2 grant programs and the TPP Program overall. The implementation data collection requested for clearance in this package will address the following primary research questions:

• How are grantees and their partners using the core elements of the Tier 1 or Tier 2 grant strategy to implement their projects?

• How did grantees’ local context influence the planning and execution of the TPP grant strategy?

• What were the facilitators and barriers to implementation?

• Tier 1 Only: How do TPP projects use a systems-thinking approach to effect change in population-level outcomes?

• Tier 1 Only: How are grantees reaching populations disproportionately affected by teen pregnancy and at what scale?

• Tier 1 Only: How did grantees identify which interventions to implement and in which systems, settings, and contexts?

• Tier 1 Only: What interventions and other strategies have shown success in the community?

• Tier 2 Only: How are grantees identifying and engaging partners in intervention planning and implementation?

• Tier 2 Only: What were the grantees’ key priority area strategies?

• Tier 2 Only: To what degree were interventions flexible or adaptable, and what adaptations or adjustments did projects make to interventions?

• Tier 2 Only: What approach did grantees take to producing program evaluation plans?

Reviewing and reporting on TPP Tier 1 and Tier 2 program planning and implementation will include the following activities:

• A review of the 75 grantees’ original applications for Tier 1 or Tier 2 grants.

• A one-time, self-administered web-based pre-interview informational form completed by all Tier 1 and Tier 2 grantees [data collection approved under OMB control number 0990-0379].

• In-depth interviews with each of the 75 grantees and up to two implementation partner organizations for 31 of the 62 Tier 1 grantees and all 13 Tier 2 grantees. If COVID-19 conditions allow, up to 6 grantees will be selected for in person site visits.

• Observations of one grantee or partner meeting for up to 31 of the 62 Tier 1 grantees and all 13 Tier 2 grantees.

• Analysis and reporting of qualitative data.

Implementation Data Collection Instruments

The TPP20 Implementation Study will collect data on program planning and implementation. For Tier 1 grantees, this will include information about the development of the TPP project, partnerships, project settings and overall reach, evidence-based interventions being implemented, services included in the projects, and how the community and youth have been engaged in the project. For Tier 2 grantees this will include information about the development of the TPP project, implementation and oversight of the Innovation and Impact Network, engagement of partners and youth, changes and adaptations made to the project over time, and program evaluation plans. For both Tier 1 and Tier 2 grantees, this will also include information about promising practices, lessons learned, and barriers relevant to program implementation.

Information on these topics will be collected from existing program documents (including the grant applications submitted by each of the 75 TPP grantees), as well as individual interviews with grantees (all 75 projects), interviews with implementation partners (from the 13 Tier 2 projects and a sample of 31 Tier 1 projects), meeting observations, and in-person site visits (up to 6 grantees). The instruments in this clearance request are:

a. An Interview Guide for Tier 1 Grantee Staff. These interviews will collect information on how agency leadership conceived of the TPP project and related program design, the community and organizational context in which the project operates, the project’s partnerships and project structure, community reach and settings, the selection and implementation of evidence-based interventions, systems and other services engaged, and their approach to community engagement. The guide will be administered once with project leadership and other key staff from all 65 Tier 1 grantees.

b. An Interview Guide for Tier 1 Partner Organization Staff. These interviews will collect information from critical partners on how they coordinate with the grantee, what role they play in the TPP project’s development and implementation, how participants are reached and what services participants are provided, and the partners’ overall perspectives on the TPP project. The guide will be administered once with key partner staff for 31 of the 65 Tier 1 grantees.

c. An Interview Guide for Tier 2 Grantee Staff. These interviews will collect information on how agency leadership conceived of the TPP project, how the network was formed and implemented, the implementation of the project, how the project engages partners and youth, how the network has adapted and changed over time, program evaluation plans, and overall perspectives on the TPP project. The guide will be administered once with project leadership and other key staff from all 13 Tier 2 grantees.

d. An Interview Guide for Tier 2 Partner Organization Staff. These interviews will collect information from critical partners on their role in developing the project, how they coordinate with the grantee, what role they play in intervention development and implementation, how partners and youth are engaged in the network’s activities, how the innovation network is coordinated and supported, intervention evaluation and dissemination, adaptation and changes over time, and the partners’ overall perspectives on the TPP project. The guide will be administered once with key partner staff for 13 Tier 2 grantees.

To collect the most relevant information on each project’s implementation, site visit team leaders will customize the topic guides for each grantee community and the associated interviews. Attachments A-D provide the interview guides for grantee and partner staff described above.

The data collected through these interview guides are not available through any current sources.

Use of the Information Collection

Through the work of this study, OPA will ultimately have a set of comprehensive documents that provide important implementation and evaluative information critical to understanding: (1) efforts to serve high need and high disparity populations with a systems-thinking approach to delivering effective programs and (2) resources and efforts to develop, test, and disseminate innovative interventions using a network-based approach to reducing teen pregnancy and STIs for key priority areas. These documents will include:

• A report synthesizing findings from the implementation study data collection: the report will include cross-grantee summaries within the Tier 1 and Tier 2 groups, with additional information summarizing findings across all 75 grantees, if appropriate, following data collection and analysis.

• A set of individual grantee profiles for each of the 75 grantees.

The data will be used for two purposes. First, the information will enable the study team to produce clear, concise profile descriptions of each project. The report will also provide a cross-grantee summary of grantee strategies for each of the two grant programs. This report will be critical in understanding the universe of strategies employed to reach the goal of serving entire communities and populations to reduce teen pregnancy and disparities. Second, the data will address the challenges and opportunities encountered by grantees and how grantees addressed or leveraged these. The data collection and analysis will also provide information critical to implementing subsequent similar community-wide initiatives.

3. Use of Improved Information Technology and Burden Reduction

The data collection plan reflects sensitivity to issues of efficiency, accuracy, and respondent burden. Where feasible, information will be gathered by extracting it from existing documents. Protocols for telephone interviews and in-person interviews and group discussions during site visits will be customized for each site to focus on information that is relevant for that site and that could not be obtained from documents.

In addition to pulling information from project documents (e.g., applications and progress reports) and performance measures, OPA’s contractor, Abt Associates, will collect a limited set of information from each grantee through a web form, including basic information about the grantee’s TPP project services and major partner organizations. (This data collection is approved under OMB control number 0990-0379). The feedback collected will inform the planning of implementation activities and the development of grantee site profiles. Collecting this information in advance of the interviews will also allow grantees to gather this factual information more easily and under a more convenient timeframe, saving time during the interviews, which will focus on grantee experiences, observations, reflections, and recommendations.

Improved information technology will be used when appropriate. For example, when program information or documents can be sent electronically, we will not request a hard copy of the documents.

4. Efforts to Identify Duplication and Use of Similar Information

While OPA (formerly the Office of Adolescent Health/OAH) has funded prior studies of other TPP cohorts (such as the TPP15 Tier 1B, and TPP18/19 cross site studies), the TPP20 Implementation Study is the only study of the unique approaches used by the TPP2020 cohort. The information collection requirements for the TPP20 Implementation Study have been carefully reviewed to determine what information is already available from existing studies and what will need to be collected for the first time. Although prior studies contribute to our understanding of teenage sexual risk behavior and past efforts to reduce it, OPA does not believe they provide sufficient information on current population-wide efforts to policymakers and stakeholders. In addition, Congress is supporting evaluations, including longitudinal evaluations, of adolescent pregnancy prevention approaches. The data collection for the TPP20 Implementation Study is an essential step in providing this information.

5. Impact on Small Businesses or Other Small Entities

Initiatives and programs in some sites may be operated by or in collaboration with small community-based organizations. The data collection plan is designed to minimize burden on such organizations by focusing interviews with their staff on their direct role in the intervention and its development or planning. The information being requested has been held to the absolute minimum required for the analyses.

6. Consequences of Collecting the Information Less Frequent Collection

It is essential to collect data directly from those responsible for implementing the program to understand and assess the systems-thinking based service provision or network design and testing efforts. We propose collecting these data once to capture implementation experiences during the planning and early implementation period. There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for the proposed data collection.

8. Comments in Response to the Federal Register Notice/Outside Consultation

The 60-day notice was published in the Federal Register on July 25, 2022, vol, 87, No. 141; pp. 44130-44131. The text is found in Attachment E. At this time there are no comments or responses to questions.

Below, we provide the names and contact information of the persons consulted in the drafting and refinement of the implementation study protocols:

Kimberly Francis, Ph.D.

Abt Associates

10 Fawcett Street

Cambridge, MA 02138

(617) 520-2502

Tanya de Sousa, MPP

Abt Associates

6130 Executive Blvd.

Rockville, MD 20852

(301) 347-5695

Lesley Freiman, MPP

Abt Associates

6130 Executive Blvd.

Rockville, MD 20852

(617) 520-2334

Cynthia Castaldo-Walsh, Ph.D.

Decision Information Resources, Inc.

3900 Essex Lane, Suite 900

Houston, TX 77027

(832) 485-3777

Alexandra (Warner) Osberg, MPH

Office of Population Affairs

Office of the Assistant Secretary for Health

U.S. Department of Health and Human Services

1101 Wootton Parkway, Suite 200

Rockville, MD 20852

(301) 348-3537

9. Explanation of any Payment/Gift to Respondents

No payment or gift will be made to grantee staff and partner agency staff for being interviewed by phone/videoconference or during site visits.

10. Assurance of Confidentiality Provided to Respondents

Privacy protections are embedded in the study design. Abt Associates will secure approval (or exemption after initial review) from its internal Institutional Review Board (IRB) for the collection of implementation study data (IRB review is currently underway and approval or exemption is anticipated by August 2022). Data collection will only occur if verbal informed consent is provided by interview respondents. Information collected will be kept private to the extent permitted by law. Implementation study respondents will receive information about how their information will be kept private to the extent allowed by law when arrangements are made for meeting with them. This information will be repeated as part of the verbal consent process prior to conducting the interviews (see Attachments A-D for the verbal consent language). Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. No confidential information will be collected or recorded. Individuals will not be cited as sources of information in prepared reports.

Site visit field staff will be informed about privacy and confidentiality procedures during training and will be prepared to describe them and to answer questions raised by local program staff. All research staff working on the project will have been trained to protect private information and have completed an IRB-approved human subjects training course (through CITI) prior to beginning the data collection effort. The research team will not collect or keep any papers that contain the names or other identifying information for people served by the TPP projects.

To further ensure privacy is maintained, the research team will also complete a Data Security Plan that will be reviewed and approved by Abt’s Cyber Security team. The Data Security Plan will describe what types of data will be collected, where and how the data will be transmitted and stored, who has access to the data, and when and how the data will be destroyed.

The study will not ask any sensitive questions.

Youth under age 18 will not participate in any direct data collection through the implementation study interviews.

11. Justification for Sensitive Questions

The interview protocols do not contain sensitive questions. Interviews will focus on components of each Tier 1 and Tier 2 grant project, interviewee experiences with planning and implementing the grant, reflections on these experiences, and lessons learned

12. Estimates of Annualized Hour and Cost Burden

A single data collection per individual is anticipated. Based on experience with similar protocols and respondents, two hours are allotted for each grantee interview and 90 minutes are allocated for each implementation partner interview. The total burden for this data collection is estimated to be 388 hours, with a total cost of $16,693.44 (please see Tables 12.1 and 12.2).

Table A12.A. Calculations of Burden Hours for Implementation Study Participants

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Tier 1 Grantee Staff	Attachment A	124	1	2	248
Tier 1 Partner Staff	Attachment B	62	1	1	62
Tier 2 Grantee Staff	Attachment C	26	1	2	52
Tier 2 Partner Staff	Attachment D	26	1	1	26
Total		238			388

Table A12.B Estimated Annualized Burden Costs for Implementation Study Participants

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Tier 1 Grantee Staff	248	$47.10a	$11,680.80
Tier 1 Partner Staff	62	$29.13 b	$1,806.06
Tier 2 Grantee Staff	52	$47.10a	$2,449.20
Tier 2 Partner Staff	26	$29.13 b	$757.39
Total	388		$16,693.44

^a The median hourly wage for grantee staff is calculated based on information from the Bureau of Labor Statistics; May 2021 National Occupational Employment and Wage Estimates found at http://www.bls.gov/oes/current/oes_nat.htm: General and Operations Managers (SOC code 11-1021).

^b The median hourly wage for employer and program partners is calculated based on information from the Bureau of Labor Statistics; May 2021 National Industry-Specific Occupational Employment and Wage Estimates found at http://www.bls.gov/oes/current/oes_nat.htm: Health Education Specialists (SOC code 21-1091).

13. Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/ Capital Costs

These information collection activities do not place any additional cost on respondents.

14. Annualized Cost to Federal Government

Data collection will be carried out by Abt Associates, under contract with OPA to conduct the TPP20 Implementation Study. The total cost for collecting the implementation data is $817,503, and the annual cost is $408,751.50.

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Virtual and In-person Site-Visits: Cross-grantee Summary Report and Grantee Profiles

Interview data for each grant project for the individual grantee profiles as well as across grantees will be summarized to highlight the range of implementation characteristics and strategies and common themes.

The interview team will code certain basic information about the grants into binary, continuous, or categorical variables as appropriate. The coding will be done by the note-taker, and a system of quality control checks will be implemented to ensure accuracy and consistency. The grantee profiles will largely consist of these data; for the cross-grantee assessment, these data will be summarized using descriptive statistics (e.g., means, frequencies, or distributions) to address relevant data elements.

A qualitative analytic approach will be used to identify themes emerging from open-ended responses and thoroughly describe the key data elements of interest. The study team will use NVivo qualitative software to systematically code and summarize all qualitative data. The study team will determine most topics and coding categories prior to coding, but we expect codes will be revised and expanded after initial review of the data. To ensure accuracy and consistency, senior qualitative researchers will review the coded transcripts and interview notes, reconcile any differences in coding, and confirm the thematic findings.

Findings from the analysis of interview data will be synthesized into a report. The report will include both a cross-grantee summary and individual grantee profiles to provide OPA, practitioners, communities, and lawmakers with a clear understanding of how Tier 1 and Tier 2 projects were designed, and how key approaches were used to expand the impact on teen pregnancy and reduce disparities at the community or focus population level. The cross-grantee summary report will include quantitative information such as the total number of youth served, the setting or system types, programs used, and populations targeted (for Tier 1 grantees), and the total number of formal network partners, the types and roles of partners, innovations developed, and stages of development and testing (for Tier 2 grantees). Individual grantee profiles will include general implementation features including, but not limited to, the name of the grantee, a description of partners and their roles, program models used, target populations, implementation settings, number of implementation sites, and other characteristics that will round out a succinct but informative picture of each project.

Publishing and Dissemination

The reports and grantee profiles, as well as derivative products or excerpts (e.g., case studies, special topics briefs, articles, presentations), will be provided publicly and targeted to specific audiences, such as Federal and local grantee staff, nonprofit service providers, and researchers. More traditional approaches (e.g., journal publications, conference presentations, and policy briefs) are frequently used for research/policy-aware audiences, while other approaches (e.g., webinars and the use of infographics, data visualizations, social media, whiteboarding, and explanatory videos) often appeal to a broader audience. The project will most likely leverage multiple strategies, given audiences that respond differently to written text versus visual images.

Project Time Schedule

OPA is requesting a two-year clearance. Data collection (in-person and virtual site visits) is expected to begin in November 2022 and continue through February 2023. Grantee profiles developed from the interviews will be completed in March 2023, and the project summary report will be completed in June 2023. Below is a schedule of the data collection efforts for the implementation study:

Table A16.1. Timeline for Use of Implementation Study Protocols

Instrument	Date of 60-Day Submission	Date of 30-Day Submission	Date Clearance Needed	Date for Use in Field
Instrument 1: Tier 1 grantee interview guide	June 2022	September 2022	November 2022	November 2022– February 2023
Instrument 2: Tier 1 grantee partner/sub-awardee interview guide	June 2022	September 2022	November 2022	November 2022– February 2023
Instrument 3: Tier 2 grantee interview guide	June 2022	September 2022	November 2022	November 2022– February 2023
Instrument 4: Tier 2 grantee partner/sub-awardee interview guide	June 2022	September 2022	November 2022	November 2022– February 2023

17. Reason(s) Display of OMB Expiration Date is Inappropriate

All protocols will display the OMB number and the expiration date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

1 For more information, see: https://www.congress.gov/116/plaws/publ94/PLAW-116publ94.pdf

2 For more information, see: https://www.reginfo.gov/public/do/DownloadDocument?objectID=92941901