0990-0290_09-02-08

08-22-08

August 2008

SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION

for

Evaluating the Title XX
Adolescent Family Life (AFL) Program:

Care Demonstration Projects

Prepared for

Johanna Nestor

Office of Population Affairs/DHHS

1101 Wotton Parkway, Suite 700

Rockville, MD 20852

(240) 453-2808

Prepared by

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

RTI Project Number 0211776

___________________

RTI International is a trade name of Research Triangle Institute.

Table of Contents

Section Page

A. Justification 2

1. Circumstances Making the Collection of Information Necessary 2

2. Purpose and Use of the Information Collection 3

3. Use of Improved Information Technology and Burden Reduction.........................4

4. Efforts to Identify Duplication and Use of Similar Information 5

5. Impact on Small Businesses or Other Small Entities 6

6. Consequences of Collecting the Information Less Frequently 6

7. Special Circumstances Relating to the Guidelines of 5CFR 1320.5 6

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 7

9. Explanation of Any Payment or Gift to Respondents 7

10. Protection of Data Security and Participant Privacy 11

11. Justification for Sensitive Questions 12

12. Burden Estimate (Total Hours & Wages) 13

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers 14

14. Annualized Cost to the Federal Government 14

15. Explanation of Program Changes or Adjustments 15

16. Plans for Tabulation and Publication and Project Time Schedule 15

17. Expiration Date 20

18. Exceptions to Certification for Paperwork Reduction Act Submissions 20

B. Collection of Information Employing Statistical Methods 21

1. Respondent Universe and Sampling Methods 21

2. Procedures for the Collection of Information 25

3. Methods to Maximize Response Rates and Deal with Nonresponse 27

4. Tests of Procedures or Methods to be Undertaken 28

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 29

References 31

Appendices

A. Statute/Regulation Mandating or Authorizing the Collection of Information

B. Cross-Site Evaluation Data Collection Materials

C. Federal Register Notice to the Public

D. RTI Institutional Review Board Approval Notice

E. Assurances of Confidentiality and Study Descriptions Provided to Respondents

F. Recruiting Materials

G. Cross-site Evaluation Study Protocol

List of Exhibits

Number Page

Exhibit 1. Cross-Site Evaluation Research Questions 4

Exhibit 2. Persons Consulted Outside the Agency 8

Exhibit 3. Studies Involving Child and Adolescent Respondents Receiving $10 Incentives and Corresponding Response Rates 10

Exhibit 4. Studies Involving Adolescent Respondents Receiving Raffle Incentives and Corresponding Response Rates 10

Exhibit 5. Studies Involving Adolescent Respondents Receiving Teaser Incentives and Corresponding Response Rates 10

Exhibit 6. Studies Involving Respondents Receiving $20 Incentives and Corresponding Response Rates 11

Exhibit 7. Description of Sensitive Questions, Justification for Inclusion, and Use of Data 13

Exhibit 8. Estimated Annualized Burden Hours 14

Exhibit 9. Estimated Annualized Cost to Respondents 14

Exhibit 10. Logic Model for the Cross-Site Evaluation of the AFL Program 16

Exhibit 11. Study Hypotheses 20

Exhibit 12. Cross-Site Evaluation Analyses 21

Exhibit 13. Time Schedule for the Entire Project 22

Exhibit 14. Longitudinal Response Rates and Numbers of Adolescents 23

Exhibit 15. Longitudinal Completion and Retention Rates for Prior Studies 24

This document provides a Supporting Statement to accompany a request for approval of revisions to the Office of Population Affairs (OPA) care demonstration project core evaluation instruments (OMB 0990-0290), which are used to collect information in order to evaluate Care demonstration projects funded by the Adolescent Family Life (AFL) Program which is administered by OPA. Approval is also requested for collection of information for the Cross-Site Evaluation of the AFL Program: Care Demonstration Projects.

A. Justification

This section provides detailed justification for the request for approval of revised core instruments and of collection of information for the Cross-Site Evaluation of the Title XX AFL Program: Care Demonstration Projects.

1. Circumstances Making the Collection of Information Necessary

The AFL Program is administered by the Office of Adolescent Pregnancy Programs (OAPP) within OPA to support Prevention demonstration projects providing abstinence education to adolescents and Care demonstration projects providing services to pregnant and parenting adolescents. The AFL program and cross-site evaluation are authorized by Title XX of the Public Health Service Act (USC 42 Chapter 6A Subchapter XVIII) (Appendix A).

The Title XX statute requires an independent evaluation of all demonstration projects funded through the AFL program. Because these evaluations are independent, the data collected from one project to another vary. Moreover, the independent evaluations do not always necessarily examine questions of particular statutory or policy relevance to the OPA. Thus, the OPA has developed core evaluation instruments for AFL Prevention and Care demonstration projects that reflect Title XX legislative requirements, as well as the A-H definition of abstinence education contained in the Welfare Reform Act of 1996. 

The use of these core evaluation instruments across AFL Prevention and Care projects enables the OPA to better monitor the direction and progress of the program. This is important on at least two counts: 

a. These are demonstration projects and are, therefore, developing and implementing new approaches to abstinence education for adolescents and services for pregnant and parenting adolescents and their families are well within the parameters of the Title XX statute. To direct its funding resources appropriately and efficiently, it is of great importance that the OPA be able to assess the success or failure of these approaches.  

b. The AFL program was recently evaluated by the OMB Program Assessment Rating Tool (PART). As a result of that evaluation, OMB recommended that the program develop and track a set of performance measures. Measures for AFL demonstration projects have been developed, and the program tracks them using data from the Prevention and Care core instruments (OMB 0990-0290 and OMB 0990-0291, expiring 9/30/2008).

The core evaluation instruments have already been approved by OMB but recommendations were made for revisions based on pilot testing and feedback from demonstration projects. This submission requests approval for the revised instruments in order to improve the ability of OPA to monitor project performance, to improve the quality of individual demonstration project evaluations, and to facilitate a cross-site evaluation of AFL demonstration projects. The OAPP estimates that 40,000 participants may use the surveys annually.

The specific aim of the cross-site evaluation is to evaluate the impact of AFL demonstration projects. Desired outcomes for Care projects providing services for pregnant and parenting adolescents include prevention and reduction of repeat pregnancy, increased educational attainment among adolescents, and increased immunizations among their children. For the cross-site evaluation, impact evaluation data will be collected by AFL Care projects from 972 adolescents aged 9 to 19 who are AFL service recipients or serve as comparison group participants. The cross-site evaluation will include demonstration projects with strong evaluation designs, namely randomized controlled trials and strong quasi-experimental designs. The research will include up to three data collection points using the three core evaluation instruments: (1) a baseline Care survey for pregnant adolescents, (2) a baseline Care survey for parenting adolescents, and (3) a follow-up Care survey to be administered 6 and 12 months after birth for adolescents pregnant at baseline and at 12 months after baseline for adolescents parenting at baseline. This submission requests approval for all three surveys.

The field of adolescent reproductive health is well poised to seize an opportunity for a large-scale evaluation with important public health and policy implications. To date, programs for pregnant and parenting adolescents have been shown to be effective in improving self-esteem, family relationships, school graduation rates, parenting attitudes and beliefs, use and/or intended use of contraceptives, and attitudes about and actual repeat pregnancy (Amin & Sato, 2004; Barnet et al., 2007; Black et al., 2006; Percy & McIntyre, 2001; Thomas & Looney, 2004). The proposed cross-site evaluation is an effort to advance the field of research and respond to calls for improvement in AFL’s program results/accountability (The White House, 2005) and for rigorous evaluation of adolescent reproductive health programs overall (Hoyer, 1998; Kirby, 2002; U.S. Government Accountability Office, 2006).

The AFL cross-site evaluation will be a meta-analysis impact evaluation to compare adolescents targeted by Care projects against adolescents not targeted. The cross-site evaluation presents a unique opportunity to evaluate the effectiveness of a multi-site funding program to improve outcomes for pregnant and parenting adolescents and their children.

2. Purpose and Use of the Information Collection

The purpose of the data collection and evaluation is to determine the impact of AFL demonstration projects on desired outcomes. Anticipated effects of Care projects providing services for pregnant and parenting adolescents include main effects on repeat pregnancy, educational attainment, and infant immunizations; mediating effects of involvement of the baby’s father, attitudes and beliefs about future education, parent involvement, sexual activity, and contraceptive use; and moderating effects of prior risk levels and demographic characteristics. Research questions that will be investigated using these instruments will vary from project to project; however, the cross-site evaluation is designed to answer specific research questions across projects. Key research questions for the cross-site evaluation are presented in Exhibit 1. Copies of data collection instruments are attached in Appendix B.

The information obtained from the proposed data collection activities will be used to inform OPA, policy makers, practitioners, and researchers about the effects of the AFL program activities. This information will enable OPA to more effectively address care for pregnant and parenting adolescents. These findings will inform the application of AFL program funds and priorities and will have policy implications for other mechanisms of providing funding for programs that provide care for pregnant and parenting teens.

Exhibit 1. Cross-Site Evaluation Research Questions

1. Was the program effective in producing the desired outcomes on the targeted mediator variables, including:

   1. Adolescent father’s involvement

   2. Adolescent attitudes and beliefs about future education

   3. Parent involvement

   4. Adolescent sexual activity and contraceptive use

2. Did the program effectively reduce the incidence of adolescent repeat pregnancy?

3. Did the program effectively increase educational attainment?

4. Did the program effectively increase compliance with recommended child immunization schedules?

5. Did the effects of the program vary based on moderator variables? Potential moderator variables include:

   1. Pre-program risk level of adolescents (prior reproductive health behavior, educational attainment, parent-child relationship, parent involvement)

   2. Demographic characteristics (adolescent age, marital status, race/ethnicity, gender, urbanicity, region, living arrangements, educational status, sources of financial support

6. Did the program achieve its effects on repeat pregnancy, educational attainment, and child immunizations by altering the mediating variables in #1?

3. Use of Improved Information Technology and Burden Reduction

The revised core evaluation instruments will be used in the conduct of the independent evaluations required of all AFL grantees by statute. Use of information technologies for these independent evaluations will therefore be dependent upon the capacities of specific grantees and their evaluators. 

The cross-site evaluation will rely on paper-and-pencil Teleform questionnaires to be self-administered by adolescents. One alternative method considered was to conduct telephone surveys. However, conducting surveys by telephone would be extremely time-consuming and costly, given the number of youth (n=972) expected to participate. In addition, response rates for telephone surveys are decreasing as new technology (answering machines, voice mail, caller identification) becomes available (O’Rourke et al., 1998), and non-locate rates in later waves of longitudinal telephone surveys are increasing, likely due to increased use of cellular phones and frequent switching of carrier companies. Further, we believe there would be serious issues of privacy and confidentiality if youth were asked to disclose sensitive information regarding reproductive health topics over the telephone. OPA’s contractor for the cross-site evaluation, RTI International, conducted a capacity assessment of Care projects to determine the best way to collect data across projects. Many participants do not have reliable access to computers, and using school computers for survey administration would not provide adequate privacy for respondents to feel comfortable answering questions honestly. Even if each classroom had a computer, there would be no privacy for the participants and little availability for all participants to use the computer to complete the survey in a timely manner. Most projects lack the capacity to use technology such as audio computer-assisted self interview (ACASI). Survey administration with Teleform instruments will minimize burden on AFL demonstration projects, while minimizing potential biases that might jeopardize our ability to address the evaluation research questions.

4. Efforts to Identify Duplication and Use of Similar Information

The purpose of the core evaluation instruments is to ensure uniform data collection across AFL demonstration projects in areas of particular statutory or policy interest to the OPA. While program evaluations might, in the absence of the core instruments, collect some similar data, core evaluation instruments ensure that this data is collected consistently. The OPA requires all AFL demonstration grantees, funded in FY 2005 and after, to incorporate the core instruments (0990-0290) into their evaluation. 

In designing the proposed data collection activities for the cross-site evaluation, we have taken several steps to ensure that this effort does not duplicate ongoing efforts and that no existing data sets would address the proposed study questions. To ensure that this study is forging new ground in our understanding of the effectiveness of the AFL program, we conducted an extensive review of the literature by examining several large periodical journal databases. We identified published articles or books containing the keywords, “adolescent,” “youth,” “pregnancy,” “parenting,” “education,” “school dropout,” and “immunizations.” In addition, to reviewing published information, we searched for “gray” literature by contacting well-known researchers in the field and by exploring the Internet. Searches were performed on several Internet search engines, including Google, Yahoo, AltaVista, Medline, and Science Direct, using search terms “adolescent,” “youth,” “pregnancy,” “parenting,” “education,” “school dropout,” and “immunizations.”

The results of the literature search and consultation with experts in the field revealed that although a fair amount of research has been conducted on programs for pregnant and parenting adolescents, little has been done to conduct a cross-site evaluation in these areas or evaluate the effectiveness of a program like AFL. Evaluations of programs for pregnant and parenting adolescents are sparse. Existing research shows that data have been analyzed within individual projects rather than across projects, and individual projects evaluated are not AFL projects (e.g., Baytop, 2006; Corcoran & Pillai, 2007, Hoyer, 1998; Seitz & Apfel, 1999). To date, no duplication of the proposed effort has been identified.

We have carefully reviewed existing data sets to determine whether any of them are sufficiently similar or could be modified to address OPA’s need for information on the effectiveness of the AFL program with respect to services for pregnant and parenting adolescents. Efforts to avoid duplication include a review of OPA’s administrative agency reporting requirement and of existing studies of OPA’s programs. We investigated the possibility of using existing data to examine our research questions, such as evaluations of past Care demonstration projects; individual and local evaluations of programs for pregnant and parenting adolescents; surveys by the National Campaign to Prevent Teen Pregnancy (2003); the National Survey on Family Growth (Abma, Martinez, Mosher, & Dawson, 2004; Albert et al., 2005); the National Longitudinal Study of Adolescent Health (1998); the National Survey of Adolescents and Young Adults: Sexual Health Knowledge, Attitudes and Experiences (Henry J. Kaiser Family Foundation, 2003); and the Youth Risk Behavior Survey (Eaton et al., 2006). However, none of these existing data included pre- and post-test data in a rigorous design using standardized instruments across multiple programs to test projects and services like the ones funded by the AFL program.

5. Impact on Small Businesses or Other Small Entities

To the extent that AFL demonstration projects might be considered small businesses or entities, the data to be collected from the core evaluation instruments (Appendix B) would still need to be collected in some form to satisfy the independent evaluation requirement of the AFL statute (Appendix A). Thus, any burden on demonstration projects will be minimal.

6. Consequences of Collecting the Information Less Frequently

While individual AFL demonstration project evaluations may collect similar data in the absence of the core evaluation instruments, the data would not be consistent across projects. This would hamper the OPA’s ability to effectively monitor and manage the direction of the program as a whole, as well as track the performance measures recommended by the OMB. 

If the cross-site evaluation were not conducted, it would be difficult to determine the value or impact of the AFL program on the lives of individuals and families that it is intended to serve. Failure to collect these data could reduce effective use of program resources to benefit adolescents and families.

The cross-site evaluation involves three data collection points—a baseline and two follow-up surveys for adolescents pregnant at baseline (6 months after birth and 12 months after birth) and a follow-up survey at one year after baseline for adolescents parenting at baseline. Serious consideration has been given to the issue of how frequently to survey respondents for the cross-site evaluation. After consulting with a committee of AFL project staff and young adult clients, an expert workgroup, and other project staff, it was determined that the data collection strategy selected would need to be sufficient in number to track and document changes in outcomes between and across individuals before exposure to a time point late enough for intervention effects to be observed on occurrence of repeat pregnancy among Care respondents. In addition, adolescents may experience several developmental changes as they experience peer pressure and opportunities to engage in sexual activity. Thus, it is important to measure attitudes, behavior, and risk and protective factors for these at several time points in order to account for changes that may occur because of adolescents’ developmental progression. Less frequent data collection would not allow for measurement of immediate program effects and long-term effects. Because of concerns about respondent attrition due to possible dropping out of the study, RTI determined that the follow-up intervals would need to be narrow enough to enable completion of survey cycles with a given individual over a reasonably short period of time.

7. Special Circumstances Relating to the Guidelines of 5CFR 1320.5

There are no special circumstances that require data collection to be conducted in a manner inconsistent with 5 CRF 1320.5(d)(2).

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A 60-day Federal Register Notice was published in the Federal Register on June 9, 2008, in Volume 73, Number 111, page 32583 and provided a 60-day period for public comments (Appendix C). There were no public comments.

A list of consultants on this project is provided in Exhibit 2. Consultants contacted included AFL project staff, AFL young adult clients and former clients (including two pregnant and/or parenting young adults), and expert researchers with a background in adolescent reproductive health and program evaluation. The information provided from these discussions was extremely helpful in informing RTI staff about suggested improvements to the core instruments for all grantees, as well as the expected reactions of adolescents who will participate in the evaluations (and of parents of unemancipated adolescents aged 17 or younger, who will need to provide consent for adolescent participation). This information helped guide the development of both the instruments and cross-site evaluation study design. Input and recommendations were incorporated into the survey and questionnaire design to the extent possible. Contact information for the consultants for this project is provided in Exhibit 2.

RTI staff consulted with respondent surrogates in connection with pre-tests of the survey instruments (which are currently approved instruments under the collection OMB 0990-0290) (as described in Section B.4). A total of 52 self-administered Care baseline questionnaires were completed by pregnant adolescent Care clients, 55 baseline questionnaires were completed by parenting adolescent Care clients, and 41 follow-up questionnaires were completed by Care clients. Refinements to the surveys were made as a direct result of these pretests.

9. Explanation of Any Payment or Gift to Respondents

For individual project evaluations that involve the core instruments, payments to respondents may be provided if they are necessary to facilitate participation and the decision to provide payments will be left to each project. 

For the cross-site evaluation, a $10 gift card incentive will be offered to participants who complete the baseline and follow-up surveys. Pregnant and parenting adolescents will also be offered a raffle opportunity to win an MP3 player ($140 value). One MP3 player will be awarded per Care program, as an incentive for adolescents to provide good contact information at baseline so that they can be reached at follow-up. Only respondents who can be contacted at 12-month follow-up (regardless of whether they complete the follow-up survey) will be eligible to win. Pregnant and parenting adolescent non-respondents to the follow-up in-person survey administration will be offered mailed surveys with a “teaser” incentive (trinket or information card worth about $1.25) with the promise of a mailed gift card when the completed survey is returned. Non-responders to this mail survey will be offered data collection via telephone using an adapted instrument that allows youth to respond to questions without disclosing information that could be understood by a bystander in the room (e.g., most responses would be yes/no). This approach has been used in other RTI studies, including the Parent Corps evaluation approved by OMB in 2004. Non-responders who are not able to be interviewed by telephone will also be offered in-field data collection at their place of residence or another private location of their choice and a $20 gift card incentive.

Exhibit 2. Persons Consulted Outside the Agency

Expert Work Group
Elaine Borawski, Ph.D., Director, Center for Health Promotion Research Case Western Reserve University 216.368.1024 elaine.borawski@case.edu	Jeff Tanner, Ph.D., Associate Dean Baylor University 254.710.3485 Jeff_Tanner@baylor.edu
Claire Brindis, Dr.P.H. Professor of Pediatrics and Health Policy Associate Director, Institute for Health Policy Studies Center for Reproductive Health Research and Policy University of California at San Francisco 415.476.5255 claire.brindis@ucsf.edu	Lynne Tingle, Ph.D., Assistant Professor University of North Carolina at Greensboro 336.334.3435 lrtingle@uncg.edu
Douglas Kirby, Ph.D., Senior Research Scientist ETR Associates 831.438.4060 dougk@etr.org	Gina Wingood, Sc.D., Associate Professor and Director, Behavioral and Social Science Core Center for AIDS Research Emory University 404.727.0241 gwingoo@sph.emory.edu
Lisa Lieberman, Ph.D., President CHES Healthy Concepts, Inc. 845.638.1619 LLHealth@optonline.net	Meredith Kelsey, Ph.D., Research and Policy Analyst Division of Children and Youth Policy Office of the Assistant Secretary for Planning and Evaluation 202-690-6652 meredith.kelsey@hhs.gov
Dennis McBride, Ph.D., Associate Director for Research The Washington Institute for Mental Illness Research and Training University of Washington 253.756.2335 dmcb@u.washington.edu	Lisa Trivits, Ph.D., Research and Policy Analyst Division of Children and Youth Policy Office of the Assistant Secretary for Planning and Evaluation 202-205-5750 Lisa.trivits@hhs.gov
Amy Ong Tsui, Ph.D., Director and Professor Bloomberg School of Public Health Johns Hopkins University 410-955-2232 atsui@jhsph.edu
Staff Committee
Anne Badgley, M.Ed., President and CEO Heritage Community Services 843-863-0508 ABadgley@heritageservices.org	David MacPhee, Ph.D., Professor Human Development & Family Studies Colorado State University 970-491-5503 macphee@CAHS.Colostate.edu
Leisa Bishop, Director of Neighborhood Services BETA Center, Inc., Project FAME 407-277-1942 ext. 134 lbishop@betacenter.org	Janet Mapp, Interim Director of Prevention Services Switchboard of Miami 305-358-1640 jmapp@switchboardmiami.org
Doreen Brown, Director of Outreach Services Healthy Connections 479-243-0279 hci_mena@sbcglobal.net	Dr. Ruben Martinez, Ph.D., Evaluator Decisions For Life of Baptist Child and Family Services 210-458-2654 Decisions4life@aol.com; Rmartinez@utsa.edu
Carl Christopher, Educator St. Vincent Mercy Family Care Center 419-251-2341 carl_christopher@mhsnr.org	Mary Lou McCloud, Director of Young Parents Support Services YWCA 585-368-2248 mmccloud@ywcarochester.org
Cheri Christopher, Young Adult Representative St. Vincent Mercy Family Care Center 419-251-2341 carl_christopher@mhsnr.org	Charnese McPherson, Young Adult Representative 202-305-0384
Audra Cummings, Young Adult Representative 479-216-0842	Alice Skenandore, Executive Director Wise Women Gathering Place 920-490-0627 wwgp@new.rr.com
Christina Diaz, Program Director Decisions For Life of Baptist Child and Family Services 210-240-8866 Decisions4life@aol.com; cdiaz@bcfs.net	Jared Stangenberg, Young Adult Representative 615-683-7106 Mrpigeonman@yahoo.com
Amy Lewin, Psy.D., Assistant Professor Center for Health Services and Community Research Healthy Generations Program Children’s National Medical Center 202-884-3106 alewin@cnmc.org	Cherie Wooden, R.N., BSN Program Manager Helping Our Parents to be Educators (HOPE) 607-584-4485 cwooden@lourdes.com

Adolescents are a difficult cohort to recruit for a 20-minute survey about this sensitive topic without the use of a small incentive, and pregnant and parenting adolescents are extremely transient. The incentives are intended to recognize the time burden placed on adolescents, encourage their cooperation, and to convey appreciation for contributing to this important study. Numerous empirical studies have shown that incentives can significantly increase response rates (e.g., Abreu & Winters, 1999; Shettle & Mooney, 1999; Singer et al., 1999). The decision to use incentives for this study is based on several projects conducted by RTI and others, which found that use of $10 to $20 incentives, raffles, and teaser incentives increased response rates among adolescents and other populations similar to the proposed study population. Exhibits 3-6 summarize several such studies and the response rates achieved. Although these studies differ in other respects that could account for some variability in response rates, overall, incentives of at least $10 were generally associated with higher response rates compared with no incentive.

Exhibit 3. Studies Involving Child and Adolescent Respondents Receiving $10 Incentives and Corresponding Response Rates

Study	Population	Incentive Provided	Response Rate Achieved
Healthy Schools/Healthy Communities (2002)	Adolescents aged 12 to 17 years	$10 gift certificate and a baseball cap or calculator	68%
Georgia Health and Behavior Study (2002)	Persons aged 9 to 17 years	$10 cash for each of two interviews	76% first interview 84% second interview
National Survey on Child and Adolescent Well-Being (2002)	Children aged 6 to 10 years	$10 gift certificate for 25-minute interview	85%
The University of California Irving Stress and Trauma Study (2001–2004)	Adolescents aged 13 to 17	$10 initial incentive Pool A, $10 initial incentive + $10 completion incentive Pool B	83%: Pool A 79%: Pool B

Exhibit 4. Studies Involving Adolescent Respondents Receiving Raffle Incentives and Corresponding Response Rates

Study	Population	Incentive Provided	Response Rate Achieved
Pregnancy Risk Assessment Monitoring System (PRAMS)	New mothers (including adolescents)	Participation in a raffle for a cash award	≥70%
Evaluation of abstinence-based pregnancy prevention program (Project IMPPACT)	7th and 8th grade students	Enrolled in a raffle for a CD player for providing complete follow up contact information at post-test	75%
Intervention trial to reduce adolescent smoking (Project C.R.A.S.H.)	Adolescents aged 14 to 17	Participation in a $200 prize drawing	63%

Exhibit 5. Studies Involving Adolescent Respondents Receiving Teaser Incentives and Corresponding Response Rates

Study	Population	Incentive Provided	Response Rate Achieved
Healthy Schools/Healthy Communities (2002)	Adolescents aged 12 to 17	$10 gift certificate and a baseball cap or calculator	68%
Pregnancy Risk Assessment Monitoring System (PRAMS)	New mothers (including adolescents)	Coupons for certified birth certificates, pre-paid telephone cards, bibs, cash (a dollar bill), and/or magnetic picture frames	≥70%
Intervention trial to reduce adolescent smoking (Project C.R.A.S.H.)	Adolescents aged 14 to 17	$2 token	69%

Exhibit 6. Studies Involving Respondents Receiving $20 Incentives and Corresponding Response Rates

Study	Population	Incentive Provided	Response Rate Achieved
National Survey on Child and Adolescent Well-Being (2002)	Adolescents aged 11 to 18	$20 gift certificate	87%
RAND Adolescent/Young Adult Panel Study	Young adults aged 23 to 24 who had participated in the survey yearly from grades 7 through 12	$20 cash after return of survey $25 cash after return of survey $20 cash mailed with survey	59% 66% 62%
Family Adolescent Risk Behavior and Communication Study (FARBCS)	Adolescents aged 14 to 16	$25 cash	87%

The use of modest incentives is expected to enhance survey response rates without biasing responses or coercing respondents to participate. A smaller incentive would not appear sufficiently attractive to adolescents. We also believe that the incentives will result in higher data validity as adolescents become more engaged in the survey process. The amount of the incentives was determined through discussions with RTI staff and expert work group members with expertise in conducting adolescent surveys about reproductive health issues. Because all selected adolescents may not be eligible for the study, we want to assure efficient project spending and only provide substantial incentives to respondents after they are determined to be eligible.

10. Protection of Data Security and Participant Privacy

For individual AFL project evaluations, specific procedures for data collection privacy and security are site specific. Each AFL applicant, however, must submit a signed acceptance of assurances required by Title XX of the Public Health Service Act. These assurances include affirmation that a system for maintaining the security of client records is in place. Compliance is monitored by OPA staff. Unless grantees have obtained a Certificate of Confidentiality, respondents are not assured that their data are confidential.

All procedures for the cross-site evaluation have been developed, in accordance with federal, state, and local guidelines, to ensure that the rights, privacy, and information security of respondents are protected and maintained. The RTI Institutional Review Board (IRB) reviewed all instruments, informed consent and assent materials, and procedures to ensure that the rights of individuals participating in the study are safeguarded. A copy of the RTI IRB approval notice is included as Appendix D. A pilot test of these procedures was conducted, and no problems were identified (see Section B.4 for a summary of the pilot test). RTI will apply for a Certificate of Confidentiality so that respondents can be assured that their information is confidential.

All respondents will be assured that the information they provide will be kept private and will be used only for the purpose of this research. A copy of this assurance provided in writing to respondents is presented in Appendix E. Respondents will be assured that their answers will not be shared with family members and that their names will not be reported with responses provided. Respondents will be told that the information obtained from all of the surveys will be combined into a summary report so that details of individual questionnaires cannot be linked to a specific participant.

All AFL projects will submit the questionnaire to their site IRB (and their Health Insurance Portability and Accountability Act [HIPAA] Privacy Board if the site is a Covered Entity) prior to initiating data collection. The questionnaire data will be treated as private and maintained in a manner that satisfies the privacy requirements set forth by the site IRB (and HIPAA Privacy Board if the site is a Covered Entity).

It is possible that another adolescent could view survey responses while survey administration is in progress, so adolescents will be spaced out around the room when the survey is administered to more than one adolescent at a time, in order to avoid the possibility of another respondent being able to view survey responses. After completion of the survey, respondents will place questionnaires in an envelope. AFL staff will seal the envelope, and it will not be unsealed in the presence of respondents. Only evaluation staff will have access to survey information provided by individual respondents.

To ensure data security, all RTI and AFL project staff are required to adhere to strict standards and to sign agreements as a condition of employment on the cross-site evaluation. Survey administrators will be thoroughly educated in methods of maximizing parent and adolescent understanding of the government’s commitment to privacy. Hard-copy data collection forms will be delivered to a locked area for receipt and processing. Individual identifying information will be kept separate from survey responses, and ID numbers will be assigned to participants for identification purposes. RTI and AFL project staff will never leave completed consent/assent forms or questionnaires unattended. All completed consent/assents forms and the list of participant names and ID numbers will be stored in separate locked filing cabinets only accessible to authorized study personnel. Survey responses will be stored on a secure, password-protected computer shared drive. RTI maintains restricted access to all data preparation areas (i.e., receipt, coding, and data entry). All data files on multi-user systems will be under the control of a database manager, with access limited to project staff on a “need-to-know” basis only. Transmission and collection protocols will be in accordance with requirements set forth by RTI’s IRB and the site IRB (and HIPAA Privacy Board if the site is a Covered Entity). No respondent identifiers will be contained in reports, and results will only present data in aggregate form.

We will seek approval and review by the OS Privacy Act Coordinator, Maggie Blackwell.

11. Justification for Sensitive Questions

The major foci of the AFL care demonstration projects are to provide demonstration projects that develop and test innovative approaches to services for pregnant and parenting adolescents. Demonstration project curricula and other materials cover issues around adolescent sexual activity, contraceptive use, and repeat pregnancy prevention. Thus, some questions included in the revised core evaluation instruments might be considered sensitive by some respondents. Exhibit 7 identifies the sensitive questions, explains the justification for their inclusion in the surveys, and describes how the data will be used in the cross-site evaluation. The informed consent and assent protocol apprises respondents that these topics will be covered during survey administration. These questions are included in the surveys because of their importance in understanding adolescent behaviors surrounding pregnancy prevention and the moderating effect of adolescents’ pre-intervention risk characteristics on the main effects of the AFL program on adolescent repeat pregnancy; educational attainment after birth; and compliance with child immunization schedules. As with all information collected, these data will be presented with all identifiers removed.

Exhibit 7. Description of Sensitive Questions, Justification for Inclusion, and Use of Data

Description of Questions	Justification for Inclusion	Use of Data
Sexual activity	Necessary to determine whether changes in sexual activity/abstinence explain effects of the AFL program on repeat pregnancy	Used as mediating variable for multivariate analysis to assess changes in sexual activity as the pathway of program effects on adolescent repeat pregnancy
Method(s) to prevent pregnancy	Necessary to determine whether changes in contraceptive behavior explain effects of the AFL program on repeat pregnancy	Used as mediating variable for multivariate analysis to assess changes in contraceptive behavior as the pathway of program effects on adolescent repeat pregnancy
Health care	Necessary to validate self-report responses about repeat pregnancy	Items about whether the respondent has received prenatal care or an abortion will be used to validate self-report responses about repeat pregnancy, which will be used as dependent variable for multivariate analysis comparing treatment and comparison adolescents

12. Burden Estimate (Total Hours & Wages)

The annual response burden is 11,030. This burden is the total response burden per year for all AFL projects, since the cross-site evaluation involves a subset of respondents completing core evaluation instruments. Exhibit 8 provides details about how the burden estimate was calculated. AFL survey respondents will be comprised of adolescents aged 9 to 19 who are selected by AFL Care demonstration projects to participate in treatment or comparison groups. Additionally, grantee staff will complete the last page of the baseline survey for parenting adolescents and of the follow up survey for each adolescent respondent. The paper-and-pencil self-administered surveys will be designed to maximize ease of response and thus decrease respondent burden. The annual respondent cost is $66,180 (Exhibit 9). This cost is the total respondent cost per year for all AFL projects, since the cross-site evaluation involves a subset of respondents completing core evaluation instruments. Respondents participate on a purely voluntary basis and, therefore, are subject to no direct costs other than time to participate; there are no start-up or maintenance costs. Timings were conducted during our pilot test procedures to determine the overall burden per respondent for the core instruments during the cross-site evaluation. Paper and pencil data collection is expected to take 23 minutes per adolescent respondent, and completion of the last page of the instrument is expected to take 4 minutes for staff to complete for each adolescent respondent. Because it is not known what the wage rate category will be for these selected adolescents (or even whether they will be employed at all) or for grantee staff, the figure of $6.00 per hour was used as an estimate of average minimum wage across the country.

Exhibit 8. Estimated Annualized Burden Hours

Type of Respondent	Form Name	No. of Respondents	No. of Responses/ Respondent	Average Burden/ Response (Hours)	Total Burden (Hours)
Adolescents aged 9 to 19	Baseline survey for pregnant adolescents	10,000	1	23/60	3833
	Baseline survey for parenting adolescents	3,000	1	23/60	1150
	Follow-up survey	13,000	1	23/60	4983
Grantee staff	Baseline survey for parenting adolescents	155	19	4/60	196
	Follow-up survey	155	84	4/60	868
	TOTAL				11,030

Exhibit 9. Estimated Annualized Cost to Respondents

Type of Respondent	Form Name	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Adolescents aged 9 to 19	Baseline survey for pregnant adolescents	3833	$6.00**	$22,998
	Baseline survey for parenting adolescents	1150	$6.00**	$6,900
	Follow-up survey	4983	$6.00**	$29,898
Grantee staff	Baseline survey for parenting adolescents	196	$6.00**	$1,176
	Follow-up survey	868	$6.00**	$5,208
	TOTAL	11,030	$6.00**	$66,180

**Estimates of average hourly living allowance for participants

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

Respondents will incur no capital or maintenance costs.

14. Annualized Cost to the Federal Government

There are no costs to the federal government associated with grantee use of the revised core evaluation instruments. The cost estimate for the completion of the cross-site evaluation will be $1,944,778 over 4 years, including a possible one-year no-cost extension for the project. This total cost covers all cross-site evaluation activities and includes information collection and other cross-site evaluation tasks not included in this OMB application. This includes the estimated cost of coordination with the OPA and AFL projects; project plan and schedule development; RTI IRB applications; overseeing of data collection; analysis; reporting; and progress reporting. Annual cost to the federal government is estimated to be $486,195 ($1,944,778/4).

15. Explanation of Program Changes or Adjustments

A previous OMB application was approved in 2005 for the core evaluation instruments to be used among AFL projects (0990-0290); thus this is not a new collection, although that collection expires 9/30/2008. There are no increases in burden requested because the grantees are already conducting baseline and follow-up data collection as part of the grant funding requirements. Furthermore, we are not creating new documents and not increasing sample size for this data collection. In fact, the instruments have been shortened from a 30-minute administration to a 23-minute administration, reducing burden.

16. Plans for Tabulation and Publication and Project Time Schedule

The OPA requires AFL demonstration projects to provide tabulations of data on basic demographics and selected questions in the core evaluation instruments in their end-of-year reports (OMB 0990-0299). These aggregated data are used to track progress on the performance measures in response to OMB’s recommendation. 

Analysis of the data for the statutorily required independent evaluation of each project will vary, and be determined, by the individual grantees and their evaluators. 

Analyses for the cross-site evaluation will consist of two phases: (1) analyses of baseline data and (2) longitudinal analyses that include multiple waves of survey data.

Baseline Analyses

Baseline analyses will begin once baseline data are available and will consist of definition of the analysis variables (individual variables, compounded scales, latent groups), descriptive statistics (frequencies, means), and basic tests of differences between treatment groups and grantees (cross-tabulations, Student’s t tests, and chi-square tests). This will include single time point tests of association between hypothesized independent, mediating, moderating, and dependent variables (as conceptualized in Exhibit 10).

Initial data quality analysis will consist of analysis of nonresponse, study dropout, and missingness patterns in the data as well as preparation of the analysis scales and measures. Although we will implement ongoing data quality control, at the end of the study we will examine the overall quality of the data, which will include but not be limited to identifying patterns of nonresponse and data missingness and examining characteristics of the sample to uncover sources of potential biases. For example, each individual risk-related item might have a small number of missing records, but when many items are combined to create a scale, the cumulative number of observations with missing data could be significant. In cases of missing or inconsistent data, we will decide on the best way to correct the data (imputation methods, reassessing the subject records, etc.).

Exhibit 10. Logic Model for the Cross-Site Evaluation of the AFL Program

 

Though expected to be minimized, particularly with the instrument reconfiguration, missingness due to item nonresponse or invalid responses will be appropriately imputed using Proc MI (multiple imputations), a procedure available in SAS (release 9.1). Compared with the more typical single imputation, this procedure offers the advantage of providing valid statistical analyses that properly reflect the uncertainty due to missing data. We will also conduct checks for outlying values and identify the best way to deal with them. We will construct the actual measures that will be used in the study, such as involvement of the baby’s father and educational attainment. Finally, we will prepare an analytic data set that will include all studied variables and measures.

We will start baseline analysis with the examination of the psychometric properties of the data, with the intention of developing reliable scales that accurately capture attitudes, beliefs, and behaviors targeted by the AFL program. Measures may be examined for appropriateness across various cultural groups, and scalar equivalence may be assessed using procedures described in Knight, Virdin, & Roosa (1994). Basic tests of association will examine the relationships between the demographic characteristics, pre-program risk levels, and outcomes at baseline to create a reference to which these characteristics will be compared at the longitudinal time points. When the construction of a compound scale is not justifiable, we will use structural equation modeling (SEM), which combines a number of variables with similar meaning into latent class variables. These latent class variables could then be related to each other and/or other variables according to the conceptual model of mediating and moderating effects. However, we will use SEM for a simpler analysis of baseline differences between the groups of variables forming major moderating, mediating, and outcome categories. Finally, SEM will be used to evaluate the multivariate relationships of baseline demographics, mediators, and outcomes to attrition from baseline to follow-up. Variables found to differ between follow-up survey responders and nonresponders will be addressed by adjusting sampling weights to the demographic composition and size of grantee populations (gleaned from end-of-year reports) or by controlling for these variables in multivariate analyses.

To test difference among treatment group variables at baseline, we will use a multilevel (hierarchical) linear model (HLM):

Dependent variable_it = ß₀ + ß₁(treatment status) +W_j+ e_ijt, (1)

where indexes i and j correspond to a subject and grantee, Wj is a grantee-specific random effect, and ß corresponds to regression parameters.

For testing the relationship between baseline outcome variables and targeted risk and protective factors, the hierarchical model will become

Outcome_it = ß₀ + ß₁(treatment status)+ ß₂(risk or protective factor)+ W_j+ e_ijt. (2)

Longitudinal Analyses

Main longitudinal analyses will be addressed by using a repeated measures regression model that controls for the baseline values and data collection time point. An interaction term between time point and treatment status will be included to estimate which program characteristics are particularly effective at which time points. Random effects will be included to account for correlations within subject (i) and within project sites (j). The nature of multilevel hierarchical analysis is to account for clustering of responses within the same individual and the same grantee (i.e., individuals within a site are more similar than individuals across sites, and individuals’ responses are correlated across time). This clustering of individuals within grantees and of responses within individuals is estimated through the variance of the random effects.

The variables for tested hypotheses will also include binary variables for which we will use a multilevel logistic regression model. To adjust for adolescents’ ages and other demographics, we will modify Equation 1 by adding the corresponding terms. We will present adjusted and unadjusted results.

Analyses of Moderating and Mediating Effects

Moderation of program effects at the organizational and individual levels will be examined to assess whether program effectiveness depends on demographic characteristics and baseline levels of the targeted mediators (i.e., baseline by treatment interaction effects) and contextual variables (e.g., prior pregnancies). To test the moderating effects of individual demographic characteristics and pre-program risk levels, corresponding covariates and their interactions with the intervention effects will be added to the longitudinal regression models described above (Baron & Kenny, 1986; Judd & Kenny, 1981).

To test the potential mediating effects of risk and protective factors, such as attitudes about future education, corresponding interaction terms (e.g., time x attitudes about future education) will be added to the regression models described above. Then, we will conduct two more regression analyses by adding the mediators as covariates to the first regression model so it becomes the following:

Outcome_it = b₀ + b₁(treatment status)_it + ... + U_i +W_j + g(attitudes about future education)_it + e_it. (3)

Attitudes about future education _it = a₀+a₁(treatment status)_it + U_i +W_j + e_it (4)

The test for mediation will consist of testing for the product ga₁ (Sobel, 1982). The procedures for estimating mediated effects in the context of multilevel analysis, outlined in Krull and MacKinnon (1999), will be used for parameter estimation. Mediation analyses at the organizational and individual levels will be conducted using the Asymmetric Confidence Interval method (MacKinnon, Taborga, & Morgan-Lopez, 2002), a state-of-the-art method of estimating confidence intervals for mediated effects. Mediational effects will be considered at all follow-up points and will be accounted for by including interaction terms for each time point. Multiple waves of data from respondents will be advantageous to these analyses because potential mechanisms of effect can be examined at intermediate time points, rather than concurrently with predictors or outcomes, which strengthens the evidence for causal associations.

Use of Sampling Weights

In conducting the analyses, we will use statistical weights that provide the projection of the sample estimates to the entire population of adolescents targeted by grantees participating in the cross-site outcome evaluation. Sampling or design weights for each unit observed are computed as the inverse of the probability of selection at each stage of selection. Typically, adjustments to these sampling weights are necessary to account for bias related to sample selection, nonresponse and/or attrition, and deficiencies in population coverage. In recent years, RTI statisticians have developed the generalized exponential model (GEM), a response propensity modeling approach for computing nonresponse and coverage adjustments (Folsom & Singh, 2000). The Folsom and Singh modeling approach is a generalization of constrained logistic models first suggested by Deville and Särndal (1992). This approach has been shown to be more effective than the more commonly used weighting-class approach in correcting for nonresponse bias. The increase in effectiveness comes from the ability to incorporate a greater number of correlates of nonresponse in the modeling approach than would be possible with traditional weighting-class methods and to allow for controlling the variability of the adjustments, which in turn decreases the variance of the resultant.

Exploratory Effect Size Analysis

Based on our expert workgroup’s recommendation from Stage 2, an additional approach for the outcome evaluation, which would involve calculating the effect sizes of individual projects, will also be used (DeCoster, 2004; Egger & Smith, 1997; Singleton & Straits, 1999). Effect sizes will be computed in Cohen’s D metric, or standardized mean differences (Cohen, 1988). Cohen’s D values will be calculated both for comparisons of preprogram versus postprogram means for each group of adolescents participating in an AFL project, as well as for comparisons between adolescents receiving AFL project services and those not receiving AFL project services. Separate Cohen’s D values will be computed for each form of comparison. Cohen’s D values will be calculated from means and standard deviations calculated from the data set for each grantee. Each effect size will be weighted by the inverse of its variance to provide more efficient estimation of true population effects (Hedges & Olkin, 1985). Findings for both unweighted and weighted effect sizes will be reported in analyses of overall program effects. Only weighted Cohen’s D values will be analyzed and reported in analyses of moderator variables. All effect sizes will be coded such that positive values indicate differences in directions consistent with a favorable effect of the AFL program.

For the additional meta-analytic approach, each project will be coded on characteristics (based on process evaluation data), such as project features (e.g., project goal, geographic location, setting in which project activities occurred, monitoring of implementation, characteristics of project staff, project staff training, inclusion of parental involvement component), characteristics of participating adolescents (gender, race/ethnicity, age, family structure, socioeconomic background, at-risk status), project dosage and exposure (actual frequency of project contact, average length), and timing of assessment. Efforts will be directed towards at least preliminarily determining whether different types (content, themes, and modes) of projects have different outcomes. The independent sample will be the primary unit of analysis. Each project will contribute one independent sample to the analysis.

The analytic strategies described above will provide an optimal design for assessing the overall impacts of the AFL program on adolescent outcomes and will allow for examination of differences in effects as a function of individual characteristics. The study hypotheses for the cross-site evaluation are outlined in Exhibit 11. Additionally, Exhibit 12 summarizes each of our planned analyses using baseline and multiple waves of follow-up data.

As the evaluation questions and hypotheses are addressed, the findings will be summarized and shared with OPA and OPA-identified stakeholders for comment and interpretation. For this study, we expect the findings to be disseminated to a number of audiences. Therefore, the evaluation reports will be written in a way that emphasizes scientific rigor for more technical audiences but is also intuitive, easily understood, and relevant to less technical audiences. The reporting and dissemination mechanism will consist of three primary components: (1) a baseline sample profile, (2) a data summary including preliminary results of follow up data, and (3) peer-reviewed journal articles. The baseline sample profile report will offer descriptive information about adolescent evaluation participant characteristics at baseline, comparisons between these data and national data about similar populations, and comparisons between treatment and comparison groups. The data summary will include data from the baseline and follow-up data collections, including follow-up response rates and characteristics of adolescents who participated in baseline and follow-up data collection, with preliminary findings from cross-tabulations, multivariable regression models, and exploratory and confirmatory factor analyses. This report will also include preliminary data on adolescents’ intermediate outcomes (attitude and belief changes). The results of our study also will be used to develop at least one peer-reviewed journal article (e.g., American Journal of Public Health, Perspectives on Sexual and Reproductive Health, Journal of Research on Adolescence, or Prevention Science) that summarizes findings on the overall effectiveness of the AFL program. With review and approval by OPA, the results of the evaluation will also be used to develop at least one conference presentation.

Exhibit 11. Study Hypotheses

The primary study hypotheses concern the effects of AFL Care projects on repeat pregnancy, educational attainment, and compliance with recommended child immunizations schedules.

Exposure to AFL Care projects will result in:

• Reduced repeat pregnancy

• Improved educational attainment

• Improved compliance with recommended infant immunization schedules

Additionally, Exhibit 10 identifies several secondary hypotheses, which represent relationships between mediating and moderating variables in the model, and the interaction between these variables, AFL program exposure, and outcomes:

• Involvement of the baby’s father mediates the relationship between program exposure and repeat pregnancy, educational attainment, and immunizations;

• Attitudes and beliefs about future education mediate the relationship between program exposure and repeat pregnancy as well as educational attainment;

• Involvement of the teen’s parents mediates the relationship between program exposure and repeat pregnancy, educational attainment, and immunizations;

• Sexual activity and contraceptive use mediate the relationship between program exposure and repeat pregnancy;

• Pre-program levels of risk, including prior reproductive health behavior and educational attainment, moderate the relationship between program exposure and the primary outcomes;

• Demographic characteristics moderate the relationship between program exposure and the primary outcomes.

The key events and reports to be prepared are listed in Exhibit 13.

17. Expiration Date

The OMB expiration date will be displayed on all data collection instruments.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

Exhibit 12. Cross-Site Evaluation Analyses

Time	Research Question/Hypothesis	Methods
Baseline	What are the sample characteristics of study participants?	Adjusted1 means and frequencies
	How are mediating/moderating variables associated with presumed outcomes?	Adjusted1 cross-tabulations, analysis of variance, and correlations
	How are mediating and moderating variables correlated?
	What are the psychometric properties of the survey data?	Exploratory and confirmatory factor analyses
	What are the differences between treatment and comparison groups at baseline?	Multilevel logistic and linear regression
Baseline through final Follow-up	Are there different patterns of change over time in continuous (scaled) outcomes as a function of exposure to AFL Care programs?	Multilevel growth curve modeling
	Are there different rates of dichotomous outcomes as a function of exposure to AFL Care programs?	Multilevel logistic regression
	What is the relationship between exposure conditions and mediating variables?	Multilevel logistic and linear regression
	What is the relationship between mediating variables and program outcomes?
	What are the pathways among exposure conditions, mediators, and outcomes?
	What is the impact of moderating variables on the relationship between exposure conditions and change over time in continuous outcomes?	Multilevel growth curve modeling
	What is the impact of moderating variables on the relationship between exposure conditions and rates of dichotomous outcomes?	Multilevel logistic regression
	What variables are linked to attrition?	Multilevel logistic regression

¹ Analyses will be adjusted for clustering of individuals within organizations and programs.

B. Collection of Information Employing Statistical Methods

Statistical methods are not used in the collection of information for all AFL demonstration projects using the revised core evaluation instruments; therefore, responses to this section apply only to the methods used for the cross-site evaluation of the AFL program.

1. Respondent Universe and Sampling Methods

The cross-site evaluation (which will be a subset of the projects and respondents to the survey) will include approximately 972 pregnant and parenting adolescents. Adolescents served by Title XX Care projects and those selected to serve as comparison groups will participate in the cross-site evaluation.

Exhibit 13. Time Schedule for the Entire Project

Task/Activity	Start Date	End Date
Start date	September 26, 2007	---
Develop project plan and schedule	September 26, 2007	August 12, 2008
Design instruments	September 26, 2007	August 8, 2008
Pilot test instruments	October 1, 2007	January 31, 2008
Main study data collection preparation activities	March 1, 2008	September 30, 2008
Collect baseline data	October 1, 2008	November 24, 2009
Collect follow-up data	April 1, 2009	July 24, 2011a
Analyze data	December 1, 2009	May 15, 2012 a
Submit baseline sample profile		January 31, 2010
Submit data summary		September 30, 2011a
Submit manuscript and conference presentation		June 1, 2012 a

^a This estimated timeline includes a possible no-cost extension for the project.

A total of 31 Care projects serve pregnant and parenting adolescents and their families. From these projects, 11 Care projects have been selected to obtain the sample of 972 participants for the cross-site evaluation. Care projects were selected for participation based on the rigor of their evaluation designs, namely those that have equivalent treatment and comparison groups and that avoid contamination by the intervention of comparison group respondents. We also prioritized projects that are located in different geographic regions in order to maximize regional diversity and projects that employ implementation strategies conducive to rigorous evaluation (including appropriate timing of program delivery). Information about evaluation design rigor, implementation strategies, and project characteristics was obtained by reviewing end-of-year reports submitted to OAPP and through discussions with OAPP project officers. Within each project, adolescents will be assigned by AFL project staff to treatment and comparison groups.

We conducted power analyses to determine the optimal sample size for detecting statistically significant differences between treatment and comparison groups. Among Care projects, the frequency with which adolescents report repeat pregnancy, that they are in compliance with infant immunization schedules, and their educational attainment in terms of rates of school dropout serve as the primary outcome measures. Power calculations were based on the comparison between treatment and comparison groups for reports of repeat pregnancy, which is the focal outcome measure and provided sample size estimates that were attainable given the numbers of programs with rigorous designs that can be included in the evaluation. Several assumptions were made concerning population parameters for power analyses. First, we assumed that all outcomes between different respondents will be uncorrelated. (Siblings or adolescents living in the same household as an enrolled study participant will be excluded.) The exception to this is that because adolescents clustered within Care projects, a program-level intraclass correlation of 0.06 was assumed, based on pilot data analyses of this variable. Second, it was assumed that 11 percent of treatment adolescents and 24 percent of comparison adolescents will experience a repeat pregnancy within 1 year of their first baby’s birth, as reported by Black and colleagues (2006). However, Black and colleagues’ (2006) follow up assessment was 24 months after birth, which may have yielded stronger effects than a 12-month follow up given that adolescents in the comparison group had more time to experience a second pregnancy. On the other hand, a longer follow up period may have allowed program effects to weaken over time. Therefore it is unclear whether our evaluation will yield stronger or weaker effects than prior research. Our assumption allows us to include enough subjects in our evaluation to detect small effects, and making a less conservative assumption would create the possibility that the Care project interventions are efficacious but that our sample size is not large enough to detect this.

To achieve 0.80 power, preliminary power analyses indicate that a total of 972 adolescents from 8 grantee sites will need to complete the baseline survey. The numbers of adolescents in the respondent universe and in each sample are shown in Exhibit 14. The expected response rate at the 12-month follow-up includes all adolescents who participate at baseline, including those who may refuse to participate in the 6-month follow-up data collection.

All decisions about assumptions that guided our power analysis were intended to err in favor of a larger sample size to safeguard for the possibility of a worst case scenario in terms of difficulty detecting effects. These assumptions increased our confidence that smaller effects produced by Care projects than those found by previous programs would be reasonably detected using the sample sizes we identified.

Exhibit 14. Longitudinal Response Rates and Numbers of Adolescents

Numbers and Response Rates	Treatment Adolescents	Comparison Adolescents	Total
Number of subjects to be contacted at baseline	684	684	1,368
Expected parent consent rate	74%	74%
Number of subjects with parent consent at baseline	506	506	1012
Expected response rate at baseline	96%	96%
Number of completed baseline surveys	486	486	972
Expected response rate 6 months after birth	87%	68%
Number of completed follow-up surveys 6 months after birth	423	330	753*
Expected response rate 12 months after birth/baseline	72%	72%
Number of completed follow-up surveys 12 months after birth/baseline	350	350	700*

*A subset of the original 972 baseline respondents.

As noted, our sample design is based on conservative assumptions about survey response. Thus, our estimates of longitudinal retention rates shown in Exhibit 14 should be viewed as “worst case” scenarios that if hold true, would still ensure sufficient sample sizes to reasonably detect small program effects. For Care, we estimate that at least 96% of adolescents who are contacted and for whom parent consent is obtained will complete the baseline survey, that at least 87% of treatment adolescents and 68% of comparison adolescents will be retained between the baseline and 6-month follow-up survey, and that at least 72% of adolescents will be retained between the baseline and 12-month follow-up surveys.

Exhibit 15 shows longitudinal retention rates for prior studies of various lengths.

Exhibit 15. Longitudinal Completion and Retention Rates for Prior Studies

Project	Institution/ Client	Sample	Survey	Time from Baseline	Follow-up Survey Completion Rate	Baseline to Follow-up Retention Rate
Evaluation of abstinence-based pregnancy prevention program (Project IMPPACT)	Inwood House/U.S. Department of Health and Human Services	7th and 8th grade students	Paper and pencil questionnaire	2 years	75%	59%
Child and Family Well-being Study (The Three Cities Study)	Johns Hopkins University/ National Institute for Child Health and Human Development	Focal children of poor households	Physical measurements and a CAPI/ ACASI questionnaire	Wave 2: 1.5 years Wave 3: 6 years	82%	80%

It should be noted that while attrition will inevitably occur in this study, as it usually does in any longitudinal study, we do not expect attrition to bias any of the study’s main findings. In sample surveys, there will almost always be missing data due to the attrition (or initial nonresponse) of selected respondents. In longitudinal surveys, this problem is typically exacerbated as a function of time because there may be further attrition at each wave of the survey. Three distinct mechanisms causing missing data can be identified and the cause of missingness determines the extent to which bias may be introduced into the study estimates. These mechanisms include the following:

Data are said to be missing completely at random (MCAR) if the probability of attrition is unrelated to study outcome variables or to the value of any other explanatory variables, including the exposure conditions. No additional bias will be introduced to estimates based on incomplete data due to missingness under MCAR. However, the reduced data set will typically result in larger standard errors.

Data are said to be missing at random (MAR) if the probability of attrition is unrelated to study outcome variables after controlling for other explanatory variables. That is, attrition may vary by demographic characteristics. For example, adolescents of lower income may be more likely to drop out of the survey compared to adolescents of higher income. Thus bias would be introduced into an overall outcome variable estimate for adolescents but not into income-specific estimates. Thus, under MAR, the potential bias in estimates due to missingness can be eliminated (or reduced significantly) if the appropriate explanatory variables, such as income, are controlled for.

Data are said to be missing not at random (MNAR) if the probability of attrition is related to the study outcome variable itself. For example, suppose that adolescents who indicate that they are not in school at baseline are more likely to drop out of the survey than adolescents who report that they are attending school. In this case, the overall estimate of educational attainment among all adolescents will be biased upward by attrition.

In practice, all three missingness mechanisms may be at work (i.e., different attriters may drop out according to different mechanisms). If MNAR is not dominant, then reasonably unbiased estimates of study outcomes can be constructed through appropriate modeling. In the case of this study, we do not expect MNAR to be present.

2. Procedures for the Collection of Information

To gather sensitive and complex data for the cross-site impact evaluation, AFL demonstration project evaluation staff will administer paper and pencil Teleform surveys with treatment and comparison adolescents. Staff will also conduct a record abstraction to complete the last page of the baseline survey for parenting adolescents and of the follow up survey for each adolescent.

In order for unemancipated adolescents aged 17 or younger to be included in the cross-site evaluation sample, their parents must be able to read English or Spanish to provide active consent for their adolescent’s participation (either in writing or by telephone with mailed documentation), and all adolescents must be able to read English or Spanish to provide written consent or assent for their own participation in the study. Consent forms and assent forms are included in Appendix E. If adolescent respondents are emancipated either in court or by statute in their state of residence (e.g. due to marriage), then they will provide consent for their own participation in evaluation activities.

Some AFL Care projects serve adolescents for whom parent consent is a particular challenge because their parents are absent, unreachable, or the parent’s/guardian’s interests may not adequately reflect the child’s interests (e.g., child abuse or neglect). Data collection and project staff will be trained to assess whether or not parent consent can be obtained. If it cannot be obtained, then a designated site staff person will be asked to serve as an advisor to the adolescent during the administration of the informed consent. For an individual to serve as a youth advisor for an unaccompanied minor, he or she must be neutral to the study, sensitive to the needs of the participant, and able to understand and answer questions about the study. It is anticipated that a site staff person who is involved in program delivery but not in program evaluation will meet these criteria. The youth advisor will be asked to be present during the administration of the informed consent, to assist the adolescent in deciding whether or not to participate in the study, and to sign the consent form as a witness. The consent form for unaccompanied minors is included in Appendix E. In these situations, a waiver for parent consent will be sought from RTI’s IRB.

All AFL sites will submit the survey instruments to their site IRB (and HIPAA Privacy Board if the site is a Covered Entity) prior to initiating data collection. Copies of local site IRB approvals will be submitted to RTI’s IRB. The questionnaire data will be treated as private and maintained in a manner that satisfies the privacy requirements set forth by the site IRB (and HIPAA Privacy Board if the site is a Covered Entity). Any and all transmission of individual or case level data will also be done in accordance with privacy requirements set forth by their site IRB (and HIPAA Privacy Board if the site is a Covered Entity).

Data collection training, monitoring, and ongoing technical assistance will be provided for projects participating in the cross-site evaluation in order to assure high quality data collection procedures. All AFL project staff administering core evaluation instruments will be trained in survey administration, including consent and assent procedures, privacy guidelines, and identifying respondent distress. In addition, the training will emphasize to AFL project staff the importance of following the data collection procedures, including mailing procedures, in order to ensure that the rationale for data collection procedures is fully understood by those responsible for data collection.

Data collection staff will be encouraged to avoid reading all questions to groups of respondents if possible in order to avoid adolescents looking at each others’ survey responses. Completed instruments will be sealed in envelopes, and project staff will not unseal envelopes containing completed surveys in the presence of respondents. The lists of identifiers and identification numbers will be sent to RTI for safekeeping during the cross-site evaluation. Standard procedures will be developed for identification number assignment and linking for the cross-site impact evaluation, with exceptions made if necessary.

Data collection staff will ask parenting adolescents to bring their child’s immunization record to the survey administration. Staff will complete a simple record abstraction, using the information from the immunization record to complete six items on the last page of the survey after the adolescent has completed the survey and is no longer present.

Cross-site evaluation baseline data will be collected by Care grantees from October 2008 through November 2009.

For the cross-site evaluation, adolescents will also receive a $10 gift card incentive for baseline data collection because adolescents are a difficult cohort to recruit for a 20-minute survey without the use of a small incentive. The decision to use incentives for this study is based on previous findings in the literature (Abreu & Winters, 1999; Shettle & Mooney, 1999; Singer et al., 1999) and by studies that incentives can significantly increase response rates among adolescents. Although these studies differ in other respects that could account for some variability in response rates, overall, incentives of at least $10 were generally associated with higher response rates compared with no incentive. It is expected that these modest incentives will enhance survey response rates without biasing responses or coercing respondents to participate, as well as higher data validity as adolescents become more engaged in the survey process. Because incentives are geographically and culturally specific, this standardized value will be offered, but individual grantees will determine what is actually provided. A protocol for standardized incentives for the cross-site impact evaluation will be suggested. Additional explanation regarding the use of incentives in this study is provided in Section A9.

Treatment and comparison group adolescents who completed the baseline surveys when pregnant will be surveyed again at 6 and 12 months postpartum, until July 2011 for the cross-site evaluation (to allow adolescents in the very earliest stage of pregnancy who completed a baseline instrument at the end of the baseline data collection period time to deliver and complete a follow-up survey 12 months after birth). Adolescents who completed the baseline surveys when parenting will be surveyed again 12 months after baseline. A potential threat to the external validity of the proposed longitudinal design is loss to follow-up or attrition (Biglan et al., 1991). In other words, the results of the evaluation may be different among the group of subjects who remain in the study after baseline from those who do not remain in the study after baseline. Potential attrition may be an important consideration in the selection of adolescents, particularly because grantees frequently recruit clients located in areas with high levels of transience and hard-to-reach populations (such as low-income families without telephones). RTI’s experience suggests that by using mail surveys and tracing and locating services and by obtaining extensive locating information from participants at baseline (i.e., cell phone, e-mail, contact information for family or friends), it becomes more likely to successfully survey at follow-up 80% of respondents who completed baseline interviews. Baseline respondents will be offered a raffle opportunity to win on MP3 player ($140 value) per Care program, with the understanding that only respondents who can be contacted at 12-month follow-up (regardless of whether they complete the follow-up survey) will be eligible to win. Questionnaires will be mailed to non-responders at follow-up, along with a “teaser” incentive (trinket or information card worth about $1.25) with the promise of a mailed gift card when the completed survey is returned. Non-responders to this mail survey will be offered data collection via telephone using an adapted instrument that allows youth to respond to questions without disclosing information that could be understood by a bystander in the room (e.g., most responses would be yes/no). This approach has been used in other RTI studies, including the Parent Corps evaluation approved by OMB in 2004. Non-responders who are not able to be interviewed by telephone will also be offered in-field data collection at their place of residence or another private location of their choice. To enhance survey response rates, a $20 gift card incentive will be provided to non-respondents to the mail survey to complete field interviewer data collection. This second incentive for follow-up data collection is needed in order to assure sufficient follow-up response rates and high data validity so that comparisons between baseline and follow-up can be analyzed to yield rigorous impact evaluation results.

All questionnaire hard copies and electronic data will be stored in a secure area designated by the site IRB and HIPAA procedures. AFL project staff will store completed parent consent and adolescent consent/assent forms in separate locked filing cabinets. Since many Care programs administer evaluation instruments on a rolling basis, they will store completed instruments and signed consent and assent forms in separate secured locations, and ship them via Federal Express to the RTI project director for the cross-site evaluation on the 15^th/16^th and 29^th/30^th of each month, sending completed instruments and signed assent/consent forms on separate days. This regular shipping schedule should serve to minimize burden for programs who conduct data collection with individuals or very small groups on a rolling basis. No respondent names will be included in the Federal Express package of completed instruments. Assent/consent forms and completed surveys must be shipped to RTI separately and on different days. RTI will be notified and provided a tracking number for each shipment. If shipments do not arrive as scheduled, tracing will immediately be initiated through Federal Express. This process will be monitored and feedback provided to AFL project staff throughout the data collection period. If needed, AFL project staff may be re-trained regarding mailing procedures.

3. Methods to Maximize Response Rates and Deal with Nonresponse

The following procedures will be used to maximize cooperation and to achieve the desired high response rates for the cross-site evaluation:

A $10 gift card will be offered to participants who complete the baseline survey. An additional $10 gift card for each follow-up survey will be offered to participants who complete the follow-up survey at 6 months after birth and 12 months after birth (or 12 months after baseline).

An attempt will be made to locate participants who leave the study before the end of the cross-site evaluation. Location efforts will include mailings of refusal conversion materials designed to persuade participants to complete the study. In addition to using mailed refusal conversion materials, the evaluation contractor may also conduct telephone-based refusal conversion, contacting each attriting participant via telephone.

For nonresponders at follow-up, a postage-paid business reply envelope will be included with each mail survey, along with a “teaser” incentive (trinket or information card worth about $1.25) with the promise of a mailed gift card when the completed survey is returned. Baseline respondents will be offered a raffle opportunity to win an MP3 player ($140 value), with the understanding that only respondents who can be contacted at 12-month follow-up (regardless of whether they complete the follow-up survey) will be eligible to win. Field data collectors will also track nonresponders at follow-up. Non-responders to the mail survey will be offered data collection via telephone or in-field data collection at their place of residence or another private location of their choice. Field interviewers will give respondents $20 gift cards for completing the survey in person.

RTI and AFL grantees will provide a toll-free telephone number to all sampled individuals and invite them to call with any questions or concerns about any aspect of the study.

AFL grantee data collection staff will work with RTI project staff to address concerns that may arise.

4. Tests of Procedures or Methods to be Undertaken

RTI implemented pilot tests of the core evaluation instruments previously approved by OMB (OMB 0990-0290) with 148 youths involved in various AFL Care demonstration projects across the nation. The purpose of the pilot tests was twofold: (1) to assess technical aspects and functionality of the survey instrument and (2) to identify areas of the survey that were either unclear or difficult to understand. In addition, another purpose of the Care pilot test was to test the feasibility and ease of the data collection protocol.

Pilot test data collection was conducted from September 2007 through March 2008. Eligible participants came from a convenience sample of clients aged 9 to19 recruited through various AFL demonstration projects. At the time of the pilot test, respondents were either service recipients or comparison group participants. Parents who expressed interest for their children aged 17 or younger to participate in the study received a lead letter from local survey administrators. A screener conducted with parents or adolescents aged 18 or older was used to determine study eligibility of participants. Respondents self-administered one instrument each. Instruments were completed at 6 project sites under the supervision of Care project survey administration staff. A total of 52 baseline instruments for pregnant adolescents, 55 baseline instruments for parenting adolescents, and 41 follow-up survey instruments were completed, including questions regarding attitudes and beliefs about future education, parenting skills and activities, repeat pregnancies, child immunizations, reproductive health, social support, and demographics. Five participants completed the instruments in Spanish.

Analyses of the pilot test data indicated that respondents had a difficult time following the skip patterns in these surveys. For instance, 25% of respondents who completed the baseline survey for parenting teens or the follow-up survey completed both of the questions about contraceptive methods, even though the first is designed for respondents who are pregnant and the second is designed for respondents who are not pregnant. RTI has changed the formatting of the skip patterns to make the directions more visual, and has added instructions about conditions under which particular questions should be answered in order to better guide respondents through the survey. RTI has also removed questions that repeat in the surveys due to skip patterns, in order to decrease burden and increase the likelihood of accurate responding. Many of the respondents in the pilot study put check marks in the boxes instead of filling them in. RTI has replaced the response boxes with circles to increase the likelihood that responses will be accurately scanned. Some respondents were unsure about what to answer for their race. RTI has created an additional response option for “other (describe ______________)” for race.

Some respondents stated that they did not understand the term “spouse.” RTI has changed the term “spouse” to “husband.” There were no outlier values, and all response options were labeled correctly. There were no non-response problems with the survey except for a substantial amount of missing data on questions assessing household composition, sources of financial aid, and reproductive health services received, all of which were in “yes/no” format. RTI has changed the response options for these items to a “mark all that apply” format. There was also a substantial amount of missing data on questions assessing infant immunizations, and several adolescents reported not knowing whether their child had received immunizations. RTI has supplemented these items with a form for program staff to complete using the child’s immunization record. Among the 80 respondents for whom time to complete was recorded, the average length of the survey was 23 minutes, with a range of 7 to 45 minutes.

Based on the findings of the pilot test, the survey appears to function as intended and is not overly burdensome, sensitive, or difficult to understand. Therefore, few substantive revisions were made to the survey instrument as a result of pilot testing.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The agency official responsible for receiving and approving contract deliverables is:

Johanna Nestor
240-453-2808
jnestor@osophs.dhhs.gov
Office of Population Affairs/DHHS
1101 Wotton Parkway, Suite 700
Rockville, MD 20852

The person who designed the data collection is:

Olivia S. Ashley, Dr.P.H.
919-541-6427
osilber@rti.org
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709

The person who will collect the data is:

Karen Morgan, Ph.,D.

(919) 485-7779

kcmorgan@rti.org

RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709

The persons who will analyze the data are:

Georgiy Bobashev, Ph.D.
919-541-6167
bobashev@rti.org
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709

Michael Penne, M.S.
919-541-5988
penne@rti.org
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709

Marni Kan, Ph.D.

919-485-2756

mkan@rti.org

RTI International

3040 Cornwallis Road
Research Triangle Park, NC 27709

References

Abma, J., Martinez, G., Mosher, W., & Dawson, B. (2004). Teenagers in the United States: Sexual activity, contraceptive use, and childbearing, 2002. Vital and Health Statistics, Series 23, No 24. Hyattsville, MD: National Center for Health Statistics.

Abreu, D. A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. In Proceedings of the Survey Research Methods Section of the American Statistical Association. http://www.amstat.org/ sections/SRMS/proceedings/. Last updated on May 24, 2007.

Albert, B., Lippman, L., Franzetta, K., Ikramullah, E., Keith, J. D., Shwalb, R., et al. (2005). Freeze frame: A snapshot of America’s teens. Washington, DC: National Campaign to Prevent Teen Pregnancy.

Amin, R., & Sato, T. (2004). Impact of a school-based comprehensive program for pregnant teens on their contraceptive use, future contraceptive intention, and desire for more children. Journal of Community Health Nursing, 21, 39-47.

Barnet, B., Liu, J., Devoe, M., Alperovitz-Bichell, K., & Duggan, A. K. (2007). Home visiting for adolescent mother: Effects on parenting, maternal life course, and primary care linkage. Annals of Family Medicine, 5, 224-232.

Baytop, C. (2006). Evaluating the effectiveness of programs to improve educational attainment of unwed African American teen mothers: A meta analysis. The Journal of Negro Education, 75, 458-477.

Biglan, A., Hood, D., Brozovsky, P., Ochs, L., Ary, D., & Black, C. (1991). Subject attrition in prevention research. In. W. Bukoski, & K. Leukefeld (Eds.), Drug abuse prevention research: Methodological issues. NIDA Research Monograph (Vol. 107, pp. 213-223). Rockville, MD: National Institute on Drug Abuse.

Black, M., Bentley, M. E., Papas, M. A., Oberlander, S. A., Teti, L. O., McNary, S., Le, K., & O’Connell, M. (2006). Delaying second births among adolescent mothers: A randomized, controlled trial of a home-based mentoring program. Pediatrics, 18, 2005-2318.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Earlbaum.

Corcoran, J., & Pillai, V. K. (2007). Effectiveness of secondary pregnancy prevention programs: A meta-analysis. Research on Social Work Practice, 17, 5-18.

DeCoster, J. (2004). Meta-analysis. In K. Kempf-Leonard (Ed.), The encyclopedia of social measurement (pp. 1-19). San Diego, CA: Academic Press.

Eaton, D. K., Kann, L., Kinchen, S., Ross, J., Harris, W. A., Lowry, R., McManus, T., Chyen, D., Shanklin, S., Lim, C., Grunbaum, J. A., & Wechsler, H. (2006). Youth Risk Behavior Surveillance—United States, 2005. Morbidity & Mortality Weekly Report, 55, SS-5, 1-108.

Egger, M., & Smith, G. D. (1997). Meta-analysis: Potential and promise. British Medical Journal, 315, 1371-1374.

Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.

Hedges, L. V., & Vevea, J. L. (1998). Fixed and random effects models in meta-analysis. Psychological Methods, 3, 486-504.

Henry J. Kaiser Family Foundation. (2003). National Survey of Adolescents and Young Adults: Sexual health knowledge, attitudes and experiences. Menlo Park, CA: Author.

Hoyer, P. J. (1998). Prenatal and parenting programs for adolescent mothers. Annual Review of Nursing Research, 16, 221-249.

Kirby, D. (2002). Do abstinence-only programs delay the initiation of sex among young people and reduce teen pregnancy? Washington, DC: National Campaign to Prevent Teen Pregnancy.

Knight, G. P., Virdin, L. M., & Roosa, M. (1994). Socialization and family correlates of mental health outcomes among Hispanic and Anglo American children: Consideration of cross-ethnic scalar equivalence. Child Development, 65, 212-224.

Krull, J. L., & MacKinnon, D. P. (1999). Multi-level mediation modeling of group-based intervention studies. Evaluation Review, 23, 418-444.

MacKinnon, D. P., Taborga, M. P., & Morgan-Lopez, A. A. (2002). Mediation designs for tobacco prevention research. Drug and Alcohol Dependence, 68, S69-S83.

The National Campaign to Prevent Teen Pregnancy. (2003). With one voice 2003: America's adults and teens sound off about teen pregnancy. Washington, DC: The National Campaign to Prevent Teen Pregnancy.

The National Longitudinal Study of Adolescent Health. (1998). Waves I & II, 1994–1996. Chapel Hill, NC: Carolina Population Center, University of North Carolina at Chapel Hill. [Need to verify reference]

O’Rourke, D., Chapa-Resendez, G., Hamilton, L., Lind, K., Owens, L., & Parker, V. (1998). An inquiry into declining RDD response rates part I: Telephone survey practices. Survey Research, 29, 1-16.

Percy, M. S., & McIntyre, L. (2001). Using Touchpoints to promote parental self-competence in low income, minority, pregnant, and parenting teen mothers. Journal of Pediatric Nursing, 16, 180-186.

Seitz, V., & Apfel, N. H. (1999). Effective interventions for adolescent mothers. Clinical Psychology: Science and Practice, 6, 50-66.

Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S. government surveys. Journal of Official Statistics, 15, 231-250.

Singer, E., Van Hoewyk, J., Gebler, N., Raghunathan, T., & McGonagle, K. (1999). The effect of incentives in interviewer-mediated surveys. Journal of Official Statistics, 15, 217-230.

Singleton, R., & Straits, B. C. (1999). Approaches to social research. New York: Oxford University Press.

Thomas, D. V., & Looney, S. W. (2004). Effectiveness of a comprehensive psychoeducational intervention with pregnant and parenting adolescents: A pilot study. Journal of Child and Adolescent Psychiatric Nursing, 17, 66-77.

U.S. Government Accountability Office. (2006). Abstinence education: Efforts to assess the accuracy and effectiveness of federally funded programs. Washington, DC: Author.

The White House. (2005). Program Assessment Rating Tool: 2006 budget. http://www.whitehouse.gov/omb/budget/fy2006/sheets/part.xls. Last updated July 23, 2005.

Public Law 98-512, 42 U.S.C. 300z-2, as amended.

Appendix A

Statute/Regulation Mandating or Authorizing the Collection of Information

TITLE XX -- ADOLESCENT FAMILY LIFE DEMONSTRATION PROJECTS

§2001. [ 300z] Findings and purposes

(a) The Congress finds that -

(1) in 1978, an estimated one million one hundred thousand teenagers became pregnant, more than five hundred thousand teenagers carried their babies to term, and over one-half of the babies born to such teenagers were born out of wedlock;

(2) adolescents aged seventeen and younger accounted for more than one-half of the out of wedlock births to teenagers;

(3) in a high proportion of cases, the pregnant adolescent is herself the product of an unmarried parenthood during adolescence and is continuing the pattern in her own lifestyle;

(4) it is estimated that approximately 80 per centum of unmarried teenagers who carry their pregnancies to term live with their families before and during their pregnancy and remain with their families after the birth of the child;

(5) pregnancy and childbirth among unmarried adolescents, particularly young adolescents, often results in severe adverse health, social, and economic consequences including: a higher percentage of pregnancy and childbirth complications; a higher incidence of low birth weight babies; a higher infant mortality and morbidity; a greater likelihood that an adolescent marriage will end in divorce; a decreased likelihood of completing schooling; and higher risks of unemployment and welfare dependency; and therefore, education, training, and job research services are important for adolescent parents;

(6) (A) adoption is a positive option for unmarried pregnant adolescents who are unwilling or unable to care for their children since adoption is a means of providing permanent families for such children from available approved couples who are unable or have difficulty in conceiving or carrying children of their own to term; and

(B) at present, only 4 per centum of unmarried pregnant adolescents who carry their babies to term enter into an adoption plan or arrange for their babies to be cared for by relatives or friends;

(7) an unmarried adolescent who becomes pregnant once is likely to experience recurrent pregnancies and childbearing, with increased risks;

(8) (A) the problems of adolescent premarital sexual relations, pregnancy, and parenthood are multiple and complex and are frequently associated with or are a cause of other troublesome situations in the family; and

(B) such problems are best approached through a variety of integrated and essential services provided to adolescents and their families by other family members, religious and charitable organizations, voluntary associations, and other groups in the private sector as well as services provided by publicly sponsored initiatives;

(9) a wide array of educational, health, and supportive services are not available to adolescents with such problems or to their families, or when available frequently are fragmented and thus are of limited effectiveness in discouraging adolescent premarital sexual relations and the consequences of such relations;

(10)(A) prevention of adolescent sexual activity and adolescent pregnancy depends primarily upon developing strong family values and close family ties, and since the family is the basic social unit in which the values and attitudes of adolescents concerning sexuality and pregnancy are formed, programs designed to deal with issues of sexuality and pregnancy will be successful to the extent that such programs encourage and sustain the role of the family in dealing with adolescent sexual activity and adolescent pregnancy;

(B) Federal policy therefore should encourage the development of appropriate health, educational, and social services where such services are now lacking or inadequate, and the better coordination of existing services where they are available; and

(C) services encouraged by the Federal Government should promote the involvement of parents with their adolescent children, and should emphasize the provision of support by other family members, religious and charitable organizations, voluntary associations, and other groups in the private sector in order to help adolescents and their families deal with complex issues of adolescent premarital sexual relations and the consequences of such relations; and

(11)(A) there has been limited research concerning the societal causes and consequences of adolescent pregnancy;

(B) there is limited knowledge concerning which means of intervention are effective in mediating or eliminating adolescent premarital sexual relations and adolescent pregnancy; and

(C) it is necessary to expand and strengthen such knowledge in order to develop an array of approaches to solving the problems of adolescent premarital sexual relations and adolescent pregnancy in both urban and rural settings.

(b) Therefore, the purposes of this subchapter are -

(1) to find effective means, within the context of the family, of reaching adolescents before they become sexually active in order to maximize the guidance and support available to adolescents from parents and other family members, and to promote self discipline and other prudent approaches to the problem of adolescent premarital sexual relations, including adolescent pregnancy;

(2) to promote adoption as an alternative for adolescent parents;

(3) to establish innovative, comprehensive, and integrated approaches to the delivery of care services both for pregnant adolescents, with primary emphasis on unmarried adolescents who are seventeen years of age or under, and for adolescent parents, which shall be based upon an assessment of existing programs and, where appropriate, upon efforts to establish better coordination, integration, and linkages among such existing programs in order to -

(A) enable pregnant adolescents to obtain proper care and assist pregnant adolescents and adolescent parents to become productive independent contributors to family and community life; and

(B) assist families of adolescents to understand and resolve the societal causes which are associated with adolescent pregnancy;

(4) to encourage and support research projects and demonstration projects concerning the societal causes and consequences of adolescent premarital sexual relations, contraceptive use, pregnancy, and child rearing;

(5) to support evaluative research to identify effective services which alleviate, eliminate, or resolve any negative consequences of adolescent premarital sexual relations and adolescent childbearing for the parents, the child, and their families; and

(6) to encourage and provide for the dissemination of results, findings, and information from programs and research projects relating to adolescent premarital sexual relations, pregnancy, and parenthood.

Appendix B

Cross-Site Evaluation Data Collection Materials

Care Core Baseline Questionnaire for Pregnant Teens (English/Spanish)

Care Core Baseline Questionnaire for Parenting Teens (English/Spanish)

Care Core Follow Up Questionnaire (English/Spanish)

[INSERT INSTRUMENTS HERE]

Appendix C

Federal Register Notice to the Public

[INSERT PDF]

Appendix D

RTI Institutional Review Board Approval Notice

[INSERT PDF]

Appendix E

Assurances of Confidentiality and Study Descriptions Provided to Respondents

• Care Survey Parent/Guardian Informed Consent for Unemancipated Youths Younger than Age 18

• Care Survey Assent for Unemancipated Youths Younger than Age 18

• Care Survey Consent for Youths Aged 18 and Older and Emancipated Minors

• Care Survey Consent for Unaccompanied Youths Younger than Age 18

• Youth Assent Script for Unemancipated Care Youths Younger Than Age 18

• Youth Consent Script for Care Youths Aged 18 or Older and Emancipated Minors

• Youth Consent Script for Care Unaccompanied Youths

• Youth Advisor Script for Care Unaccompanied Youths

Adolescent Family Life (AFL) Care Survey

Parent/Guardian Informed Consent for Unemancipated Youths Younger than Age 18

Protocol Title: AFL Care Core Evaluation Instruments

Sponsor: Department of Health and Human Services

Office of Population Affairs

AFL Care Program Director: NAME

Directions for parents: Please read this information. If all of your questions are answered and you agree to allow your child to try this survey, mail the last page of this consent form with your signature and your child’s full name to [Name of AFL Care Program Director]. Please keep the other pages for your records.

Introduction

We are inviting your child to be part of a research study to evaluate [PROGRAM NAME] as part of our involvement with the Office of Population Affairs, Adolescent Family Life Care Program. Your child was selected because of her involvement with [PROGRAM NAME]. This information will be used to help improve programs like ours. Before you decide whether you want your child to take part in this study, you need to read this Informed Consent form so that you understand what the study is about and what your child will be asked to do. This form also tells you who can be in the study, the risks and benefits of the study, how we will protect your information, and who you can call if you have questions. Please call Dr. Olivia Ashley, the researcher responsible for this study, at (800) 334-8571 ext. 6427 (a toll-free number) about anything you don’t understand before you make your decision.

Purpose

This study sponsored by the Office of Population Affairs (OPA), Department of Health and Human Services (DHHS), is being conducted by RTI International, a research organization located in North Carolina. This national study will involve more than 900 youths. The purpose of this study is to learn about youths who are served by programs like [PROGRAM NAME].

Procedures

If you agree to let your child participate, she will be asked to complete a questionnaire. The questions ask about things like their pregnancy, future goals, sexual activity, method(s) to prevent pregnancy and sexually transmitted diseases, health care, baby, parenting, and the baby’s father. Most of the questions are multiple choice. This is not a test. There are no right or wrong answers. If your child prefers, we can read the questions to her.

If your child is a parent, she will be asked to bring her child’s immunization records to the survey administrations. She will also be asked to write down the month and the year that her child was born. (If she has more than one child, she will be asked about her youngest child.) This information will be kept private. A staff member will also record information about the child’s immunizations.

Study Duration

Completing the questionnaire will take about 20 minutes of your child’s time. If your child is pregnant now, then there will be another survey six months and 12 months after her baby is born. If your child is not pregnant now, there will only be one additional survey, next year. Each of these surveys will take about 20 minutes of your youth’s time.

Possible Risks or Discomforts

Some of the questions may seem personal or make your child feel uncomfortable. There is a risk that your child’s answers could be seen by someone else other than the project staff, which could create problems among friends or teachers, but we promise to do our best to keep this from happening. In addition to the risks and discomforts listed here, there may be uncommon or previously unknown risks. You should report any problems to Dr. Olivia Ashley at (800) 334-8571 ext. 6427 (a toll-free number).

Benefits

Your Benefits

There are no direct benefits to your child from participating in this study. However, the survey could help service providers learn about ways to improve your youth’s services.

Benefits for Other People

We hope that this research will help us understand how to improve programs for youths who are pregnant and parenting.

Payment for Participation

Your child will receive a $10.00 gift card for trying any part of the questionnaire. She will receive a $10.00 gift card each time she takes a follow-up questionnaire. Your child may also be offered a raffle opportunity to win an MP-3 player at the end of the study. Only youths who can be reached at final follow-up will be eligible to win, regardless of whether they complete the survey.

Privacy

All the questionnaire answers will be kept private. We will not allow anyone outside the program evaluation staff know which answers are your child’s, except when required by law. There are two exceptions: 1) if your child reveals that she is a danger to self or others, or 2) if she reveals abuse or neglect committed against a child. In either of these cases, we must report it to the appropriate authorities. This includes suspected abuse or neglect of your child or of a friend of your child, or suspected abuse or neglect by your child. We may want to share the results of the study with other people who worked on the survey and the funding agency, but no names will be included. Your child’s name will be replaced with a number for the purposes of this study. After all surveys are completed, a summary will be written that contains information from all participants, but no names. It will not be possible to determine who wrote what on a questionnaire.

The Institutional Review Board (IRB) at RTI has reviewed this research. An IRB is a group of people who are responsible for assuring that the rights of participants in research are protected. The IRB may review the records of your child’s participation in this research to assure that proper procedures were followed. A representative of the IRB may contact you for information about your child’s experience with this research. This representative will be given your name, but will not be given any of your child’s private study data. If you wish, you may refuse to answer any questions this person may ask.

Future Contacts

We will contact your child again when it is time for her to take a follow-up survey. As part of your child’s participation in [PROGRAM NAME], your child may be contacted by [PROGRAM NAME], but not about this study.

Your Rights

Your child’s decision to take part in this research study is completely voluntary. You do not have to agree to allow your child to take the questionnaire in order for him or her to get services here or anywhere else. Your child will also be asked if he or she is willing to voluntarily participate in the study. In order for your child to complete the questionnaire, BOTH you and your child must agree to participation. If your child does participate in the study, he or she can skip any questions. If your child feels like the questionnaire is taking too long, gets tired, or if for any other reason he or she wants to stop, they may do so at any time.

Your Questions

If you have any questions about this study, you can contact the AFL Program Project Director, [PROGRAM DIRECTOR], at [PROGRAM NAME] at [LOCAL NUMBER] or Dr. Olivia Ashley at RTI at (800) 334-8571 ext. 6427 (a toll-free number). If you have any questions about protecting your privacy on this survey, please call [LOCAL IRB LIASION NAME] at [LOCAL NUMBER]. If you have any questions about your rights as a study participant, you may call RTI’s Office of Research Protection at 1-866-214-2043 (a toll-free number).

RTI ID:

KEEP PAGES 1-3 FOR YOUR RECORDS.

COMPLETE AND RETURN THIS FORM TO [NAME OF AFL PROGRAM DATA COLLECTOR].

By signing this form, you are letting us know that you have read it, received answers to your questions, and freely decided to allow your child to try this survey. Signing this form will not affect your child receiving services here or anywhere else.

_______________________

Your Child’s Name

_______________________ _________ _________________________ ______

Parent/Guardian Signature Date Parent/Guardian Printed Name Date

_______________________ __________ __________________________ ______

Service Provider’s Signature Date Service Provider’s Printed Name Date

Adolescent Family Life (AFL) Care Survey Assent for Unemancipated Youths Younger than Age 18

Protocol Title: AFL Care Core Evaluation Instruments

Sponsor: Department of Health and Human Services

Office of Population Affairs

AFL Care Program Director: NAME

Directions: Please read this form. If all of your questions are answered and you agree to try this survey, sign the last page of this form and return to [Name of AFL Care Program Data Collector]. Please keep the other pages.

Introduction

We are inviting you to be part of a research study about [PROGRAM NAME]. You were chosen because you are part of [PROGRAM NAME]. This information will be used to help improve programs like ours. Before you decide whether you want to take part in this study, you need to read this form so that you know what the study is about and what you will be asked to do. This form also tells you who can be in the study, the risks and benefits of the study, how we will protect your privacy, and who you can call if you have questions. Please call Dr. Olivia Ashley, who is in charge of this study, at (800) 334-8571 ext. 6427 (a toll-free number) about anything you want to ask before you decide.

Purpose

This study is sponsored by the Office of Population Affairs (OPA), Department of Health and Human Services (DHHS). It is being done by RTI International, a research firm in North Carolina. This national study will involve more than 900 youths. The purpose of this study is to learn about youths served by programs like [PROGRAM NAME].

Procedures

If you agree to take part, you will be asked to take a survey. The questions ask about things like your pregnancy, future goals, sexual activity, ways to avoid pregnancy and diseases people can get when having sex, health care, baby, parenting, and the baby’s father. Most of the questions are multiple choice. This is not a test. There are no right or wrong answers. If you prefer, we can read the questions to you.

If you are a parent, you will be asked to bring your child’s immunization records to the survey administrations. You will also be asked to write down the month and the year that your child was born. (If you have more than one child, you will be asked about your youngest child.) This answer will be kept private. A staff member will also record information about your baby’s immunizations on this survey.

Study Duration

Completing the questionnaire will take about 20 minutes of your time. If you are pregnant now, then there will be another survey six months and 12 months after your baby is born. If you are not pregnant now, there will only be one additional survey, next year. Each of these surveys will take about 20 minutes.

Risks or Discomforts That May Happen

Some of the questions may seem personal or bother you. There is a risk that your answers could be seen by someone other than the project staff. This could create problems among friends or teachers, but we promise to do our best to prevent this.

In addition to these risks and discomforts, other risks may happen that are not common or that we don’t expect. You should report any problems to Dr. Olivia Ashley at (800) 334-8571 ext. 6427 (a toll-free number).

Benefits

Your Benefits

There are no direct benefits to you from taking part in this study. However, the survey could help staff learn about ways to improve your services.

Benefits for Other People

We hope that this research will help us improve programs like [PROGRAM NAME.]

Payment for Taking Part

You will receive a $10.00 gift card for trying any part of the survey. You will receive a $10.00 gift card each time you take a follow-up survey. You may also be offered a raffle opportunity to win an MP-3 player at the end of the study. Only youths who can be reached at final follow-up will be eligible to win, regardless of whether they complete the survey.

Privacy

All the survey answers are private. We will not allow people outside the study staff know which answers are yours except when required by law. There are two reasons we would do this: 1) if you reveal that you are a danger to yourself or others or 2) if you reveal a child is being hurt or not taken care of. In either of these cases, we must report it to the authorities. This includes if you are being hurt or not taken care of, if a friend of yours is being hurt or not taken care of, or if you are hurting or not taking care of a child you are responsible for. We will combine your answers with those of other youths. We may share these results with other people who worked on the survey and the funding group, but we will not share names. Your name will be replaced with a number for the purposes of this study. After all surveys are done, we will write a summary that contains answers from all youths. The staff doing the study will not use your name in the report and will keep your answers private. Readers will not be able to tell who wrote what on a survey.

The Institutional Review Board (IRB) at RTI has reviewed this research. An IRB is a group of people who must make sure that the rights of people who take part in research are protected. The IRB may review the records of your taking part in this research to make sure that proper rules were followed.

Future Contacts

We will contact you again when it is time for you to take a follow-up survey. As part of your participation in [PROGRAM NAME], you may be contacted by [PROGRAM NAME].

Your Rights

Your decision to take part in this research study is your choice. You do not have to take the survey in order for you to get services here or anywhere else. In order for you to take the survey, BOTH you and your parent/guardian must agree that you can take part. If you do take part in the study, you can skip any questions. If you feel like the survey is taking too long, you are getting tired, or if for any other reason you want to stop, you may do so at any time.

Your Questions

If you have any questions about this study, you can contact the AFL Program Project Director, [PROGRAM DIRECTOR], at [PROGRAM NAME] at [LOCAL NUMBER] or Dr. Olivia Ashley at RTI at (800) 334-8571 ext. 6427 (a toll-free number). If you have any questions about your privacy on this survey, please call [LOCAL IRB LIASION NAME] at [LOCAL NUMBER]. If you have any questions about your rights as a person taking part in the study, you may call RTI’s Office of Research Protection at 1-866-214-2043 (a toll-free number).

KEEP PAGES 1-3.

RTI ID:

FILL OUT AND GIVE THIS PAGE TO [Name of AFL Care Program Data Collector].

By signing this form, you are letting us know that you have read it, got answers to your questions, and freely decided to try this survey. Signing this form will not affect your receiving services here or anywhere else.

_______________________ ______

Youth’s Signature Date

_______________________ __________ _________________________ _____

Service Provider’s Signature Date Service Provider’s Printed Name Date

Adolescent Family Life (AFL) Care Survey

Consent for Youths Aged 18 and Older and Emancipated Minors

Protocol Title: AFL Care Core Evaluation Instruments

Sponsor: Department of Health and Human Services

Office of Population Affairs

AFL Care Program Director: NAME

Directions for youths: Please read this information. If all of your questions are answered and you agree to try this survey, sign the last page of this consent form and return to [Name of AFL Care Program Data Collector]. Please keep the other pages for your records.

Introduction

We are inviting you to be part of a research study to evaluate [PROGRAM NAME] as part of our involvement with the Office of Population Affairs, Adolescent Family Life Care Program. You were chosen because you are part of [PROGRAM NAME]. This information will be used to help improve programs like ours. Before you decide whether you want to take part in this study, you need to read this Informed Consent form so that you understand what the study is about and what you will be asked to do. This form also tells you who can be in the study, the risks and benefits of the study, how we will protect your information, and who you can call if you have questions. Please call Dr. Olivia Ashley, the researcher responsible for this study, at (800) 334-8571 ext. 6427 (a toll-free number) about anything you don’t understand before you make your decision.

Purpose

This study sponsored by the Office of Population Affairs (OPA), Department of Health and Human Services (DHHS), is being conducted by RTI International, a research organization located in North Carolina. This national study will involve more than 900 youths. The purpose of this study is to learn about youths who are served by programs like [PROGRAM NAME].

Procedures

If you agree to participate, you will be asked to complete a questionnaire. The questions ask about things like your pregnancy, future goals, sexual activity, method(s) to prevent pregnancy and sexually transmitted diseases, health care, baby, parenting, and the baby’s father. Most of the questions are multiple choice. This is not a test. There are no right or wrong answers. If you prefer, we can read the questions to you.

If you are a parent, you will be asked to bring your child’s immunization records to the survey administrations. You will also be asked to write down the month and the year that your child was born. (If you have more than one child, you will be asked about your youngest child.) This answer will be kept private. A staff member will also record information about your baby’s immunizations.

Study Duration

Completing the questionnaire will take about 20 minutes of your time. If you are pregnant now, then there will be another survey six months and 12 months after your baby is botn. If you are not pregnant now, there will only be one additional survey, next year. Each of these surveys will take about 20 minutes.

Possible Risks or Discomforts

Some of the questions may seem personal or make you feel uncomfortable. There is a risk that your answers could be seen by someone else other than the project staff, which could create problems among friends or teachers, but we promise to do our best to keep this from happening.

In addition to the risks and discomforts listed here, there may be uncommon or previously unknown risks. You should report any problems to Dr. Olivia Ashley at (800) 334-8571 ext. 6427 (a toll-free number).

Benefits

Your Benefits

There are no direct benefits to you from participating in this study. However, the survey could help service providers learn about ways to improve your services.

Benefits for Other People

We hope that this research will help us improve programs like [PROGRAM NAME].

Payment for Participation

You will receive a $10.00 gift card for trying any part of the survey. You will receive a $10.00 gift card each time you take a follow-up survey. You may also be offered a raffle opportunity to win an MP-3 player at the end of the study. Only youths who can be reached at final follow-up will be eligible to win, regardless of whether they complete the survey.

Privacy

All the questionnaire answers will be kept private. We will not allow anyone outside the program evaluation staff know which answers are yours except when required by law. There are two exceptions: 1) if you reveal that you are a danger to yourself or others, or 2) if you reveal a child is being hurt or not taken care of. In either of these cases, we must report it to the appropriate authorities. This includes if a friend of yours is being hurt or not taken care of, or if you are hurting or not taking care of a child you are responsible for. We may want to share the results of the study with other people who worked on the survey and the funding agency, but no names will be included. Your name will be replaced with a number for the purposes of this study. After all surveys are completed, a summary will be written that contains information from all participants. The staff doing the study will not use your name in the report, and will keep your answers private. It will not be possible to determine who wrote what on a questionnaire.

The Institutional Review Board (IRB) at RTI has reviewed this research. An IRB is a group of people who are responsible for assuring that the rights of participants in research are protected. The IRB may review the records of your participation in this research to assure that proper procedures were followed.

Future Contacts

We will contact you when it is time to take a follow-up survey. As part of your services with [PROGRAM NAME], [PROGRAM NAME] may contact you, but not about this study.

Your Rights

Your decision to take part in this research study is your choice. You do not have to agree to take the questionnaire in order for you to get services here or anywhere else. If you do participate in the study, you can skip any questions. If you feel like the questionnaire is taking too long, you are getting tired, or if for any other reason you want to stop, you may do so at any time.

Your Questions

If you have any questions about this study, you can contact the AFL Program Project Director, [PROGRAM DIRECTOR], at [PROGRAM NAME] at [LOCAL NUMBER] or Dr. Olivia Ashley at RTI at (800) 334-8571 ext. 6427 (a toll-free number). If you have any questions about protecting your privacy on this survey, please call [LOCAL IRB LIASION NAME] at [LOCAL NUMBER]. If you have any questions about your rights as a study participant, you may call RTI’s Office of Research Protection at 1-866-214-2043 (a toll-free number).

KEEP PAGES 1-3 FOR YOUR RECORDS.

RTI ID:

COMPLETE AND RETURN THIS PAGE TO [Name of AFL Care Program Data Collector].

By signing this form, you are letting us know that you have read it, received answers to your questions, and freely decided to try this survey. Signing this form will not affect your receiving services here or anywhere else.

_______________________ _________ _________________________ ______

Youth’s Signature Date Youth’s Printed Name Date

_______________________ __________ __________________________ ______

Service Provider’s Signature Date Service Provider’s Printed Name Date

Adolescent Family Life (AFL) Care Survey Consent

for Unaccompanied Youths Younger than Age 18

Protocol Title: AFL Care Core Evaluation Instruments

Sponsor: Department of Health and Human Services

Office of Population Affairs

AFL Care Program Director: NAME

Directions: Please read this form. If all of your questions are answered and you agree to try this survey, sign the last page of this form and return to [Name of AFL Care Program Data Collector]. Please keep the other pages.

Introduction

We are inviting you to be part of a research study about [PROGRAM NAME]. You were chosen because you are part of [PROGRAM NAME]. This information will be used to help improve programs like ours. Before you decide whether you want to take part in this study, you need to read this form so that you know what the study is about and what you will be asked to do. This form also tells you who can be in the study, the risks and benefits of the study, how we will protect your privacy, and who you can call if you have questions. Please call Dr. Olivia Ashley, who is in charge of this study, at (800) 334-8571 ext. 6427 (a toll-free number) about anything you want to ask before you decide.

Purpose

This study is sponsored by the Office of Population Affairs (OPA), Department of Health and Human Services (DHHS). It is being done by RTI International, a research firm in North Carolina. This national study will involve more than 900 youths. The purpose of this study is to learn about youths served by programs like [PROGRAM NAME].

Procedures

If you agree to take part, you will be asked to take a survey. The questions ask about things like your pregnancy, future goals, sexual activity, ways to avoid pregnancy and diseases people can get when having sex, health care, baby, parenting, and the baby’s father. Most of the questions are multiple choice. This is not a test. There are no right or wrong answers. If you prefer, we can read the questions to you.

If you are a parent, you will be asked to bring your child’s immunization records to the survey administrations. You will also be asked to write down the month and the year that your child was born. (If you have more than one child, you will be asked about your youngest child.) This answer will be kept private. A staff member will also record information about your baby’s immunizations.

Study Duration

Completing the questionnaire will take about 20 minutes of your time. If you are pregnant now, then there will be another survey six months and 12 months after your baby is born. If you are not pregnant now, there will only be one additional survey, next year. Each of these surveys will take about 20 minutes.

Risks or Discomforts That May Happen

Some of the questions may seem personal or bother you. There is a risk that your answers could be seen by someone other than the project staff. This could create problems among friends or teachers, but we promise to do our best to prevent this.

In addition to these risks and discomforts, other risks may happen that are not common or that we don’t expect. You should report any problems to Dr. Olivia Ashley at (800) 334-8571 ext. 6427 (a toll-free number).

Benefits

Your Benefits

There are no direct benefits to you from taking part in this study. However, the survey could help staff learn about ways to improve your services.

Benefits for Other People

We hope that this research will help us improve programs like [PROGRAM NAME].

Payment for Taking Part

You will receive a $10.00 gift card for trying any part of the survey. You will receive a $10.00 gift card each time you take a follow-up survey. You may also be offered a raffle opportunity to win an MP-3 player at the end of the study. Only youths who can be reached at final follow-up will be eligible to win, regardless of whether they complete the survey.

Privacy

All the survey answers are private. We will not allow people outside the study staff know which answers are yours except when required by law. There are two reasons we would do this: 1) if you reveal that you are a danger to yourself or others or 2) if you reveal a child is being hurt or not taken care of. In either of these cases, we must report it to the authorities. This includes if you are being hurt or not taken care of, if a friend of yours is being hurt or not taken care of, or if you are hurting or not taking care of a child you are responsible for. We will combine your answers with those of other youths. We may share these results with other people who worked on the survey and the funding group, but we will not share names. Your name will be replaced with a number for the purposes of this study. After all surveys are done, we will write a summary that contains answers from all youths. The staff doing the study will not use your name in the report and will keep your answers private. Readers will not be able to tell who wrote what on a survey.

The Institutional Review Board (IRB) at RTI has reviewed this research. An IRB is a group of people who must make sure that the rights of people who take part in research are protected. The IRB may review the records of your taking part in this research to make sure that proper rules were followed.

Future Contacts

We will contact you again in the future when it is time to take a follow-up survey. As part of your services with [PROGRAM NAME], [PROGRAM NAME] may contact you, but not about this pilot study.

Your Rights

Your decision to take part in this research study is your choice. You do not have to take the survey in order for you to get services here or anywhere else. If you do take part in the study, you can skip any questions. If you feel like the survey is taking too long, you are getting tired, or if for any other reason you want to stop, you may do so at any time. [PROGRAM NAME] is providing an advisor for you whose job it is to make sure you understand your rights and want to participate. This person can also answer any questions you have about this study.

Your Questions

If you have any questions about this study, you can contact the AFL Program Project Director, [PROGRAM DIRECTOR], at [PROGRAM NAME] at [LOCAL NUMBER] or Dr. Olivia Ashley at RTI at (800) 334-8571 ext. 6427 (a toll-free number). If you have any questions about your privacy on this survey, please call [LOCAL IRB LIASION NAME] at [LOCAL NUMBER]. If you have any questions about your rights as a person taking part in the study, you may call RTI’s Office of Research Protection at 1-866-214-2043 (a toll-free number).

KEEP PAGES 1-3.

RTI ID:

FILL OUT AND GIVE THIS PAGE TO [Name of AFL Care Program Data Collector].

By signing this form, you are letting us know that you have read it, got answers to your questions, and freely decided to try this survey. Signing this form will not affect your receiving services here or anywhere else.

_______________________ _________ _________________________ ______

Youth’s Signature Date Youth’s Printed Name Date

_______________________ __________ __________________________ ______

Service Provider’s Signature Date Service Provider’s Printed Name Date

_______________________ __________ __________________________ ______

Advocate’s Signature Date Advocate’s Printed Name Date

Youth Assent Script for Unemancipated Care Youths Younger Than Age 18

[To be read to youths by survey administrators during youth assent form distribution]

We’re inviting you to be in a research study to evaluate [PROGRAM NAME] as part of your involvement with the Office of Population Affairs, Adolescent Family Life Care Program. You were chosen because you are a part of [PROGRAM NAME.] There is a toll-free phone number on the form for Dr. Ashley, who leads the study at RTI in North Carolina, that you can call with any questions.

The study is sponsored by the federal government and is conducted by RTI in North Carolina. The study is to learn about youths who are served by programs like [PROGRAM NAME].

The first survey will take place between October 2008 and November 2009. If you complete the first survey when you are pregnant, you will be surveyed again 6 and 12 months after the birth, until July 2011. If you agree, we will ask you to complete a survey. The form tells you what the questions are about: your pregnancy, future goals, sexual activity, ways to avoid pregnancy and diseases people can get when having sex, health care, your baby, parenting, and the baby’s father.

You will be asked to bring your child’s immunization records to the survey administrations. You will also be asked to write down the month and the year that your child was born. If you have more than one child, you will be asked about your youngest child. A staff member will also record information about your baby’s shots.

Each survey will take about 20 minutes.

Some of the questions might be personal or make you uncomfortable. If anyone who doesn’t work on the study saw your answers, it might create problems for you, so we are going to try very hard to protect your privacy. Dr. Ashley’s toll-free phone number is listed again for you to call if you have any problems.

The study results won’t help you directly but could help service providers learn how to give better services.

We hope that this research will help us understand how to improve programs like [PROGRAM NAME].

We will give you a $10 gift card for each survey if you try to answer any of the questions. You may also be offered a raffle opportunity to win an MP-3 player at the end of the study. Only youths who can be reached at final follow-up will be eligible to win, regardless of whether they complete the survey.

All of your answers will be kept private. We will not let anyone outside the study know your answers except if the law makes us. There are two reasons we would have to do this: 1) If you say you are a danger to yourself or others, or 2) If you say that a child is being hurt or is not being taken care of--we would have to report either of these to the authorities. This includes if you are being hurt or not being taken care of, if you are hurting or not taking care of a child you are responsible for, or if a friend of yours is being hurt or not being taken care of. Your answers will be combined with other answers, but we will replace your name with a number. So when a report is written, it will contain information from everyone who took the survey but no names.

There is a group of people at RTI who have reviewed our privacy procedures. This group might review our records about your taking the survey, and may be given your name (but not your answers).

[We will contact you again in the future when it is time to take a follow-up survey.]

Taking this survey is completely your choice. You don’t have to do it to get services here or anywhere else. Your parent has already said that it is okay for you to take the survey. If you do try the survey, you can skip any questions or you can stop at any time.

There are phone numbers you can call with questions about the study or about your privacy and rights.

Keep the copy of this form that explains all of this.

So if you sign the last page of this form, you are saying you’ve read the form, got all your questions answered, and are deciding to try the survey. Signing does not affect your legal rights.

Youth Consent Script for Care Youths Aged 18 or Older and Emancipated Minors

[To be read to youths by survey administrators during youth consent form distribution]

We’re inviting you to be in a research study to evaluate [PROGRAM NAME] as part of your involvement with the Office of Population Affairs, Adolescent Family Life Care Program. You were chosen because you are a part of [PROGRAM NAME.] There is a toll-free phone number on the form for Dr. Ashley, who leads the study at RTI in North Carolina, that you can call with any questions.

The study is sponsored by the federal government and is conducted by RTI in North Carolina. The study is to learn about youths who are served by programs like [PROGRAM NAME].

The first survey will take place between October 2008 and November 2009. If you complete the first survey when you are pregnant, you will be surveyed again 6 and 12 months after the birth, until July 2011. If you agree, we will ask you to complete a survey. The form tells you what the questions are about: your pregnancy, future goals, sexual activity, ways to avoid pregnancy and diseases people can get when having sex, health care, your baby, parenting, and the baby’s father.

You will be asked to bring your child’s immunization records to the survey administrations. You will also be asked to write down the month and the year that your child was born. If you have more than one child, you will be asked about your youngest child. A staff member will also record information about your baby’s shots.

Each survey will take about 20 minutes.

Some of the questions might be personal or make you uncomfortable. If anyone who doesn’t work on the study saw your answers, it might create problems for you, so we are going to try very hard to protect your privacy. Dr. Ashley’s toll-free phone number is listed again for you to call if you have any problems.

The study results won’t help you directly but could help service providers learn how to give better services.

We hope that this research will help us understand how to improve programs like [PROGRAM NAME].

We will give you a $10 gift card for each survey if you try to answer any of the questions. You may also be offered a raffle opportunity to win an MP-3 player at the end of the study. Only youths who can be reached at final follow-up will be eligible to win, regardless of whether they complete the survey.

All of your answers will be kept private. We will not let anyone outside the study know your answers except if the law makes us. There are two reasons we would have to do this: 1) If you say you are a danger to yourself or others, or 2) If you say that a child is being hurt or is not being taken care of--we would have to report either of these to the authorities. This includes if you are hurting or not taking care of a child you are responsible for, or if a friend of yours is being hurt or not being taken care of. Your answers will be combined with other answers, but we will replace your name with a number. So when a report is written, it will contain information from everyone who took the survey but no names.

There is a group of people at RTI who have reviewed our privacy procedures. This group might review our records about your taking the survey, may be given your name (but not your answers), and may contact you to ask you about how things went, but you don’t have to answer any of their questions if you don’t want to. It’s up to you.

[We will contact you again in the future when it is time to take a follow-up survey.]

Taking this survey is completely your choice. You don’t have to do it to get services here or anywhere else. If you do try the survey, you can skip any questions or you can stop at any time.

There are phone numbers you can call with questions about the study or about your privacy and rights.

I’ll make a copy of this form for you to keep.

So if you sign, you are saying you’ve read the form, got all your questions answered, and are deciding to try to survey. Signing does not affect your legal rights.

Youth Consent Script for Care Unaccompanied Youths

[To be read to youths by survey administrators during youth consent form distribution]

We’re inviting you to be in a research study to evaluate [PROGRAM NAME] as part of your involvement with the Office of Population Affairs, Adolescent Family Life Prevention Program. You were chosen because you are a part of [PROGRAM NAME.] There is a toll-free phone number on the form for Dr. Ashley, who leads the study at RTI in North Carolina, that you can call with any questions.

The study is sponsored by the federal government and is conducted by RTI in North Carolina. The study is to learn about youths who are served by programs like [PROGRAM NAME].

The first survey will take place between October 2008 and November 2009. If you complete the first survey when you are pregnant, you will be surveyed again 6 and 12 months after the birth, until July 2011. If you agree, we will ask you to complete a survey. The form tells you what the questions are about: your pregnancy, future goals, sexual activity, ways to avoid pregnancy and diseases people can get when having sex, health care, your baby, parenting, and the baby’s father.

You will be asked to bring your child’s immunization records to the survey administrations. You will also be asked to write down the month and the year that your child was born. If you have more than one child, you will be asked about your youngest child. A staff member will also record information about your baby’s shots.

Each survey will take about 20 minutes.

Some of the questions might be personal or make you uncomfortable. If anyone who doesn’t work on the study saw your answers, it might create problems for you, so we are going to try very hard to protect your privacy. Dr. Ashley’s toll-free phone number is listed again for you to call if you have any problems.

The study results won’t help you directly but could help service providers learn how to give better services.

We hope that this research will help us understand how to improve programs like [PROGRAM NAME].

We will give you a $10 gift card for each survey if you try to answer any of the questions. You may also be offered a raffle opportunity to win an MP-3 player at the end of the study. Only youths who can be reached at final follow-up will be eligible to win, regardless of whether they complete the survey.

All of your answers will be kept private. We will not let anyone outside the study know your answers except if the law makes us. There are two reasons we would have to do this: 1) If you say you are a danger to yourself or others, or 2) If you say that a child is being hurt or is not being taken care of--we would have to report either of these to the authorities. This includes if you are being hurt or not being taken care of, if you are hurting or not taking care of a child you are responsible for, or if a friend of yours is being hurt or not being taken care of. Your answers will be combined with other answers, but we will replace your name with a number. So when a report is written, it will contain information from everyone who took the survey but no names.

There is a group of people at RTI who have reviewed our privacy procedures. This group might review our records about your taking the survey, may be given your name (but not your answers), and may contact you to ask you about how things went, but you don’t have to answer any of their questions if you don’t want to. It’s up to you.

[We will contact you again in the future when it is time to take a follow-up survey.]

Taking this survey is completely your choice. You don’t have to do it to get services here or anywhere else. If you do try the survey, you can skip any questions or you can stop at any time.

There are phone numbers you can call with questions about the study or about your privacy and rights.

I’ll make a copy of this form for you to keep.

So if you sign, you are saying you’ve read the form, got all your questions answered, and are deciding to try to survey. Signing does not affect your legal rights.

Youth Advisor Script for Care Unaccompanied Adolescents

[To be read to youths by designated youth advisors during youth consent form distribution]

Because you are under 18, I have been assigned to be an advisor to you, which means that it is my job to make sure that you understand the study and your rights, and that you want to participate. I want to make sure that you know that your decision to take part in this research study is your choice. You do not have to take the survey in order for you to get services here or anywhere else. If you do take part in the study, you can skip any questions. If you feel like the survey is taking too long, you are getting tired, or if for any other reason you want to stop, you may do so at any time.

Do you feel like you understand your rights?

Do you have any questions about participating in this study?

If you wish to participate in this study, then you can sign the last page of your consent form. If you sign, you are saying you’ve read the form, got all your questions answered, and are deciding to try the survey. Signing does not affect your legal rights.

Appendix F

Recruiting Materials

OPA Lead Letter

[OPA LETTERHEAD]

TO: [AFL PROGRAM DIRECTOR]

FROM: Johanna Nestor

CC: Olivia Silber Ashley, RTI International

DATE: [DATE]

SUBJECT: Evaluating the Title XX Adolescent Family Life (AFL) Program

The Office of Population Affairs (OPA) has contracted with RTI International, a not-for-profit organization in Durham, NC, to design a cross-site evaluation of the AFL program. We have selected your project to participate in the cross-site evaluation. Participating in the cross-site evaluation is a condition of your grant funding.

Baseline data collection for the cross-site evaluation will begin in October 2008. RTI will contact you to begin the process of obtaining local Institutional Review Board (IRB) approval for this data collection.

If you have any questions about your participation in the cross-site evaluation, please contact RTI’s Project Director, Dr. Olivia Ashley, at (800)334-8571 ext. 6427 or osilber@rti.org or me at (240) 453-2808 or Johanna.Nestor@hhs.gov.

Thank you for your help as we learn about the impacts of the AFL demonstration projects.

Parent Lead Letters

Youth (aged 18 or older) Lead Letters

Care Lead Letter to Parents of Unemancipated Youths

(CARE PROGRAM LETTERHEAD)

[DATE]

(PARENT NAME)

(PARENT ADDRESS)

Dear (PARENT NAME):

This letter is to invite (YOUTH NAME) to participate in a research study being conducted by RTI International, a not-for-profit research organization in Durham, North Carolina. (YOUTH NAME) was selected for this study because of his/her participation in (CARE PROGRAM NAME). RTI is conducting a national study funded by the Office of Population Affairs in the U.S. Department of Health and Human Services about youths served by programs like (CARE PROGRAM NAME). This national study will involve more than 900 youths.

If you and (YOUTH NAME) agree, we would like for (YOUTH NAME) to complete written questionnaires. If your youth is pregnant now, then there will be another survey six months after birth and 12 months after birth. If your youth is not pregnant now, there will only be one additional survey, next year. Each of these surveys will take about 20 minutes of your youth’s time.

The questions ask about things like (YOUTH NAME)’s pregnancy, future goals, sexual activity, method(s) to prevent pregnancy and sexually transmitted diseases, health care, baby, parenting, and the baby’s father. She will be asked to bring her child’s immunization records to the survey administrations. A staff member from (CARE PROGRAM NAME) will record information about immunizations. After each questionnaire is completed, we will give a $10 gift card to (YOUTH NAME). Your youth may also be offered a raffle opportunity to win an MP-3 player at the end of the study.

If you have any questions about the current study, please contact me at (LOCAL NUMBER) or the RTI Project Director, Dr. Olivia Ashley, toll-free at (800) 334-8571 ext. 6427. If you have questions about your rights as a study participant, please call (LOCAL IRB) at (LOCAL NUMBER) or RTI's Office of Research Protection toll-free at (866) 214-2043.

Thank you for considering this request.

Sincerely,

(AFL CARE PROGRAM DIRECTOR)

Lead Letter for Care Youths Aged 18 or older, Emancipated Minors, and Unaccompanied Youths

(CARE PROGRAM LETTERHEAD)

[DATE]

(YOUTH NAME)

(YOUTH ADDRESS)

Dear (YOUTH NAME):

This letter is to ask you to be part of a research study done by RTI International, a not-for-profit research firm in Durham, North Carolina. We asked you to be in the study because you participate in (CARE PROGRAM NAME). This is a national study paid for by the Office of Population Affairs in the U.S. Department of Health and Human Services about youths served by programs like (CARE PROGRAM NAME). This national study will involve over 900 youths.

If you agree, we would like you to fill out a survey. It should take about 20 minutes of your time. If you are pregnant now, then there will be another survey six months and 12 months after your baby is born. If you are a parent, there will only be one additional survey, next year. Each of these surveys will take about 20 minutes of your time.

The survey asks about things like your pregnancy, future goals, sexual activity, method(s) to prevent pregnancy and sexually transmitted diseases, health care, baby, parenting, and the baby’s father. If you are a parent, you will be asked to bring your child’s immunization records to the survey administrations. Staff members from (CARE PROGRAM NAME) will record information about your child’s immunizations from these records. After the survey is finished, we will give you a $10 gift card. You will also be offered a raffle opportunity to win a prize for giving good contact information at the first survey so that you can be reached for future surveys.

If you have any questions about the study, please call me at (LOCAL NUMBER) or the RTI Project Director, Dr. Olivia Ashley, toll-free at (800) 334-8571 ext. 6427. If you have questions about your rights as a study member, please call (LOCAL IRB) at (LOCAL NUMBER) or RTI's Office of Research Protection toll-free at (866) 214-2043.

Thank you,

(AFL CARE PROGRAM DIRECTOR)

Appendix G

Cross-Site Evaluation Study Protocol

Care

1) Care – Baseline Survey

• Participants assigned to treatment and comparison conditions within AFL projects

• All participants complete baseline survey

• Treatment group participants will receive services such as enhanced program services, home visitation and outreach, case management, clinical services, transition to parenthood curricula, couples classes, grandparent and sibling education, activities and education for fathers, life skills, leadership activities, career exploration, mental health services, and child assessments

• Comparison group participants will either receive basic services provided by Care projects or other health systems or no program

2) Care – 1^st Follow-Up Survey (at 6 months after birth of child)

• All participants who are pregnant at baseline (treatment and comparison conditions combined) complete this 1^st follow-up survey

3) Care – 2^nd Follow-Up Survey (at 12 months after birth of child)

• All participants (treatment and comparison conditions combined) complete this 2^nd follow-up survey