YEARS SS-Part A REVISED 11 30 15_clean

The Office of Planning, Research and Evaluation (OPRE) at the U.S. Department of Health and Human Services (HHS) seeks approval for information collection (IC) activities as part of their effort to better understand the services that Office of Family Assistance (OFA)-funded Healthy Marriage and Relationship Education (HMRE) grantees are providing to youth aged 14 to 24. In order to carry out this work, OPRE has contracted with Child Trends. The study team will draw from both existing documentation and grantee reports (see Appendix A: Pre Existing Data Sources) and newly collected data from a web-based staff survey and information collected during three grantee pilot site visits and six grantee site visits, which will include semi-structured interviews with program staff. The team will also draw on information from program observations conducted during the site visits. These observations do not impose any respondent burden beyond the initial screening and recruitment efforts needed. The major goal of this project is to help OPRE understand the current state of HMRE programming for youth.

The proposed project will address the following objectives:

Objective 1: Describe the organizations implementing HMRE programs and the youth these programs serve by collecting and analyzing multiple sources of quantitative and qualitative data.

Objective 2: Assess the alignment of HMRE programs with best practices in the field.

Objective 3: Identify promising approaches used by grantees to better serve youth in HMRE programs.

Study Background

Romantic relationships during adolescence are developmentally appropriate and important for youth. Healthy relationships, both romantic and platonic, can be a positive developmental influence. The majority of first romantic relationships take place during adolescence, and adolescents’ self-reports of dating increase as they progress through high school (Carver, Joyner, & Udry, 2003; Collins, 2003). Romantic relationships help shape a variety of behaviors (both positive and negative) during adolescence, contribute to identity development, and influence adolescents’ romantic relationships in young adulthood and beyond (Wildsmith, 2013). A recent review of the literature suggests that high-quality romantic relationships in adolescence may be associated with an increased likelihood of positive relationships and relationship commitment in early adulthood and later in life (Collins, Welsh, & Furman, 2009). Thus, providing adolescents with the tools necessary to create and sustain healthy relationships (with romantic partners as well as peers, employers, teachers, and parents), both during adolescence and into young adulthood, may help positively contribute to their overall development, and set the stage for the creation and sustainability of healthy relationships later in life. Effective HMRE programs for youth can contribute to this by improving adolescents’ attitudes and ideals concerning romantic relationships, and by helping adolescents develop key skills to form healthy relationships and avoid unhealthy relationships.

The ultimate goal of this IC is to inform future efforts to improve youth-serving HMRE programs. To this end, Child Trends has developed a series of research questions to address each objective, covering three broad topics: 1) HMRE program and partner characteristics and settings (Objective 1); 2) HMRE youth participant characteristics (Objective 1); and 3) implementation strategies employed by HMRE grantees (Objectives 2 and 3).

Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. OPRE is undertaking the collection at the discretion of the agency.

A2. Purpose of Survey and Data Collection Procedures

Overview of Purpose and Approach

The purpose of gathering and analyzing this information is to better understand the services OFA-funded HMRE programs provide to youth aged 14 to 24. Collected data will be used to describe youth-serving HMRE programs and partners, participant characteristics, and program implementation characteristics.

The data collection plan incorporates a mixed-methods approach to address each proposed research question detailed in Appendix B YEARS Research Questions, Data Sources, and Respondents. Specifically, we have obtained pre-existing data from: Grantee applications; OLDC Report Questions/Information; Performance Progress Reports; and Grantee Profiles. Pre-existing data will provide valuable information about the characteristics of HMRE grantees and the youth participants they serve. These data will be analyzed prior to the start of new data collection activities. The use of existing grantee data does not impose a burden on respondents or record keepers.

Building upon these pre-existing data, we will also collect new data (beginning in winter 2016, upon OMB and IRB approval) through: A web-based survey for HMRE grantee staff (to be completed by Program Directors/Administrators and Program Facilitators); and site visits that will include semi-structured interviews (with Program Directors/Administrators, Program Facilitators, and Partner Organizations/Providers) and program observations. IC will focus on HMRE programs serving youth aged 14 to24. The program observation does not impose any respondent burden.

Child Trends piloted the web-based staff survey with nine respondents in June 2015 and revised the survey based on respondent feedback. The final version of the web-based staff survey will be distributed after OMB approval in winter 2016 with data collection to take place over a two month period.

Child Trends will visit three pilot sites in the fall of 2015 to pilot test the interview protocol with up to nine participants. If edits are found necessary based on piloting, these changes will be submitted to OMB. Upon OMB approval, Child Trends will visit an additional six sites, beginning in winter 2016 with data collection expected to take place over a two month period.

Research Questions

Approximately half of the 2011 OFA-funded healthy marriage and relationship education grantees elected the activity that supports education in high schools, and roughly half of all individuals served through OFA grantees are under age 18, which further emphasizes the critical need to understand: 1) how programs tailor their services to adolescents; 2) how they address different youth subgroups (e.g., lesbian, gay, bisexual, transgender, and questioning (LGBTQ), racial/ethnic minority, foster care); 3) by what means they reach them (i.e. in addition to high schools); and 4) the degree to which the programs are informed by research and best practices. This information will ultimately help fill gaps in services for adolescents and in the research base for HMRE programs.

While there are many evidence-based HMRE programs and curricula for adults, little research and policy, so far, has focused on effective strategies to serve adolescents (14-18 year olds) and older youth (19-24 year olds) (Karney, Beckett, Collins, & Shaw, 2007; Office of Family Assistance, 2012). Because of this gap in youth-serving HMRE programs as compared to non-youth HMRE programs that mainly serve adults, we will examine OFA-funded HMRE programs serving young people aged 14 to 24. Although older youth may require programming that includes more in-depth content related to topics (e.g., money management and parenting), little is known about the similarities and differences between younger and older youth in terms of their HMRE program needs and experiences.

We have developed a series of research questions to address each stated objective. The research questions fall under three broad topics, which in turn fulfill all three objectives, including: 1) HMRE program and partner characteristics and settings; 2) HMRE youth participant characteristics; and 3) implementation strategies employed by HMRE grantees. Appendix B YEARS Research Questions, Data Sources, and Respondents lists all research questions under each broad topic category.

Study Design

A mixed-method approach allows us to use multiple data sources to answer our research questions and better understand the OFA-funded HMRE services provided to youth aged 14 to 24.

Existing data sources will provide select information about the program activities and implementation strategies used by the grantees. When possible, pre-existing data from multiple time points will be collected and analyzed to better understand how, if at all, implementation strategies evolve over time. Each pre-existing data source is described in Appendix A Preexisting Data Sources.

New data will be collected from OFA-funded HMRE grantees to supplement and expand upon the information collected from the pre-existing data sources. New quantitative data will be collected through a web-based staff survey. The purpose of the quantitative data collection is to describe youth-serving HMRE program and participant characteristics, and their implementation characteristics. Qualitative data will be gathered through semi-structured interviews with Program Facilitators and Partner Organizations/Providers, and program observations conducted during site visits with up to nine grantees to understand in more depth youth-serving HMRE programs and participants, including promising approaches used by grantees, obstacles to service delivery, and other information program staff have to share about their experiences implementing youth-serving HMRE programs.

Qualitative interviewing is ideal for the research questions because it allows flexibility for the respondent to guide the interview. Moreover, the semi-structured interviews can be adapted to a variety of respondent types (i.e., Program Directors/Administrators, Program Facilitators, and Partner Organizations/Providers), which supports data collection from multiple participants with varied perspectives and improves the quality of the data (Rubin, 2011).

Semi-structured interviews with program staff will be conducted with Program Directors/Administrators, Program Facilitators, and Partner Organizations/Providers. Program Director/Administrator is a term we use broadly to refer to an individual who oversees program operations. This individual may oversee more than one type of program and may or may not implement or facilitate sessions directly with youth. They are likely responsible for tracking participant data, hiring and training staff, reporting to funders, and maintaining partnerships with other organizations. Program Facilitators are the individuals working directly with youth to implement the program. The dynamics of these roles may overlap and vary by program. Partner Organizations/Providers could include a range of personnel who work indirectly with youth-serving HMRE programs, most likely in the organization where the HMRE program is provided, such as a school or community center. For example, Partner Organizations/Providers could include school principals, school faculty, administrators, clinic staff, university staff, or others identified during the recruitment process. The goal of speaking with Partner Organizations/Providers is to get a sense of how youth-serving HMRE programs are perceived by the partner organization, how they are received by the broader community, how partner organizations find out about these programs, and why they choose to partner with an HMRE program to serve youth.

Universe of Data Collection Efforts

The research approach calls for the development of a customized YEARS web-based staff survey, a semi-structured interview protocol, and a program observation protocol. For a detailed list of questions that each instrument will answer, see Appendix B YEARS Research Questions, Data Sources, and Respondents. The current request for IC includes the following:

YEARS web-based staff survey: The web-based staff survey is designed to explore characteristics of HMRE programs for youth and to address the research questions posed by the current project (see Attachment 1 YEARS Web-Based Staff Survey for Program Directors/Administrators and Attachment 2 YEARS Web-Based Staff Survey for Program Facilitators). Survey respondents will be asked to give their consent before beginning the survey (see Appendix C Consent Form YEARS Web-based Survey).
Semi-structured interviews: The qualitative data derived from the proposed interview protocol will fill gaps in the information collected from existing data sources and the proposed survey (see Attachment 3 YEARS Semi-Structured Interview Protocol). The interviewees will be asked to give their consent before beginning the interview (see Appendix D Consent Form YEARS Interviews).
Program observations: Program observations will allow the team to answer research questions regarding participant engagement, facilitator characteristics, and environmental characteristics that may affect program implementation (see Attachment 4 YEARS Program Observation Protocol). These observations do not impose any burden on the observed.

Information will also be used from existing data sources, which do not impose any respondent burden.

A3. Improved Information Technology to Reduce Burden

Whenever possible, we will use advanced technology to collect and process data to reduce respondent burden and make data processing and reporting more timely and efficient. For the staff survey, we have developed a web-based electronic survey using Survey Gizmo. The electronic format of the survey will facilitate rapid dissemination, collection, and data management while reducing costs and grantee burden. A digital audio recorder will be used in all semi-structured interviews, if respondents agree to be recorded. Before using the audio recorder, verbal consent will be obtained from the Program Directors/Administrators, Program Facilitators, and Partner Organizations/Providers Child Trends is interviewing.

A4. Efforts to Identify Duplication

Every effort has been made to determine whether similar research and information exists by searching existing studies and reports, and in consultation with federal staff. As part of these efforts, we reviewed reports that specifically focus on HMRE programs for youth, as well as literature related to positive youth development. Though existing research about HMRE programs for youth is available, additional information is needed to better understand how HMRE programs are being implemented with youth. We concluded that no existing data source can provide the data needed to answer the study’s research questions.

Within the current study, there is some duplication of questions across instruments and across reporters, although we have taken steps to minimize duplication and the associated burden. However, some duplication is intentional. For example, we will ask some of the same questions of multiple respondents (Program Directors/Administrators, Program Facilitators, and in some cases Partner Organizations/Providers) to gain different perspectives about certain topics (e.g., how well Program Facilitators versus Program Directors/Administrators know the organization and the program). This overlap in burden is intentional to measure the extent to which the Program Facilitators understand and have familiarity with the program, and to assess the extent to which different stakeholders (Program Directors/Administrators, Program Facilitators, and Partner Organizations/Providers) perceive how the program is implemented in similar or different ways.

We have made efforts to streamline the instruments themselves so that there is not overlap in the information collected between instruments. For example, we did not include any questions in the web-based survey that could be answered with the pre-existing grantee data unless a question was necessary for follow up questions that provided unique information. In sum, we ask the same questions only when the purpose is to obtain multiple perspectives about how programs operate. In general, each data source asks about different reporting periods or has unique questions, even if the questions cover the same broad research topics addressed in other instruments.

A5. Involvement of Small Organizations

No information will be collected from small businesses or other small entities.

A6. Consequences of Less Frequent Data Collection

This is a one-time data collection.

A7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register Notice and Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on February 26, 2015, Volume 80, Number 38, page 10492, and provided a sixty-day period for public comment. During the notice and comment period, the government did not receive any comments in response to the Federal Register notice.

In order to fully address the objectives outlined for this project, it was determined that additional information collection beyond what was proposed in the 60 day Federal Register notice is necessary. Therefore, the proposed site visits submitted with this request (including site visit screening and recruitment, and semi structured interviews with Program Directors/Administrators, Program Facilitators, and Partner organizations/providers) require additional burden beyond that originally estimated in the 60 day Federal Register notice.

Consultation with Experts Outside of the Study

No experts outside of the study have been consulted.

A9. Incentives for Respondents

For their participation in the web-based staff survey and the semi-structured interview, respondents will receive a gift card as a ‘thank you.’ We will provide gift cards as a token of appreciation for new data collection activities for two primary reasons 1) because we are asking participants to travel for an in-person interview in most cases, which incurs a cost and inconvenience to the participant, and 2) to help reduce nonresponse bias. While we will make every effort to meet participants in a location that is convenient for them, there will ultimately be some cost incurred due to transportation to the interview. Additionally, nonresponse bias may occur if the participants who agree to participate provide responses that differ from responses that would have been provided by individuals who choose not to participate. A monetary incentive helps to reduce nonresponse bias because individuals who may not otherwise have agreed to participate because, for example, of the financial burden of getting to the interview, can now participate.

In the case of the web-based surveys, we will be asking staff whose programs may no longer have federal funding to participate. A monetary incentive may help to reduce nonresponse bias for the web-based staff survey by encouraging staff whose programs no longer have federal funding to participate at the same rate as staff from re-funded programs. We believe an incentive is necessary given that many 2011 grantees that were not re-funded in 2015 may be reluctant to respond to a request to provide information about their previous programming. Moreover, we anticipate that the re-funded grantees will be extremely busy during the planning phase for the new grants, and may need encouragement to take time to complete a survey about their previous programming. The amounts of the gift cards will express gratitude for the respondents’ participation, without being an excessive amount that could be coercive (Wertheimer & Miller, 2008). Therefore, participants who do not want to participate will not agree to participate solely for monetary gain.

For the web-based staff survey, we plan for each participant to receive $10 as a ‘thank you’ for their participation. To help ensure anonymity of respondents and to help ensure respondents’ contact information is not linked with their survey responses, at the end of the survey or the time at which participants choose to stop their participation, participants will be provided a link for a separate form where they can enter their name and the address where they would like the $10 gift card to be sent.

Respondents will receive a $25 gift card as a ‘thank you’ for participating in the semi-structured interview.

The gift will be provided at the beginning of the study, and respondents who withdraw during an interview will still be offered the indicated amount.

Both incentive amounts - $10 for the web-based survey and $25 for the in-person interviews – are OMB suggested, and similar incentives have been used in other OMB-approved studies. Specifically, the Early Learning Mentor Coach project (contract HHSP23320095626W) used a $25 gift card incentive for an hour long in-person interview, and the MIHOPE study (contract HHSP23320095644WC) also used a $25 incentive for an hour long interview. The EHSREP Tracking study (contract HHSP23320072914YC) gave a $10 incentive for a telephone check-in, which is similar in burden to the web-based survey in the proposed study.

There is no incentive for participation in a program observation session.

A10. Privacy of Respondents

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law.

For the web-based staff survey, the study team will email grantees to inform them about the survey and to provide background about the goals and objectives of the study. We will send the electronic link to each designated contact person at each grantee organization (identified by the contact information found in the grantee applications and grantee profiles). This email will also contain an attached study consent form (see section B.2 in Supporting Statement Part B for additional information) which will outline the participant’s rights as a respondent in the study, including the right not to participate at all, indicate that all responses will be aggregated and not linked to individual names, and indicate that all information will be kept private to the extent permitted by law. IC for this aspect of the research plan will be at the grantee level and will not require personally identifiable information about the respondent. Prior to beginning the survey, respondents will be asked to read the study consent form (see section B.2 in Supporting Statement Part B for additional information) which describes the study in detail including their rights as a participant. If participants do not agree (they will be given two boxes “I agree” or “I do not agree”) to the terms in the consent form, they will be taken to a screen that ends the survey.

For the semi-structured interviews, the study team will ask the point of contact at the HMRE program site to recommend potential participants. We will then follow up with the potential interviewees to describe the study and consent process and to gather their contact information for further follow-up. After potential participants agree to participate in the interview, we will send them an interview packet, which will include the consent form. Prior to the start of the interview, the interviewer will also give the respondent another copy of the consent form. Before the interview begins, the interviewer will ask the respondent if she or he has read the form and then will answer any questions. If the respondent replies that she or he has not read the consent form, the interviewer will read the form aloud, and then will answer any questions. Only the interviewer will sign the form acknowledging that the participant consents to participate. By not requiring the participant to sign and print her or his name on a consent form, the study team will minimize the amount of personally identifiable information collected. We will request a “Waiver of signed consent” through the Child Trends IRB. If any semi-structured interviews need to be conducted over the phone (e.g., if staff are unavailable to attend in-person interviews during the site visits), this approach will also reduce burden to respondents participating by phone by not requiring the participant to sign and return a hardcopy. After consent is obtained, the interview will begin. Semi-structured interviews will be audio recorded, if the participants agree.

Participants in all IC activities (web-based staff survey and semi-structured interviews) will be notified that their results will be combined with the results of others and that their names will not be used in reports of data. They will also be notified that the funder will not be made aware of their participation or refusal to participate. They will also be made aware that their decision to participate will have no bearing on their funding.

Child Trends will not begin any data collection activities prior to receipt of IRB and OMB approval.

As specified in the contract, the Contractor (Child Trends) shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor who perform work under this contract/subcontract are trained on data privacy issues and comply with the above requirements.

As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard (currently, FIPS 140-2) compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.

The Office of Planning, Research and Evaluation is in the process of completing a Privacy Impact Assessment (PIA) to ensure that information handling conforms with applicable legal, regulatory, and policy requirements regarding privacy; determine the risks of collecting and maintaining PII; assists in identifying protections and alternative processes for handling PII to mitigate potential privacy risks; and communicates an information system’s privacy practices to the public. This PIA, titled ACF Research and Evaluation Studies, will be available online through the Department of Health and Human Services.

A11. Sensitive Questions

There are no sensitive questions in this data collection. However, we will take measures to ensure that participants are aware of their rights in the study and their ability to skip any questions they do not wish to answer.

A12. Estimation of Information Collection Burden

Newly Requested Information Collections

The chart below shows estimated burden of the information collection for the staff web-based survey and the semi-structured interview protocol, separating the Program Facilitator, the Program Director/Administrator, and the Partner Organization/Provider. The site observation protocol will not impose additional burden beyond the initial site visit screening/recruitment.

Total Burden Requested Under this Information Collection

Instrument	Total/Annual Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Annual Burden Hours	Average Hourly Wage	Total Annual Cost
YEARS web-based staff survey (Program Director/Administrator)	44	1	0.5	22	$32.56	$716.32
YEARS web-based staff survey (Program Facilitator)	44	1	0.5	22	$19.72	$433.84
Site visit screening and recruiting	15	1	0.5	7.5	$32.56	$244.2
Semi-structured interview (Program Director/Administrator)	6	1	1	6	$32.56	$195.36
Semi-structured interview (Program Facilitator)	6	1	1	6	$19.72	$118.32
Semi-structured interview (Partner organization/Provider)	3	1	1	3	$44.13	$132.39
Estimated Annual Burden Total				66.5		$1,840.43

Total Annual Cost

There is an estimated annualized cost to respondents of $1,840.43. An hourly wage of $19.72 was assumed for Program Facilitators based on the Bureau of Labor Statistics’ (BLS) report on average earnings for Self-Enrichment Education Teachers, the job description that describes the work of Program Facilitators (Bureau of Labor Statistics, 2015c). Program Directors/Administrators’ hourly wage estimate is $32.56 based on BLS average earnings for Social and Community Service Managers. Partner Organizations/Providers’ hourly wage estimate is $44.13 based on BLS average earnings for Education Administrators (Bureau of Labor Statistics, 2015a).

There will be no direct cost to the respondents other than their time to participate in the study.

A13. Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents.

A14. Estimate of Cost to the Federal Government

The total cost for the data collection activities under this current request will be $95,437. Annual costs to the Federal government will be $95,437 for the ICR. This includes direct and indirect costs of information collection, which covers staff time and other affiliated costs for the IRB process, survey development, data collection, and data analysis.

A15. Change in Burden

This is a new data collection.

A16. Plan and Time Schedule for Information Collection, Tabulation and Publication

Analysis Plan

Upon OMB approval, data will be collected over two months in the winter of 2016. Data analysis of the information gathered from the web-based staff survey, semi-structured interviews with staff, and site visits will be completed before summer of 2016. A public interim dissemination product is expected in fall 2015, and the final report is expected in summer 2016.

Time Schedule

Timing	Activity
Summer 2015	PRE-TEST Web-based staff survey
Fall 2015	PRE-TEST Site visits Interviews INTERIM DISSEMINATION
Winter 2016	DATA COLLECTION Survey Site visits Interviews
Spring 2016	DATA ANALYSIS
Summer 2016	FINAL REPORT AND DISSEMINATION

A17. Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

File Type	application/msword
File Title	OPRE OMB Clearance Manual
Author	DHHS
Last Modified By	Molly
File Modified	2015-12-04
File Created	2015-12-04