CEY OMB Case Studies Part A (2)

OMB Clearance Submission

Communities Empowering Youth (CEY) Evaluation Study

June 10, 2008

Office of Planning, Research and Evaluation

Administration for Children and Families

US Department of Health and Human Services

370 L’Enfant Promenade SW, 7th Floor

Washington, DC 20447

Part A: Justification

A.1 Explanation of Circumstances That Make Collection of Data Necessary

Following Congressional authorization, the Department of Health and Human Services (Office of Community Services, Administration for Children and Families [ACF]) established the Compassion Capital Fund (CCF) in 2002. CCF is charged with enhancing the capacity and organizational capability of faith-based and community organizations (FBCOs) to better serve at-risk and in-need populations.

In FY 2006, ACF’s Office of Community Services began a new demonstration program under CCF, the Communities Empowering Youth (CEY) program This program blends the capacity building mission of CCF with the Administration’s emphasis on meeting the needs of America’s disadvantaged youth through “supporting the cornerstones for youth development: strong families, effective schools, and caring communities.” There were 100 CEY grants awarded in FY 06, the first year of the program, and 31 grants in FY 07.

A distinguishing aspect of the CEY demonstration is the selection and funding of partnerships headed by a “lead” FBCO. The lead FBCO is responsible for forming a coalition of partners who are collectively committed to addressing local issues of youth violence, gang activity, and child abuse and neglect, and fostering supportive relationships with youth. Consistent with the mission of CCF, lead FBCOs are responsible for providing technical assistance, training and financial assistance to their FBCO partners in four critical developmental areas: leadership development, organizational development, program development, and community engagement. Lead organizations and their partners are also encouraged to collaborate with local government agencies providing services to youth.

The CEY approach recognizes that successfully addressing the challenges facing disadvantaged youth requires a highly integrated, coordinated and professional social service delivery network at the local level. In addition, the complexity and multi-faceted nature of the task requires community partnerships that generate a level of efficiency and synergy that a single organization cannot provide on its own. In this spirit, the CEY grants are designed to foster and equip community partnerships with the support needed to develop and sustain organizational capacity.

In September 2007, ACF awarded a contract to Abt Associates Inc. to conduct an evaluation of the CEY Program. The CEY evaluation, as a whole, is a multi-method study that couples quantitative survey methods and qualitative case study methods. This multi-method approach will provide generalizable findings via the survey,¹ supplemented by more in-depth qualitative information from case studies about the interactions among partner agencies and the specific processes of capacity building utilized by partnering agencies over time. The case study component will include a purposive sample of 10 CEY grantees and their partners. This proposed data collection supports the longitudinal case studies.

While the survey data will provide information about general capacity building, the case studies will provide an in-depth understanding of aspects of the partnerships such as the dynamics of the partner relationships, how they form and grow, how they interact with other relevant community organizations, and how they utilize the CEY funding can best be observed and studied fully and in depth through the case studies. Longitudinal case studies, with multiple site visits to the same group of grantees and their partners, have the added benefit of being able to track changes in capacity building and partnership development over time. These insights will complement what is learned from the quantitative components of the evaluation and will provide a richer context for interpreting the survey results.

The case studies add value to the evaluation by obtaining more in-depth information about some of the same topics that are addressed on the survey. For instance, the baseline survey asks a single question about the CEY partnership’s structure (to which respondents select one response). The case studies allow us to delve into this topic by asking how the partnership decided on its structure, how the structure has evolved over time, and the benefits and drawbacks of this structure for the partnership. Similarly, the survey asks yes/no questions such as, “Has your CEY partnership met since the grant was awarded?” “Does your partnership have a regular calendar of meetings?,” and “Does your organization communicate as needed…?” The case study interviews allow for multiple questions and follow-up probes on these topics.

In addition, the case studies add value to the evaluation by addressing topics that are not covered on the survey. For instance, questions about the community needs assessment and how that influenced CEY activities, strategies and challenges associated with building and maintaining relationships among the partners, and how roles and responsibilities for carrying out CEY activities are allocated among the partners are among the topics that will be primarily addressed through the CEY case studies.

A.2 How the Information Will Be Collected, by Whom, and For What Purpose

The CEY longitudinal case studies will be used to collect data from 10 CEY grantees and their partners over a three-year period. Data will be collected primarily through annual site visits to each selected lead organization as well as its partners. Within each site we will obtain multiple viewpoints by interviewing key staff from the lead grantee organization and CEY partner organizations. On average, CEY grants include six partner agencies in addition to the lead agency. We will attempt to schedule interviews with executive and/or program directors and other key staff who are familiar with the activities carried out under the CEY grant, in order to collect information about which respondents have direct knowledge.

These site visits will be conducted over a three-day period by a team of two researchers. During the site visits, we will conduct interviews using the attached Lead Organization Interview Protocol and Partner Organization Interview Protocol, and observe partnership activities (including technical assistance activities and general meetings) using the attached Observation Protocol. The primary respondent for these interviews will be the director of the lead agency and each partner, but may also include other key staff that play a prominent role in staffing the CEY partnership.

In addition to the annual on-site in-person visits, we will collect additional data on the 10 case study grantees and their partners via telephone. These check-in telephone calls will occur twice—after the first and second rounds of site visits—and will also be conducted with the directors of the lead FBCO agencies. A separate “Check-In Protocol” (see appendix D) will guide these conversations.

Research Questions

The longitudinal cases studies will address the following research questions:

CEY partnerships: how they are structured, the dynamics between the lead and partner organizations, and how these change over time

1. How does the governance and structure of CEY partnerships evolve over time?

2. Does there appear to be a relationship between the partnership structure and changes in partnerships’ capacities?

3. How successfully does the lead organization engage its partners?

Improving organizational and partnership capacity over time

4. In what areas and how do partnerships improve the capacity of the partnership as a whole?

5. In what areas and how do lead organizations build their own capacity?

6. In what areas and how do individual partners build their own capacity?

7. How substantial are changes in the capacity of the lead, the partners, and the partnership?

CEY implementation

8. What implementation challenges have projects confronted and what strategies have been most useful for overcoming such challenges?

9. What are examples of promising practices that appear to be linked with improved capacity?

A.3 Use of Improved Information Technology to Reduce Burden

The kind of information being sought through this collection is not routinely recorded in any electronic information medium. Thus, technology cannot be used to substantially reduce burden. To enhance the quality and accuracy of the data collection, the interviews will be recorded if there is no objection from the FBCO staff. In addition, the contractor will collect information about the organizations electronically prior to the site visits to the extent available (e.g., from websites, data files key staff can make available, etc.).

A.4 Efforts to Identify and Avoid Duplication

We are not aware of similar prior or ongoing data collection that duplicates the efforts of the proposed case studies. While CEY grantees report some similar information through progress reports to ACF, the information is not uniform or consistent. CEY grantees will also be asked to complete a survey as a part of the evaluation; the intent of this proposed information collection is to build on and complement information obtained through the survey, not duplicate it. The proposed data collection for the case studies will not ask the same questions as those on the survey, but will probe similar topics in greater depth.

A.5 Efforts to Minimize Burden on Small Business or Other Entities

No small businesses will be involved. This information collection will, however, involve gathering information from other small entities, namely faith-based and community nonprofit organizations. Every effort will be made to minimize the burden on these organizations. Site visits will be scheduled at the respondents’ convenience and information about the purpose and scope of the visits will be provided in advance.

A.6 Consequences of Less-Frequent Data Collection

On-site data collection will occur one time per year over the three-year grant period (for a total of 3 site visits per site). The benefit of multiple visits to the same group of grantees and their partners is the ability to learn about and document changes in capacity building and partnership development over time. The annual schedule is important to document incremental changes in activities or relationships among partner agencies that might be forgotten if less than annual visits were scheduled.

A.7 Special Circumstances Requiring Collection of Information in a Manner Inconsistent with Section 1320.5(d)(2) of the Code of Federal Regulations

There are no special circumstances associated with this data collection.

A.8 Federal Register Comments and Persons Consulted Outside the Agency

In accordance with the Paperwork Reduction Act of 1995, ACF published notice in the Federal Register announcing the intention to obtain information about CEY grantees and to request an OMB review of data collection protocols. The first notice was published on March 31, 2008 in volume 73, number 62, page 16860, and provided a 60-day period for public comments. There have been no public comments received in response to the first notice prior to this submission.

The interview protocols and observation guide were developed by research team members at Abt Associates Inc. and Branch Associates, Inc. who are integrally involved in the CEY evaluation and are very knowledgeable about the roles and functioning of CEY grantee organizations. Many of the researchers have also participated in the development of data collection instruments used in the evaluation of other components of the CCF program that also address organizational capacity building. Researchers with primary responsibility for protocol development include: \

Project Director, JoAnn Jastrzab, Abt Associates Inc., 617-349-2372

Deputy Project Director, Ryoko Yamaguchi, Abt Associates Inc., 301-634-1778

Task Leader, Barbara Fink, Branch Associates, Inc., 215-731-9980

Cynthia Sipe, Branch Associates, Inc., 215-731-9980

Matthew Coll, Branch Associates, Inc. 215-731-9980

A.9 Payments to Respondents

There will be no payments made to respondents. Experience on previous studies indicates that payments are not needed for this type of research.

A.10 Assurance of Confidentiality

Every effort will be made to maintain the privacy and/or confidentiality of respondents, using several procedural and control measures to protect the data from unauthorized use. Other than name, no personal identifying information is proposed to be collected. The confidentiality procedures for this study during data collection, data processing, and analysis activities are as follows:

• All respondents included in the study sample will be informed that the information they provide will be kept private and/or confidential, to the best of our ability using several procedural and control measures to protect the data from unauthorized use, and will be used by researchers only for the purpose of this study.

• All individuals hired by Abt Associates and Branch Associates are required to adhere to strict standards and sign an oath of confidentiality as a condition of employment.

• Branch Associates and Abt Associates maintain restricted access to all data preparation areas (receipt, coding, and data entry). All data files on multi-user systems will be under the control of a database manager, with access limited to project staff on a “need to know” basis only.

• Any individual identifying information will be maintained separately from completed data collection forms and from computerized data files used for analysis. No respondent identifiers will be contained in public use files made available from the study.

• Information on laptop computers will be encrypted and all laptops used for this study contain encryption software that meets Federal standards.

A.11 Questions of a Sensitive Nature

The questions included on the data collection instruments for this study do not involve sensitive topics.

A.12 Estimates of Respondent Burden

Exhibit 1 presents our estimate of the burden for all respondents. Time estimates are based on experience with similar instruments in other studies involving comparable types of organizations.

ANNUAL BURDEN ESTIMATES

Instrument	Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours
Lead Organization Executive Director	10	1	3.5	35
Lead Organization Key Staff	20	1	2.5	50
Partner Organization Executive Director	60	1	3.5	210
Partner Organization Key Staff	60	1	2.5	150

Estimated Total Annual Burden Hours 445

Exhibit 1: Estimates of Annualized Cost
Respondent	Response Burden in Hours	Estimated Cost Per Hourb	Costs per Respondent	Total Cost
CEY Lead Organization
Executive Director	35	$24.73	$98.92	$989
Key Staff	50	$19.42	$19.42	$388
CEY Partner Organization2
Executive Director	210	$24.73	$49.46	$2,968
Key Staff	150	$19.42	$19.42	$1,165
Total				$5,510
Notes: a Assumes 100 percent response rate. b U.S. Department of Labor, Bureau of Labor Statistics, National Compensation Survey: Occupational Wages in the United States, June 2006, Managers, social and community service organizations.http://www.bls.gov/ncs/ocs/sp/ncbl0910.pdf

A.13 Estimates of the Cost Burden to Respondents

There is no annualized capital/startup or ongoing operation and maintenance costs associated with collecting the information. Other than their time to complete the interview, which is estimated in Exhibit 1, there are no direct monetary costs to respondents.

A.14 Estimates of Annualized Government Costs

The information collection activity and associated forms have been developed in the performance of the U.S. Department of Health and Human Services Contract Number: GS-10F-0086K, Order Number: HHSP233200800067G. The total cost to the Federal government for the CEY Evaluation is $2,087,750. Of that total, the annual cost for the case study data collection, analysis and reporting activities is approximately $238,390.

A.15 Changes in Hour Burden

No change in burden is requested. This submission to OMB is for an initial request for approval.

A.16 Time Schedule, Publication, and Analysis Plan

The schedule shown below in Exhibit 2 displays the sequence of activities required to conduct these information collection activities and includes key dates for activities related to instrument design, data collection, analysis, and reporting.

Exhibit 2: Time Schedule
Activities and Deliverables	Date
Round 1
Site Visit Protocol development	December 2007 – March 2008
Pilot testing	June 2008 – August 2008
Site visits to grantees	September 2008 – November 2008
Data analysis	November 2008 – January 2009
Draft CEY Case Study Report	March 2009
Round 2
Phone check-in interview	February 2009
Follow-up site visits to grantees	April 2009 – June 2009
Data analysis	June 2009 – August 2009
Draft CEY Case Study Interim Report	November 2009
Round 3
Phone check-in interview	November 2009
Final site visits to grantees	April 2010 – June 2010
Data analysis	June 2010 – August 2010
Draft CEY Case Study Final Report	November 2010

A.17 Display of Expiration Date for OMB Approval

ACF is not requesting a waiver for the display of the OMB approval number and expiration date on the data collection instruments.

A.18 Exceptions to Certification Statement

This submission does not require an exception to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).

1 The baseline survey data collection was approved by OMB in December 2007 (OMB control number: 0970-0335, expiration date 12/31/2010) and is presently underway.

2 The estimated number of partner organizations is based on the average of six partners per lead organization.

Part A: Justification 2