OMB Just Pkg - Part B 4-28-08

Part B

Statistical Methods (used for collection of information employing statistical methods)

B.1 Respondent Universe and Sampling Methods

Background

The CCF Impact Evaluation is designed to assess the effectiveness of the CCF Demonstration program. The CCF Demonstration program provides grants to intermediary organizations that provide organizational capacity building services to grassroots faith-based and community organizations (FBCOs). The Impact Evaluation is being conducted with the organizations that were initially awarded CCF Demonstration program grants in Federal Fiscal Year 2006.

The Impact Evaluation includes a structured baseline and follow-up survey of FBCOs (OMB # 0970-0293 ) as well as a survey of intermediaries (OMB # 0970-0316). The FBCOs found eligible by each intermediary to receive their services were randomly assigned to a program group that could receive training and other assistance from intermediaries or to a control group that was precluded from receipt of services for the period of the evaluation study.

The structured FBCO baseline and follow-up surveys of all FBCOs in both program and control groups will provide the primary data for assessing the effectiveness of the CCF Demonstration Program. The structured intermediary survey will provide the primary data for describing the characteristics of the intermediaries and the services the offered/provided. The interviews to be conducted within the process study are expected to provide additional contextual information that will be helpful in interpreting responses to the intermediary survey including understanding how service delivery was operationalized by intermediaries and received by a subset of FBCOs

Sample Selection

Ten intermediary organizations were awarded Compassion Capital Fund (CCF) Demonstration grants in the FY 2006 grant cycle. All ten of the intermediary organizations are included in the Impact Evaluation and all ten intermediaries are to be included in the process study component of the Impact Evaluation.

A purposive sample of 20-30 FBCOs assigned to the program group are to be included in process study (2-3 per intermediary). There are a total of 237 FBCOs in the Impact Evaluation research sample program group (eligible to receive intermediary services). The average number of FBCOs assigned to the program group across intermediaries is 24; this number ranged from 9 to 40.

The purposive sample of FBCOs is not intended to be statistically representative, nor will the information from FBCOs obtained through the process study be considered representative.

The FBCOs to be considered for the process study are those that are located in geographic proximity to the intermediary and can be visited during the period of time the site visit is conducted with the intermediary. Within the 20-30 FBCOs to be included in the process study, we wish to include both faith-based and secular community organizations, to the extent that both types are in close proximity to the respective intermediary. Further, in order to better understand how FBCOs experienced the service delivery of intermediaries, we wish to spend the site visit time with both organizations that were successfully engaged with the intermediary as well as those that may have struggled to make use of the intermediary services. Additional information about the circumstances of the FBCOs in close proximity will be obtained from the intermediary. When there are numerous faith-based and secular organizations that could be included in the process study, the three to be interviewed within a given location will be selected randomly. Because the sample of FBCOs is purposive, if the Executive Director and other key staff would not be able to be interviewed during the time period the visit is schedule, we will replace an organization selected with the next organization on the selection list.

B.2 Information Collection Procedures

We propose to collect information through semi-structured interviews following the proposed protocols. We will schedule interviews with the Executive Director or Project Director within each of the ten intermediary organizations and within the selected FBCOs. We will also schedule time with other key personnel who are familiar with the intermediary services provided or received within in each type of organization.

B.3 Methods to Maximize Response Rates

ACF expects to achieve a 100% response rate with the intermediary organizations because we have a good working relationship with each organization involved in the Impact Evaluation and the visits are scheduled at their convenience.

We achieved a 100% response rate on the baseline survey completed by the FBCOs in the research sample. Based on their understanding of the research study and cooperation to date, we also expect to receive a high response rate, at least 80%, among FBCOs in the process study. We expect to be able to reach agreement on schedules for interviews with at least 2 FBCOs, possibly 3, during each site visit.

B.4 Test of Procedures

The interview protocols have been drafted and reviewed by the contractor and ACF staff. The protocols include lines of questioning/discussion that have been used in the previous on-site visits with intermediaries and FBCOs in case studies that were completed as a part of another study. The protocols have been pre-tested with officials of intermediary and grassroots organization. The pre-tests indicated that the questions were clear to respondents and addressed organizational capacity development areas relevant to their experience.

B.5 Individuals Consulted on Statistical Aspects of Design

Design aspects of the process study were developed by Branch Associates Inc. in collaboration with Abt Associates, Inc. The team is led by Barbara Fink. Other research team members include, Cynthia Sipe. Contact information is provided below.

Barbara Fink and Cynthia Sipe

Branch Associates, Inc.

Philadelphia, PA 19019

215-731-9980

Branch Associates Inc. Part B: Collection of Information Employing Statistical Methods 3