AFI FE part B collection and statistical methods 7 13 11

Part B: Collection of Information Employing Statistical Methods

B.1 Respondent Universe and Sampling Methods

The target population for the Survey of Financial Education Practices of AFI Grantees and the AFI Financial Education Cost Data Form are current recipients of AFI grant funds. These are primarily community-based nonprofit organizations, with some State, local, and tribal government agencies included. Specifically, 32 percent of AFI grantees are community action agencies (the largest category), 13 percent are community development corporations (the second largest category), and other kinds of nonprofit organizations make up most of the remaining grantee types. Just seven percent are State, local, and tribal government agencies.

Depending on the structure of each AFI project, the survey respondents may differ, although the grantee is responsible for completing the survey and cost form. We expect the grantees to enlist the help of the most knowledgeable respondents for their own financial education programs. In some cases, the grantee may ask for help from their partner agencies—such as housing counseling agencies or financial institutions—that actually deliver the financial education programs or asset-specific training. Also, some AFI grantees—known as “network grantees”—are coordinating agencies that pass the implementation and funding along to sub-grantees (ranging in number from two to 53¹). To gather accurate and detailed data on the financial education practices of network grantees, we think the most likely survey respondents for network projects will be the sub-grantees or partners working directly with program participants²

For the Survey of Financial Education Practices of AFI Grantees, we propose to include the entire population of active grantees in the sample (approximately 300). There will be no sampling of grantees for this survey.

For the survey to obtain AFI Financial Education Cost Data, we propose to select a sample of 30 grantees that are eligible for the survey. The population of grantees that is eligible for this survey are all the grantees that are in the fourth or fifth year of their AFI grant so that they have experience to answer the questions on financial education costs. This sample will be a probability sample.

Answering the questions on the AFI Financial Education Cost Data Form requires a fairly comprehensive level of financial information about the organization implementing the financial education or asset-specific training. Therefore, we expect the cost sample to be made up of both grantees and sub-grantees. Partner agencies are less likely to be appropriate respondents for this form.

B.1.2 Sampling Plan

As indicated earlier, there is no sampling of grantees for the Survey of Financial Education Practices of AFI Grantees.

Exhibit B-1 provides a tabulation of selected characteristics of the universe of AFI grantee organizations based on preliminary data collected from the 2008 Annual Data Report data collection. It shows the types of organizations currently providing financial education for AFI participants and offers some information on their size (measured by agency budget), the number of AFI grants currently in operation, and the number of asset-specific programs (which can include homeownership, business development, and post-secondary education).

Exhibit B-1: Characteristics of AFI Grantee Organizations

AFI Grantee Organization Characteristics	Single Grantees		Network Grantees
	Number	Percent	Number		Percent
Total Grantee Organizations, 2008a	204	100%	88		100%
Number of Current AFI Grants
One	141	69.1%	39		44.3%
Two or more	63	30.9%	49		55.7%
Type of Organization
Non-profit Community Action Agency	60	29.4%	29		33.0%
Government Community Action Agency	2	1.0%	4		4.5%
Community Development Corporation	22	10.8%	12		13.6%
Community Development Financial Institution	12	5.9%	6		6.8%
Credit Union	3	1.5%	0		0.0%
State/County/City Economic Development Agency	4	2.0%	1		1.1%
Educational Institution	2	1.0%	0		0.0%
Faith-based Organization	9	4.4%	4		4.5%
State/County/City Housing Agency	6	2.9%	3		3.4%
Non-Profit Housing Organization	16	7.8%	1		1.1%
Non-Profit Human Service Organization	30	14.7%	3		3.4%
State/County/City Human Services Agency	3	1.5%	1		1.1%
Micro-enterprise Development Organization	6	2.9%	0		0.0%
Refugee Resettlement Organization	2	1.0%	0		0.0%
United Way Organization	7	3.4%	14		15.9%
State/County/City Workforce Development Agency	2	1.0%	1		1.1%
Youth Development Organization	2	1.0%	2		2.3%
Other	12	5.9%	7		8.0%
Unreported	4	2.0%	0		0.0%
Number of Asset-Specific Priograms
One	38	18.6%	10		11.4%
Two	35	17.2%	2		2.3%
Three or more	127	62.3%	75		85.2%
Unreported	4	2.0%	1		1.1%
Grantee Organization Overall Budget
Under $1 million	48	23.5%	20		22.7%
$1 million – under $10 million	97	47.5%	36		40.9%
$10 million – under $50 million	40	19.6%	23		26.1%
$50 million and overb	12	5.9%	9		10.2%
Unreported	7	3.4%	0		0%
Mean	$48,145,519	-	$28,501,162		-
Median	$3,502,146	-	$3,944,309		-
Network Grantees’ Number of Partners (Subgrantees)
One	N/A	N/A	15	17.0%
Two-five	N/A	N/A	27	30.7%
Six-ten	N/A	N/A	11	12.5%
Eleven-twenty	N/A	N/A	12	13.6%
Over twenty	N/A	N/A	22	25.0%
MISSING	N/A	N/A	1	1.1%
Mean, all network grantees	N/A	N/A	14.5	-
Median, all network grantees	N/A	N/A	8	-
Mean, excluding grantees with > 20 subs	N/A	N/A	5.7	-

NOTES: a. Preliminary data from 2008 Annual Data Report collection.

b. These are primarily large public agencies (such as the Philadelphia Housing Authority and the City of San Antonio, TX).

From Exhibit B-1, we can see that the preponderance of single grantees have one current AFI grant, while the majority of network grantees have multiple grants. The mix of organization types differs, too; although community action agencies and community development corporations are large categories for both groups, human services organizations constitute a larger share of the single grantees, while United Way organizations are prominent among the network grantees. Also, more than a third of the single grantees offer only one or two asset-specific programs, while nearly all of the network grantees offer at least three.

Exhibit B-1 shows one very important similarity between the single and network grantee organizations. In terms of overall agency budget size, a quarter of each group consists of small entities (reporting under $1 million in total budget); another 41 to 48 percent of them manage budgets over $1 million but under $10 million. And in each group are a few large public agencies with budgets over $50 million. The similarity in size distribution between single and network grantees suggests that it is not necessary to further stratify the sample by agency size.

The last panel of Exhibit B-1 displays data on the number of partners reported by network grantees. This information is relevant to the cost data collection, because most network grantees have partners—sub-grantees—that deliver financial education to AFI participants using AFI grant funds. The sub-grantees will therefore need to be included in the sample for the Cost Data Form, as described further in Section B.2 below.

B.2 Procedures for the Collection of Information

B.2.1 Design

Based on the grantee characteristics shown in Exhibit B-1 and discussed in Section B.1 above, the proposed sampling plan is shown in Exhibit B-2. In the exhibit, the first panel shows the sampling for the Survey of Financial Education Practices of AFI Grantees. All grantees with current AFI grants will be asked to participate in the Survey of Financial Education Practices (approximately 300). The total number of current grantees may vary slightly from the 2008 count shown, as new AFI grants are awarded by OCS and old ones are completed.

Since OCS is requesting that the survey be mandatory, a 100 percent response rate is expected from the grantees. The results will produce statistically valid program-wide information on numerous aspects of financial education practices across the AFI projects.

The second panel of Exhibit B-2 shows the sampling plan for the proposed AFI Financial Education Cost Form data collection. OCS considers this research component to be exploratory, because nothing systematic is known yet about the full mix and scale of resources (beyond the AFI grants) supporting this vital work with AFI participants. Therefore, a small sample will be drawn, stratified between single and network grantees. The cost data effort will be focused on collecting complete and correct data from the sample.

Twenty single agency grantees and ten networks will be sampled with equal probability from within the universe of active AFI grantees that are in the fourth or fifth year of their grant. This ensures that the grantees selected will have had some experience in implementing their financial education programs before we ask about the costs of implementation. It is anticipated that this sample size will provide sufficient information on the variation in financial education delivery and funding patterns to allow reporting on the resources that support basic and asset-specific training.

Exhibit B-2: Sampling Plan for the Survey of AFI Grantee Financial Education Practices

Organization Characteristics	Single Grantees		Network Grantees
	Number	Percent	Number	Percent	Expected Response Rate
Total Grantees Organizations, 2008a	204	100%	88	100%	100%
Grantee Sample for the Survey of Financial Education Practices
Year One	204	100%	88	100%		100%
Sample for the AFI Financial Education Cost Data Formb
Grantees	20	9.8%	10	11.4%		100%
Sub-grantees	0	0.0%	42			80

B.2.2 Estimation Procedures

The grantee sample for the Survey of Financial Education Practices of AFI Grantees will consist of the entire current AFI grantee population. As indicated earlier, 100% response to the survey is expected. Therefore there will be no sampling weights to estimate the various parameters. Estimation will consist of computing the various population parameters for the whole population and subgroups.

For the AFI Financial Education Cost Form, appropriate sampling weights will be constructed based on the number of grantees in the population in stratum and the number selected in the sample. These weights will be used to obtain the estimates. One hundred percent response to survey is expected and therefore there will not be any nonresponse adjustment to the sampling weights.

For the intended descriptive analysis, most measures to be examined will be proportions, means, and differences of means. Standard errors of the estimates will be computed based on the sample size and the population size in each stratum. As indicated earlier the study is exploratory and not intended to produce precise population estimates because of the small sample size.

B.2.3 Degree of Accuracy Required

There are no subgroup comparisons involved.

B.2.4 Procedures with Special Populations

There are no special populations involved in the proposed data collection.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

Grantees sampled for the Survey of Financial Education Practices of AFI Grantees and the AFI Financial Education Cost Data Form will be notified by OCS or its contractor that they have been selected to provide information about their financial education programs. Grantees in the sample will be engaged and encouraged to respond through several media currently supported by OCS. Information about the survey and requests for completing it will be disseminated through the following channels:

• The AFI Resource Center, at IDAresources.org

• The weekly IDAresources Update e-newsletter, which can be accessed at http://www.acf.hhs.gov/programs/ocs/afi/resource_center.html

• AFI Resource Center Financial Education Practice Group conference calls, posted on the AFI Calendar of Events;

• Periodic Financial Education academies, also posted on the AFI Calendar of Events; and

• Other program communications from OCS to the AFI grantees.

Because of the existence of network grantees and the role of partner organizations in delivering financial education, we anticipate there could be multiple responses to the Survey of Financial Practices of AFI Grantees for some number of grantees; these will be identified through matching the information in the initial question (grantee name, name of partner or sub-grantee organization represented in the survey, AFI grant number(s) represented in the survey). Multiple responses about the same grantee will be merged for tabulations at the grantee level, but they will be useful as individual responses for tabulations at the program level.

Grantees sampled for the AFI Financial Education Cost Data Form will follow a different process to respond. This form requires more background information on the organization as a whole; it will also take longer to complete than the survey. Forms will be distributed electronically to all sample members, and two forms of assistance will be offered to help maximize response rates. First, the forms will be pre-populated with data for the grantee from the most recent Annual Data Report; this will let the respondents simply update those fields as necessary.

Second, the contractor to OCS for the AFI Financial Education Enhancement Initiative will actively engage all the cost form sample members and provide assistance to them in completing the cost form. The contractor will assist the grantee in identifying sub-grantee respondents (as needed) and will work with both grantee and sub-grantee organizations to help them respond.

B.4 Test of Procedures or Methods to be Undertaken

Both the Survey of Financial Education Practices of AFI Grantees and the AFI Financial Education Cost Data Form were pretested with nine grantees attending the financial training academies presented under the AFI Financial Literacy Enhancement Initiative.. These pretests allowed us to test the questions on grantees from both networks and single agencies, as well as with grantees from different agency types. The testing helped us clarify some of the question wording and build upon the response categories, as well as give us some timing estimates which were used in determining respondent burden.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The individuals shown in Exhibit B-3 assisted OCS in the statistical design of this data collection on AFI Grantee Financial Education Practices. OCS has contracted with Abt Associates, Inc. to conduct the data collection and analysis.

Exhibit B-3: Individuals Consulted on the Study Design

Name	Telephone	Role in Study
K.P. Srinath	301-634-1836	Sampling Statistician
Judith Feins	617-349-2370	Project Quality Advisor
Donna DeMarco	617-349-2322	Data Collection Task Leader

Inquiries regarding the statistical aspects of the study's planned analysis should be directed to Project Director Eliza Kean, Abt Associates, 301-634-1743.

1 See idaresources.org/files/netproj.projmodels.pdf, (downloaded November 2, 2009). A network project is “a form of AFI project in which the AFI grantee implements the project through formal relationships with one or more other entities that provide IDA services directly to participants. The AFI grantee applies for and is responsible for the Federal AFI grant funds, coordinates development of nonfederal funds and resources, manages the work of the sub-recipient organizations, and administers other key tasks and activities.” Defined at Terms:http://www.acf.hhs.gov/programs/ocs/afi/afi_network_project.html.

2 See Report to Congress: Assets for Independence Program, Status at the Conclusion of the Eighth Year (OCS, ACF, HHS), p. 15.

5

Section B Page 5 AFI Financial Education Practices and Cost Surveys OMB Justification Package July 13, 2011