SPF-SIG Community Outcomes_SS-B-3_3_16

Strategic Prevention Framework State Incentive Grant (SPF SIG) Program: Community Outcomes

Supporting Statement

Part B. Collections of Information Employing Statistical Methods

B1. Respondent Universe and Sampling Methods

SAMHSA’s CSAP funded 24 SPF SIG grantees in July 2009 (Cohort IV), which included 12 states, the District of Columbia, 6 tribal entities, and 5 jurisdictions, and 10 grantees in October 2010 (Cohort V), which included 3 states and 7 tribal entities. SPF SIG grantees select, assist, and support selected community subrecipients (i.e., communities that receive SPF SIG funds from the Cohort IV and V grantees) to implement effective programs, policies, and practices to reduce substance use and its related problems. Each grantee can pursue diverse strategies and methods for selecting the community subrecipients in their areas. An average of 10-15 community subrecipients are awarded by each grantee.

A census of all Cohort IV and V grantees is necessary because these data will be used by SAMHSA to assess program performance. SPF SIG grantees encompass a wide variety of organizational types and structures that are implementing a range of prevention interventions targeted to different populations and with various outcome goals. The variety between grantee initiatives makes it critical to the cross-site evaluation to assess which initiative characteristics and interventions (already captured through the current OMB-approved instruments) are associated with better outcomes for particular types of communities.

Submission of community outcomes data will be required of all active Cohort V SPF SIG grantees on behalf of their active community subrecipients. Cohort IV grantees will be asked to voluntarily provide the data. Most Cohort IV grantees have indicated willingness to provide the data to the cross-site evaluation, because the data are generally those they have gathered from existing administrative and survey datasets for the purposes of their own grantee-level evaluations. The estimated response rate for this activity across Cohort IV and V is 65 percent (22 of 34 grantees), because some grantees have indicated a lack of available data.

B2. Information Collection Procedures

To initiate collection of community outcomes data, the contractor will contact each Cohort IV and V SPF SIG Project Director and Evaluator by e-mail to request submission of the data via a spreadsheet template. The spreadsheet will be pre-populated with any community outcomes data that the contractor has in our cross-site database. The Excel format will allow grantees to copy and paste information across multiple subrecipients with common data elements (e.g., data source name, survey item wording). Closed-ended fields will contain drop-down menus that allow for the quick selection from among response options. Given the non-confidential nature of aggregate community outcomes data, grantees will be asked to return the data entry spreadsheet to the contractor via email.

Grantees will have access to technical assistance in completing the spreadsheets. The contractor’s online data system that is used to collect the other cross-site evaluation instruments includes a “Help” or “Support” link, which will allow the respondent to access the following support resources:

1. The Knowledge Base. Community outcomes guidance materials will be available on the Knowledge Base to assist grantees in populating the spreadsheet.

1. Contact Us. Respondents may request assistance by calling a provided toll-free number, sending an email request, or submitting a technical assistance submission form as desired. The toll-free line will be routed to an email system that is checked daily by members of the training and technical assistance team. Staff responding to technical assistance requests will be trained in the instrument and have ready access to the community outcomes guidance materials on the Knowledge Base. Training and technical assistance team staff will monitor all submitted tickets to ensure timely response and resolution of technical assistance requests.

Over the two-month data collection period, grantees will receive follow-up contact by email from the technical assistance team to ensure that all questions are answered and that grantees are reminded of the submission deadline.

B3. Methods to Maximize Response Rates

All Cohort V grantees will be required to provide community outcomes data. We anticipate that, the response rate will be less than 100 percent, however, as not all tribal grantees had available data sources or conducted primary data collection for their evaluations. For Cohort IV grantees, although submission of community outcomes data will be voluntary, most have indicated willingness to provide data as well as availability of existing data. The estimated response rate for this activity across Cohort IV and V is 65 percent (22 of 34 grantees). The follow-up procedures, described in the preceding section, will increase the likelihood that a very high percentage of grantees will respond.

B4. Tests of Procedures

As part of the broader contract, three staff completed a test version of a community outcomes instrument that is very similar in content to the one proposed in this package. These staff members have experience with SPF initiatives, including serving as local evaluators for SPF SIG grantees.

The Community-Level Outcome Data for Subrecipients instrument is estimated to take 4 hours to complete; this includes 3 hours to look up and compile information and 1 hour to complete the web instrument.

B5. Statistical Consultants

The contractor team comprises several experts who will be directly involved in data collection and statistical analysis. Also, contractor in-house experts will be consulted throughout the program on various statistical aspects of the design, methodological issues, and data analysis, including leveraged funding analysis. Finally, the project has an External Steering Committee. Members of this External Steering Committee have already provided feedback on the instruments and the evaluation/analysis plan and will continue to provide advice and feedback through scheduled quarterly meetings and ad hoc e-mails as needed. Table 3 provides details of these team members and advisors.

Table 3. Statistical Consultants for the Program Evaluation for Prevention Contract

Name & Role in Evaluation	Title & Address	Contact Information
Evaluation Staff
Elvira Elek, PhD Deputy Director	Research Public Health Analyst Public Health Policy Research RTI International 701 13th Street, NW, Suite 750 Washington, DC 20005	Telephone: (202) 728–2048 Email: eelek@rti.org
Phillip Graham, PhD Project Director	Senior Research Public Health Analyst Crime, Violence, and Justice Program RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709	Telephone: (919) 485–7752 Email: pgraham@rti.org
Lori-Ann Palen, PhD SPF SIG Cross-Site Evaluation Co-Leader	Research Associate Risk Behavior and Family Research Program RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709	Telephone: (919) 541-6129 Email: lpalen@rti.org
Gillian J. Leichtling SPF SIG Cross-Site Evaluation Co-Leader	Senior Research Associate RMC Research Corporation 111 SW Columbia Street  Suite 1030  Portland, OR 97201-5883	Telephone: (503) 223-8248 x735 Email:GLeichtling@rmccorp.com
Antonio Morgan-Lopez, PhD Analysis Team Leader	Senior Research Quantitative Psychologist Risk Behavior and Family Research RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709	Telephone: (919) 316–3436 Email: amorganlopez@rti.org
Michelle Hendricks, PhD Senior Analyst	Research Associate RMC Research Corporation 111 SW Columbia Street  Suite 1030  Portland, OR 97201-5883	Telephone: (503) 223-8248 x760 Email: mhendricks@rmccorp.com
Steve Sullivan ESC Task Team Leader	Senior Director Cloudburst Consulting Group, Inc. 8400 Corporate Drive, Suite 550 Landover, MD 20785-2238	Telephone: (301) 918-4400 Email: steven.sullivan@cloudburstgroup.com
Government Project Officers
Sara Azimi-Boularian, PhD Contracting Officer’s Representative	Senior Public Health Analyst CSAP, SAMHSA 1 Choke Cherry Road, Room 4-1025 Rockville, MD 20857	Telephone: (240) 276-2708 Email: Sara.Azimi-Bolourian@SAMHSA.hhs.gov
Thomas Clarke, PhD Alternate Contracting Officer’s Representative	Senior Public Health Analyst CSAP, SAMHSA 1 Choke Cherry Road, Room 4-1031 Rockville, MD 20857	Telephone: (240) 276-0493 Email: Thomas.Clarke@samhsa.hhs.gov
External Steering Committee
Bethany Bray, PhD Methods/Statistics	Research Associate The Methodology Center The Pennsylvania State University 400 Calder Square II State College, PA 16801	Telephone: (814) 865-1225 Email: bcbray@psu.edu
William DeJong, PhD Evaluating Environmental Strategies	Professor Boston University School of Public Health Community Health Sciences 801 Mass Ave Crosstown Center Boston MA 02118	Telephone: (508) 954-0224 Email: wdejong@bu.edu
Brian Flay, DrPH Prevention Science	Professor Oregon State University College Of Public Health and Human Sciences 457 Waldo Hall Corvallis, OR 9733	Telephone: (541) 737-3837 Email: Brian.Flay@oregonstate.edu
Rick Harwood Economics, Cost Analyses	Director of Research and Program Applications National Association of State Alcohol and Drug Abuse Directors, Inc., (NASADAD) 1025 Connecticut Avenue NW, Suite 605 Washington, DC 20036	Telephone: (202) 293-0090, ext. 104 Email: rharwood@nasadad.org
Dottie Natal IT, Data Collection Systems	CEO Imagen Multimedia Corp	Email:dottie@imagenmm.com
Chris Ringwalt, DrPH Intervention Implementation and Dissemination	Public Health Senior Research Scientist Pacific Institute for Research and Evaluation 1516 E. Franklin Street, Suite 200 Chapel Hill, NC 27514-2812	Telephone: (919) 259-0643 Email: ringwalt@PIRE.org;

Attachments:

A: SPF SIG National Outcome Measures

B: Community Outcomes Data Fields

4