Mai_mrt_ss_b 2024

Monitoring Data Collection Tools for the Minority AIDS Initiative (MAI)

Supporting Statement

PART B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B1. Respondent Universe and Sampling Methods

The Substance Abuse Mental Health Services Administration’s response universe for this data collection includes all active CSAP MAI grantees funded between now and 2025. In order to ensure accountability for the spending of federal funds, SAMHSA has employed the use of these data to ensure that grantees are meeting the goals and objectives of the grant.  Data are used to monitor performance throughout the grant period. The Public Health Service Act Sec. 501 [290aa] (d) (13) with respect to grant programs authorized under this title, assure that “all grants that are awarded for the provision of services are subject to performance and outcome data collections.” SAMHSA interprets these requirements to indicate the need for data to be collected on all grantees receiving federal funds in order to provide services.

B2. Information Collection Procedures

Both the quarterly progress report and participant/client level data will be submitted to SAMHSA via an online, web-based data entry system. Data collection procedures will be shared with grantees and grantees will be trained on how and when to submit data.

The Quarterly Progress Report

The Quarterly Progress Report (Attachment 1) is a modular instrument structured around the Strategic Prevention Framework (SPF). Each section or module corresponds to a SPF step with an additional section dedicated to cultural competence and efforts to address behavioral health disparities, which is an overarching principle of the framework. A section-by-section description of the instrument is provided in Exhibit 4 below.

Exhibit 4. Structure and Frequency of Biannual Progress Reports

Section	Contents	Frequency
Health Disparities	Activities implemented during the reporting period to address behavioral health disparities in the community Specific accomplishments and barriers related to health disparities	Biannual, as part of the second and fourth quarterly progress reports
Assessment	Synopsis of the needs assessment conducted by the grantee and key findings: High-risk groups identified Infrastructure and capacity enhancement targets Biannual accomplishments and barriers associated with needs assessment Biannual updates as needed Accomplishments and barriers related to planning	At least once during the needs assessment phase of the SPF, updated as needed
Capacity	Staff, advisory group, governing board, and collaborator rosters Advisory council meetings Training and technical assistance events Accomplishments and barriers related to capacity building	At least once during the capacity building phase of the SPF, updated as needed
Planning	Planned target population(s) Project goals and objectives Targeted outcome measures Planned direct-service interventions Planned indirect services (i.e. environmental strategies and information dissemination) Planned HIV/HCV testing and hepatitis vaccination activities Accomplishments and barriers associated with strategic planning	At least once during the strategic planning phase, updated as needed
Implementation	Numbers served through direct services and numbers reached through indirect services, by demographic group Grant expenditures Direct-service intervention implementation (timing, frequency, dosage, location, adaptations) Detailed list of direct services delivered Environmental strategies and information dissemination activities (i.e. indirect services) implemented Numbers referred outside the grantee organization for further services, by service type Outreach/recruitment activities Accomplishments and barriers associated with intervention implementation and service delivery	Quarterly during the implementation phase
Monitoring	Link to upload evaluation plan (online only) Accomplishments and barriers associated with evaluation	At least once during the evaluation phase

Before implementing any intervention, the grantees are required to enter planning information about that intervention in the Planning module. The Planning and Implementation modules are similarly structured to allow the evaluators to assess the degree to which the implementation followed the plan (i.e. fidelity to the plan). The Implementation section provides each grantee with a dropdown menu consisting of the interventions they entered in the Planning section. This dynamically updated grantee-specific dropdown menu is also linked to the online participant-level instruments in order to link each participant’s pre-post and follow-up data to the direct-service intervention(s) they received.

Youth and Adult Outcome Questionnaires

Two common questionnaires will be administered to direct-service program participants. The Youth Questionnaire is designed for persons aged between 12 and 17 and the Adult Questionnaire is designed for persons aged 18 and older. For all common measures, administration guides are under development to assist program sites with administering and proctoring the surveys.

The major constructs for the youth outcome questionnaire include demographics, 30-day substance use, disapproval of peer substance use, perception of risk of substance use, perception of peers’ risky behaviors, sexual behavior, sexual self-efficacy, school connectedness, and knowledge of HIV (See Attachment 3 for a copy of the Youth Questionnaire).

The major constructs for the adult questionnaire include demographics, employment, 30-day substance use, perception of risk of substance use, sexual behavior, sexual self-efficacy, availability of social/emotional support, perception of risk of unprotected sex, knowledge of HIV, and perception of peers’ risky behaviors (See Attachment 2 for a copy of the Adult Questionnaire).

Data Collection

The last column of Exhibit 4 indicates the frequency with which each section (or module) of the Quarterly Progress Report will be completed. Data collection will start with a baseline data submission preceding the onset of funded activities and will be updated every year during the grant period. The Adult and Youth questionnaires will be administered to participants at baseline, exit, and 90-day post exit follow-up.

Data Management

Storage System: The Quarterly progress report and client level instruments are submitted to SAMHSA’s online data entry system allowing grantees to enter item-by-item or to upload in batch file format all completed instruments. To ensure data quality and minimize data cleaning efforts, all online instruments include online validation checks that issue error messages and suggestions for correcting data entry errors such as inconsistencies or out-of-range values.

B3. Methods to Maximize Response Rates

Issues related to response rates, as well as other data collection issues, are discussed at grantee meetings in order for project officers to identify problems and provide technical assistance. In addition, project officers monitor data collection efforts and provide technical assistance to individual grantees as necessary. Because collection of these data is a stipulation of the grants, it is anticipated that all grantees will comply (as appropriate). The participants at each site to whom these measures will be administered are all voluntary respondents; therefore, grantees cannot guarantee full cooperation on the part of participants. Historically, however, participant response rates at exit across grantee sites have averaged around 75 percent. Due to the high-risk nature of the participants, a substantial proportion have historically been transient populations without permanent residence (e.g., homeless, unemployed, or reentry individuals). Post-exit follow-up rates have been around 35 percent. SAMHSA and its contractors continue to provide training and technical assistance to grantees for enhancing these response rates.

B4. Tests of Procedures

Prior to its implementation, CSAP and its contractor reviewed the Youth and Adult Questionnaires as well as the quarterly progress report to clarify terminology and language and eliminate unnecessary questions.

ATTACHMENTS

1. MAI Quarterly Progress Reporting Tool

2. Adult Questionnaires (English and Spanish versions)

3. Youth Questionnaires (English and Spanish versions)

2