Unhealthy Alcohol Use_OMB-SSB_5.28.20_final

The data collection described below is part of the multisite evaluation of the AHRQ Addressing Unhealthy Alcohol Use in Primary Care Initiative. The multisite evaluation will draw upon primary data collection by the evaluator in the form of semi-structured Key Informant Interviews and secondary data collected by six grantee teams working with primary care practices. Together, this data will be used to assess the efforts of the six grantees funded by AHRQ and in particular will evaluate: (1) the success of grantees in engaging and recruiting primary care practices for implementation of screening and brief intervention, referral to treatment, and medication-assisted therapy; (2) the effectiveness of the grantees’ collective dissemination and implementation strategies, and the factors associated with the success and/or failure of the strategies as they relate to populations, settings, and the influence of contextual factors; (3) success at the practice level in increasing the number of patients screened, identified, and treated; and (4) overall impact on changes in processes or outcomes that can be attributed to the initiative.

B1. Respondent Universe and Sampling Methods

Recruitment Methods, Respondent Selection, and Sample Size

The multisite, mixed-methods evaluation will include primary data collection by the evaluator, NORC at the University of Chicago. NORC will conduct an independent evaluation using this data as well as a core set of data collected by the six grantee teams working with 750 primary care practices and shared with the evaluator. Collectively the data will allow the evaluator to assess the implementation and impact of the six grants.

Primary Qualitative Data Collection by Evaluator

The evaluator will conduct semi-structured qualitative interviews with two groups of respondents: grantee teams and primary care practice staff, for a total of 60 respondents annually (See Exhibit B.1, below).

For grantee Key Informant Interviews, the evaluator will aim to recruit approximately five respondents from each of the six grantee teams for a total of 30 respondents. Respondents for the grantee team interviews will be selected in order to represent the key roles on the grantee teams, namely: principal investigator, co-investigator, evaluation lead, practice facilitator/implementation lead, and project manager. The evaluator will utilize existing relationships with the staff in these roles on grantee teams and in consultation with AHRQ to determine who will be interviewed.

For primary care practice Key Informant Interviews, the evaluator will aim to recruit approximately five respondents from one primary care practice selected from each of the six grantees (a total of six primary care practices) for a total of 30 respondents. Primary care practices will be selected by the evaluator in consultation with AHRQ and grantees. Respondents for the primary care practice staff interviews will be selected in order to represent key roles within primary care practices who are implementing the initiative, namely: one clinician champion, two frontline clinicians, one practice manager, and one non-clinical staff member (e.g., medical assistant, rooming staff). For the practices involved in qualitative interviews, the grantee teams will help identify staff members in these roles.

Exhibit B.1 Qualitative Semi-Structured Interview Respondents

Respondent Type	Key Roles Represented	Respondents
Grantee Teams	principal investigator, co-investigator, evaluation lead, practice facilitator/implementation lead, project manager	30
Primary Care Practice Staff	1 clinician champion, 2 frontline clinicians, 1 practice manager, 1 non-clinical staff (e.g., medical assistant, front/back office staff)	30
Total		60

Secondary Data Collected By Grantees & Analyzed By Evaluator

Grantees will be conducting their own data collection with primary care practices. Collected data will be submitted to the evaluator for analysis and evaluation of the following items: changes in the number of patients receiving screening, brief intervention, referral to treatment, and medication-assisted therapy; the relationship between practice characteristics and the number of patients receiving screening, brief intervention, referral to treatment, and medication-assisted therapy; the relationship between the type and frequency of practice interventions and the number of patients screened, receiving brief intervention, referral to treatment, or medication-assisted therapy; and changes over time in the activities practices conduct to identify and manage unhealthy alcohol use. Aggregate data from these instruments will be shared with the evaluator for analysis at the grantee and initiative-wide level; the evaluator will not be involved in the collection of this data.

Response Rates

Qualitative Key Informant Interviews

Based on our experience with previous qualitative recruitment efforts in health care organizations, we anticipate a 90 percent positive response to invitations for semi-structured, key informant interviews with grantee teams and primary care practice staff.

B2. Procedures for Collection of Information

This project consists of one primary qualitative data collection activity by the evaluator and will contribute substantively to the multisite.

Semi-Structured Qualitative Interviews. The evaluator will conduct in-person and/or telephone discussions with key staff from each grantee team (i.e., principal investigator, co-investigator, evaluation lead, practice facilitation/implementation lead, and project manager) and with clinicians and staff at one primary care practice working with each grantee. Key Informant Interviews will be conducted annually beginning at the end of Year 1, for a total of three time points per grantee. During Years 1 and 3 the interviews will be conducted by phone, while Year 2 interviews will be collected in-person. The interviews for both grantee teams and primary care practice staff will cover domains such as understanding the practice implementation and changes overtime, methods of supporting practices, barriers and facilitators to dissemination and implementation, strategies to overcome barriers, and the number and type of staff implementing screening, brief intervention, referral to treatment, and medication-assisted therapy. Semi-structured interview guides for grantee teams and primary care practice staff are contained in Attachments A and B, respectively. In order to test the validity and reliability of the interview guides, the evaluator conducted pilot testing with technical experts and project consultants. The evaluator conducted mock interviews of both practice and grantee interview guides to gain feedback on question comprehension and wording.

Evaluation Design

Qualitative Analysis. The project will conduct semi-structured Key Informant Interviews annually to better understand grantees’ approaches to disseminating and implementing patient-centered outcomes findings, primary care practices’ experiences with implementation, and barriers and facilitators to improving the management of unhealthy alcohol use in primary care. These interviews will occur three times during the three year study period and interviews will be coded for themes using NVivo qualitative data analysis software. Qualitative analysis will also be used to help understand differences in implementation across the grantees, and to supplement quantitative findings about changes in performance rates on the process measures.

B3. Methods to Maximize Response Rates

The qualitative data collection planned under this project is part of a cross-grantee evaluation to assess implementation of screening, brief intervention, referral to treatment, and medication-assisted therapy for unhealthy alcohol use in primary care practices. As a part of AHRQ’s unhealthy alcohol use initiative funding, grantees agreed to engage with the evaluator, including participating in telephone and in-person interviews and site visits. The grantees will assist in recruiting primary care practices that have indicated a willingness to participate for the practice site visits.

B4. Tests of Procedures or Methods to Be Undertaken

The Key Informant Interview Guides were developed by drawing on the evaluator’s past experience conducting research projects aimed at programs designed to reduce substance use and program evaluations involving qualitative data collection and analysis. They were also developed in collaboration with AHRQ and with input on feasibility collected from grantee teams to capture information specific to AHRQs Addressing Unhealthy Alcohol Use in Primary Care Initiative. The project will utilize regular phone meetings to gather feedback from the grantees and assess factors that facilitated or hindered implementation of the data collection.

B5. Statistical Consultants

NORC at the University of Chicago will serve as the primary consultant for statistical aspects of the design and analysis of the evaluation data. See Exhibit B.2 for a list of statistical consultants.

Exhibit B.2 List of Statistical Consultants

Name	Title and Institution
Weiwei Liu, PhD	Senior Research Scientist, NORC

The secondary data will be collected by grantees and shared with NORC at the University of Chicago for multisite analysis.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	NORC
File Modified	0000-00-00
File Created	2021-01-13