Assessing
Care and Health Outcomes Among Ryan White HIV/AIDS Program (RWHAP)
Clients Who Do Not Receive RWHAP-Funded Outpatient Ambulatory Health
Services (OAHS)
OMB SUPPORTING STATEMENT
PART B
OMB
Control No. 0915-XXXX
Collections
of Information Employing Statistical Methods
November
2019
Submitted By:
HIV/AIDS Bureau
Health Resources and Services Administration
U.S. Department of Health and Human Services
5600 Fishers Lane
Bldg: 09N190B
Rockville, MD 20857
Project Officer: Robert Mills
CONTENTS
B.1. Respondent universe and sampling methods 2
B.2. Procedures for collection of information 3
B.3. Methods to maximize response rates and data reliability 10
B.4. Tests of methods or procedures to be undertaken 12
B.5. Individuals consulted on statistical aspects and individuals collecting and/or analyzing data 13
The HIV/AIDS Bureau (HAB) within the Health Resources and Services Administration (HRSA) in the Department of Health and Human Services (HHS) is embarking on a 24-month qualitative and quantitative study called Assessing Care and Health Outcomes Among Ryan White HIV/AIDS Program (RWHAP) Clients Who Do Not Receive RWHAP-Funded Outpatient Ambulatory Health Services (OAHS). The goal of the RWHAP is to improve the availability and quality of health care and supportive services for low-income and medically underserved individuals and families living with HIV and AIDS. The study focuses on the quality of care and viral suppression rate of non-OAHS RWHAP clients. The findings from this study will help us assess HIV care and health outcomes among RWHAP clients who do not receive OAHS RWHAP-funded, identify where these clients receive HIV care services, identify unmet needs among these clients who do not receive OAHS anywhere, and collect clinical data for these clients so HAB can estimate and improve retention rates and viral suppression for all RWHAP clients.
For this study, we propose conducting (1) semi-structured interviews with program managers, medical directors, case managers and other frontline clinicians, and clients and (2) medical record abstractions. The recipient/subrecipient, non-OAHS funded provider, and client interviews will provide rich, nuanced data on HIV care use, HIV-related outcomes, barriers to care, and potential opportunities to improve care and HIV-related outcomes. Through medical chart abstractions, we will obtain clinical information on a representative sample of clients who do not receive RWHAP-funded OAHS. We will use the information to estimate three key outcomes for this population: (1) retention in care, (2) initiation of antiretroviral therapy (ART), and (3) viral suppression.
There are three scenarios in which a RWHAP client could receive some services through the RWHAP but not receive OAHS directly funded through the RWHAP; we refer to these as Scenarios A–C below and in Table 1. Throughout this statement, we refer to the RWHAP-funded provider as the Type 1 provider and the non-funded OAHS provider as the Type 2 provider.
The Type 1 provider does not offer medical services. In this scenario, we assume that clients either receive their medical care from a Type 2 provider and their support services from the Type 1 provider or that they are not receiving medical care at all. We can observe this scenario in the RWHAP Services Report (RSR) by looking at RWHAP-funded providers who report delivering support services or non-OAHS core medical services, but do not report providing OAHS.
The Type 1 provider delivers medical care but is not directly funded for OAHS. In this scenario, we assume that most clients receive their medical care on-site (but the clinical data are not reported to HAB because the provider is not directly funded for OAHS) or that clients are not receiving medical care at all. We will use the provider type variable in the RSR to identify medical providers (such as hospital-based or community health centers) that do not report furnishing OAHS.
The Type 1 provider is directly funded for OAHS, but only some of its clients receive OAHS on-site. In this scenario, we assume that clients either receive their medical care from a Type 2 provider and their support services from the Type 1 provider or that they are not receiving medical care at all. For OAHS-funded providers with non-OAHS clients, we will assess the number of clients who did versus did not receive RWHAP-funded OAHS based on the “OAHS service visits delivered” client-level item in the RSR.
Table 1. Three scenarios under which RWHAP clients do not receive OAHS directly funded through the RWHAP
Type |
Scenario A |
Scenario B |
Scenario C |
Type 1 |
RWHAP
provider |
RWHAP
provider |
RWHAP
provider |
Type 2 |
Non-funded OAHS provider (medical) |
Non-funded OAHS provider (medical) |
In this study, we will select a nationally diverse convenience sample of RWHAP-funded recipients and subrecipients (providers) that do not provide some or all of their clients with RWHAP-funded OAHS and then a stratified random sample of clients within those sampled providers, who do not receive directly RWHAP-funded OAHS. A recipient is a provider organization that receives RWHAP funds directly (usually through Parts C and D of the program). A subrecipient is a provider organization that receives RWHAP funds indirectly through a subcontract with the city or state where it is located (usually through Parts A and B of the program). Some provider organizations are both direct recipients and indirect subrecipients of RWHAP funding.
We will sample and recruit RWHAP providers that either are not funded for OAHS or are funded for OAHS but do not provide this service to all of their clients. For these providers, we assume that some, if not all, of their clients receive medical care from a non-OAHS funded provider. We will gather information on the care that these non-OAHS clients receive through interviews with program managers, clinicians, and case managers or other frontline staff. If the clinical information needed to estimate the core study outcomes is available in case management notes, the study team will attempt to recruit the RWHAP provider to participate in the study. In this case, we will instruct the provider staff to abstract medical record data for a random sample of clients (or offer onsite or remote abstraction by the study team, if preferred). If the RWHAP provider does not have the relevant clinical notes, we will ask the provider to refer our study team to the clients’ non-OAHS funded medical provider(s) associated with a random sample of clients.
We will not know the number of non-OAHS RWHAP providers in the respondent universe until we construct the sampling frame. We will construct the provider sampling frame using data from the 2017 RWHAP Service Report (RSR). To do this, we will first identify the relevant clients (that is, those who received only a non-OAHS RWHAP-funded service from a RWHAP provider) and then identify the recipient or subrecipient organizations they go to for RWHAP-funded services. To ensure we have sufficient power to produce estimates of process and clinical outcomes, we will exclude sites with fewer than 25 non-OAHS clients from the sample frame. We will sample 30 non-OAHS RWHAP providers as described in section B.2.1.
Within each selected and participating recipient or subrecipient, we will seek access to the clinical data for their RWHAP clients for whom we do not already have clinical data in the RSR. If the RWHAP provider collects or has access to the relevant clinical information, we will abstract the information from the sampled provider organization (or instruct the site to do so, if preferred). If the RWHAP does not have access to this information, we will attempt to obtain the data from the non-OAHS funded medical providers where their clients go for medical care. If a provider has more than 50 non-OAHS clients, we will select a stratified random sample of 50. We will sample clients as described in section B.2.1.
We expect that most of the sampled RWHAP recipients or subrecipients will have access to the encrypted unique client IDs (eUCIs) that we use and report in the RSR. (If a Type 1 provider does not maintain the eUCIs in its records, we will offer assistance in generating these numbers based on its clients’ name and demographic characteristics.) For these providers, we will use the eUCI and demographic information on the RSR to identify (and sample if needed) the non-OAHS clients associated with each selected provider. It is possible that the RWHAP recipient or subrecipient will have access to their clients’ clinical data through an affiliated clinic (and a shared health record system) or in their own case management records. If the RWHAP providers do not collect or have access to the relevant clinical data, we will ask the provider to refer and introduce our study team to the sampled clients’ non-OAHS funded medical provider(s). Because the non-OAHS medical provider is unlikely to have access to the clients’ eUCIs, we will ask the sampled RWHAP provider to prepare a list that includes the first and last name, age, and gender associated with the eUCI. We will ask the RWHAP provider to send the list of names to the non-OAHS medical provider(s) where the client goes for medical care. This list will remain with the RWHAP and non-OAHS funded providers and accessible to the study team only to verify the medical records abstracted (on-site or by the non-OAHS practice, if preferred).
After constructing a list of Type 1 RWHAP providers that meet the threshold criteria, we will select a stratified random sample of providers based on the characteristics shown in Table 2. We will explicitly stratify the sample frame by (1) whether or not the provider is located in a state that expanded its Medicaid program after implementation of the 2010 Affordable Care Act and (2) the non-OAHS scenarios previously described in this statement. We believe that the reason for not receiving OAHS under the RWHAP will vary between states that expanded Medicaid versus those that did not. States that expanded Medicaid are likely to have alternative sources of care and treatment and better clinical outcomes among their underserved residents. For sampling purposes, we will also collapse the three scenarios into two groups: nonmedical RWHAP providers (Scenario A) and medical RWHAP providers (Scenarios B and C). We propose to collapse Scenarios B and C into a single stratum because they both represent medical providers and thus their non-OAHS clients are likely to have better access to medical care and treatment than those who are not linked to a RWHAP-funded medical provider. We will allocate the sample based on the number of providers in each type of state and scenario stratum.
Table 2. Sampling strata for Type 1 RWHAP providers
Potential sampling strata |
Stratification method |
Categories |
Medicaid expansion state |
Explicit |
Expansion versus nonexpansion state |
OAHS scenario |
Explicit |
|
Provider type |
Implicit |
Based on categories reported in RSR |
Given the small number of providers in the study, we will implicitly stratify within the explicit strata in two ways. First, for those in the Scenario B and C group, we will implicitly stratify by scenario. Second, for all groups, we will implicitly stratify by the type of provider organization, as reported in the 2017 RSR. Implicit stratification refers to the sorting of the sampling frame by certain characteristics before sampling so that when a systematic or sequential sample is drawn from that frame, the resulting sample’s distribution will resemble that of the frame on those characteristics.
Finally, we will select an augmented sample of 90 Type 1 RWHAP providers and then initially release a random subsample of 30 to recruit for the study. Should any provider refuse to participate in the study or be found ineligible during the initial telephone screening (discussed in Section II.C), we will release additional sample members from the same stratum in random order until we have recruited 30 for the study. We will set up an electronic system to track the final status of all sampled providers. This will help with study operations and with generating response rates and analysis weights at the end of the study.
Screening for eligibility. After we select the subsample of 30 Type 1 RWHAP providers, we will conduct a pre-enrollment telephone screening with the administrator of each provider organization to confirm that the sampled provider meets the conditions of participation.1 We will send the screener to the providers ahead of the screening so they can have data-driven answers ready when we call. If the provider is ineligible or unwilling to participate, we will conduct the same screening with a replacement provider. Specifically, during the call, we will seek to:
Confirm that the categories of services offered by the Type 1 provider (for example, OAHS, other core medical services, or support services) are consistent with those reported in the RSR. Services that would otherwise fall into RWHAP service categories might not appear in the RSR if the provider uses other sources of funding (for example, rebate dollars, program income, or external funding sources).
Determine whether the provider knows where its non-OAHS clients go for medical care and, if the provider does know, identify the Type 2 providers and obtain an estimate of the number of their non-OAHS clients who go to each one.
Assess the feasibility of medical record abstraction of non-OAHS clients. We will determine whether the sampled Type 1 provider collects and maintains records to track retention in care, prescription of ART, and viral suppression for non-OAHS clients. If not, we will assess the Type 1 provider’s relationship with non-funded OAHS providers and whether they think these Type 2 providers would be willing to perform medical record abstractions (or give the study team access to the clinical records of their OAHS clients, if preferred).
We will use the information collected through the pre-enrollment telephone screening to determine whether the sampled Type 1 provider is eligible for the study. Table 3 shows our criteria and how we will apply them. If the sampled provider serves at least 25 non-OAHS clients, collects the data elements needed for this study, and can participate in medical record abstraction, we will recruit that organization. However, if the sample Type 1 provider does not have this information, we will restrict recruitment to Type 1 providers that (1) know where a minimum of 20 of their non-OAHS clients go for medical care, (2) have at least 10 of their clients going to the same Type 2 medical provider, and (3) believe that the Type 2 provider will participate in medical record abstraction for the OAHS clients. We will visit up to two Type 2 providers per site visit.
Table 3. Criteria for recruiting sampled providers to participate in study
Inclusion criteria |
If yes |
If no |
Serves at least 25 non-OAHS clients |
Move to criterion #2 |
Terminate recruitment |
Collects needed service and clinical data elements |
Initiate recruitment |
Move to criterion #3 |
Knows where at least 20 non-OAHS clients go for medical care |
Move to criterion #4 |
Terminate recruitment |
Has at least 10 non-OAHS clients who see the same Type 2 medical provider |
Move to criterion #5 |
Terminate recruitment |
Believes it is feasible to collect data from the Type 2 medical providers |
Initiate recruitment |
Terminate recruitment |
If a sampled Type 1 provider does not meet these criteria, we will document the reason for exclusion for future analysis and select a replacement sample member from the same stratum to screen and recruit. We will compare the demographic characteristics of clients served by excluded Type 1 providers with those of clients served by included Type 1 providers to assess the potential selection bias caused by these exclusion criteria. If we observe systematic differences between clients from included and excluded Type 1 providers, we will weight the records to account for this difference.
If a recruited Type 1 provider has more than 50 non-OAHS RWHAP clients, we will select a stratified random sample of 50 for chart review and abstraction. Otherwise, we will include all of the Type 1 provider’s non-OAHS clients who receive medical care for their HIV disease in the study. Using information from the 2017 RSR, we will implicitly stratify clients by one or more of the characteristics in Table 4. These include HIV risk group, age group, gender, and race/ethnicity. We will finalize our sampling strategy after reviewing the distribution of non-OAHS clients across these characteristics.
Table 4. Potential sampling strata for RWHAP clients
Potential sampling strata |
Stratification method |
Categories |
HIV risk group |
Implicit |
MSM only, IDU only, MSM + IDU, not MSM or IDU |
Age group (years) |
Implicit |
18–24, 25–54, 55+ |
Gender |
Implicit |
Male, female, transgender |
Race/ethnicity |
Implicit |
White, black, Hispanic, all other |
Medical record data will be abstracted for a maximum of 50 non-OAHS clients per Type 1 provider. First, we will select an augmented sample of 75 non-OAHS clients to allow for missing data at the client level, initially releasing a random subsample of 50 and additional clients from the same strata as needed. There could be missing data if the information is unavailable or turns out to be incorrect or outdated, or if the Type 2 provider of those services refuses to participate in the medical record abstraction required for this study. If we cannot locate (or do not accept) an individual client’s medical records, we will randomly select a replacement client from the same provider.
We will continue with this procedure until medical record data have been abstracted for 50 non-OAHS clients per Type 1 provider (or all of its clients if it has fewer than 50). As noted, the non-OAHS clients could be distributed across multiple Type 2 providers, but we will limit chart reviews to two providers, and those two must serve a minimum of 10 non-OAHS clients. We will set up a system to track the status of medical record abstraction for all sampled clients.
After data collection, we will construct analysis weights that account for the probability of selection of each provider and client, for provider nonparticipation, and for missing client information within participating providers. These weights will mitigate the risk of bias in our estimates, given differential selection probabilities and differential patterns of nonresponse across the sample. The approximately 1,500 clients in the study sample (30 providers multiplied by 50 clients each) should be similar to the population of RWHAP clients in those recipient and subrecipient providers who do not provide—or whose clients do not receive—RWHAP-funded OAHS. However, since the study is based on convenience samples of clients and providers, the client- and provider-level estimates will be treated as exploratory. We will conduct sensitivity tests to assess the impact of the weights on the results. Given the difficulty of identifying a truly nationally representative sample for this study, we will describe the sample as a nationally diverse convenience sample and we will provide both unweighted and weighted counts in all written documents resulting from this study.
Because we will have only 30 providers in the sample, any provider-level estimates will have large sampling errors and therefore will not provide sufficient precision for estimating the measure in the population of providers. However, the 1,500 clients in the sample should provide relatively precise estimates of the roughly 130,000 clients in the study-eligible population. Because these 1,500 clients are clustered within a sample of only 30 providers, the effective sample size will be substantially less than the nominal sample size of 1,500 when it comes to measuring precision. The extent to which the clustering impacts the effective sample size depends on how homogeneous a particular outcome measure is within providers relative to the overall variance across providers (indicated by the intraclass correlation coefficient, or ICC). Table 5 shows the 95 percent confidence intervals for two outcome variables, both proportions of around 0.5. One variable has a relatively low clustering effect (ICC = .01) and one has a relatively high clustering effect (ICC = .10). The design effects show the impact of the sampling (S), clustering (C), and nonresponse weighting (W) adjustments on the variance of estimates. A design effect of 1 would indicate no impact on the variance. A design effect of 2 would indicate that the design feature doubles the variance and effectively halves the effective sample size.
Table 5. 95 percent confidence intervals for client-level estimates (outcome proportion of 0.5)
Providers |
Clients/ |
Clients |
ICC |
Design effects |
Effective number of clients |
Half-width confidence interval |
Lower bound |
Upper bound |
||
S |
C |
W |
||||||||
30 |
50 |
1,500 |
0.01 |
1.1 |
1.49 |
1.1 |
832 |
.034 |
.466 |
.534 |
30 |
50 |
1,500 |
0.10 |
1.1 |
5.90 |
1.1 |
210 |
.068 |
.432 |
.567 |
S = sampling adjustment; C = clustering adjustment; W = nonresponse weighting adjustments.
As described above, we might need to tailor our client-level sampling procedure to different types of providers, depending on what client identifiers are available for this study.
We do not plan to break the data collection into cycles to reduce burden. We plan a single data collection cycle for each provider and client in the sample. We will begin recruitment in November 2019. We will begin semi-structured interviews and medical record abstraction in January 2020 and end by June 2020.
Semi-structured interview design. A key component of the study will be interviews with Type 1 provider staff (administrative and frontline service staff for a RWHAP provider), Type 2 provider staff (the medical director or other clinical staff at a non-funded OAHS provider), and clients. These interviews will provide rich, nuanced data on the use of HIV care, HIV-related process and clinical outcomes, barriers to care, and potential opportunities to improve medical care and HIV-related outcomes.
Semi-structured interviews will be conducted via telephone, unless a provider site requests on-site interviews. A qualitative researcher will coordinate and conduct the interviews. The medical record abstraction team will work with the practice site to determine the optimal means of data abstraction—on-site by the study team, remotely by the study team, or by the practice itself—and provide necessary logistical support and instruction .
The semi-structured interview respondents will vary depending on the three non-OAHS scenarios described earlier in this statement, but in general, we will speak with the RWHAP program administrator, the frontline service delivery staff (for example, the case manager or mental health professional), and the physician or other medical provider, as well as up to four clients at each Type 1 provider site. At the Type 2 provider sites, we will limit our interviews to the medical provider. We will schedule 30-minute interviews with clients and 30- to 60-minute interviews with all other respondents, depending on staff position and expected knowledge of the use of HIV care and outcomes for non-OAHS clients. We will rely on the Type 1 provider to help us identify and connect with the non-OAHS clients and with the Type 2 providers.
The qualitative researcher will conduct most interviews via telephone, unless an on-site visit is requested by the provider. We expect to interview each respondent individually in order to ensure we hear multiple perspectives independently and to encourage the respondent to speak freely. However, if a provider site prefers to include multiple respondents in the same interview, we will accommodate their request.
When conducting interviews, we will request permission to record the conversation. There might be occasions when respondents, particularly clients, do not grant this permission. In these cases, the interviewer will take handwritten notes during the interview. We will also describe our process for protecting confidentiality to each interviewee. Namely, during the project, only staff working on the qualitative data collection and analysis task will have access to the data, and we will destroy all notes, recordings, and transcripts of interviews at the end of the project. We will not link interviewees’ names to their responses or clinical information, and we will summarize the information in aggregate only.
Chart abstraction design. Prior to chart abstraction, we will ask all Type I RWHAP providers to post a Study Notification Sign in one or more well-traveled areas of their physical location (Appendix D). The Study Notification Sign will alert patients to the fact that their medical data may be abstracted as part of this study and that any information resulting from the abstraction will be kept confidential, and findings presented in aggregate form only.
Data will be abstracted from medical records at 30 provider sites. Data abstraction will done by the provider, with instruction from the study team, or will be done on-site or remotely by the study team, if preferred. Key data elements for up to 50 clients will be abstracted. Key data elements include year of initial HIV diagnosis, dates of all outpatient medical visits, dates and types of HIV medications prescribed, and dates and results of all viral load lab tests ordered. We will limit the extraction to visits, prescriptions, and tests occurring from January through December 2018. We will pre-populate the abstraction tool with a list eUCIs for the clients for whom data will be collected. We will also pre-populate the tool with the client’s age and gender from information reported in the 2017 RSR.
We consider each medical record a single case. The abstractor will confirm that the client in the medical record is the same client as was sampled from the RSR. This will be done by manually matching on client eUCI (or name for Type II provider), age category, and gender. For those records confirmed to match, medical record data will be abstracted. The reviewers will check all of the case-related encounters during the reporting period to determine the presence or absence of each specified data element to be abstracted.
When expected components of the patients’ records are not available for review, the abstractor will review the case with the abstraction team lead and document the specifics in the notes field of the abstraction tool. The abstractor will also report any questions about validity of information to the abstraction lead. The abstraction lead will discuss the issue with the study team and help the provider site resolve data discrepancies. Follow-up will include a reference to the expected but missing or questionable client record information to minimize incomplete data collection and eliminate missing data related to information that was inadvertently not supplied.
After medical record abstraction is complete, the abstractor will upload the requested data files to a secure, site-specific document library dedicated to this study. We will develop and post instructions for abstractors to upload the data elements in the secure, restricted electronic data storage folder. We will build in automated checks and prompts during data entry and dropdown menus of expected data values. This will help prevent data entry errors such as invalid values. The abstraction lead and the abstractor will then meet for a data review debriefing and, if indicated, the abstractor will be asked to clarify his or her results. Once we have processed the site data, we will transfer them to our most secure electronic storage with access granted to only those staff responsible for analyzing the data.
We will monitor each abstractor’s performance throughout the data collection period. We will review each abstractor’s metadata, such as length of time within a case and volume of cases abstracted, to provide additional support as needed. We will request that provider sites use an experienced health professional such as a licensed registered nurse, registered health information administrator or technician, certified physician-based coding specialist, and/or certified professional coder, to perform the medical record abstractions. If the provider prefers an on-site abstractor from the study team, a similarly credentialed abstractor will be provided.
We will train the abstractors before starting chart reviews to ensure the protocols are implemented identically no matter which abstractor conducts the review. We will prepare an abstraction manual for the training. It will include an overview of the project, the purpose of the data abstraction, the data elements to collect, a review of the abstraction tool, and a demonstration of how to use the tool and dropdown menus. During the training, we will review documentation practices by provider type, security and privacy requirements, operational procedures for review and data entry, and review problem solving. Each abstractor performing chart reviews will be able to reference the manual during abstraction. In addition, a separate worksheet embedded within the tool will define the required data elements and provide instructions for using the Excel-based tool.
Table 6 shows the schedule for conducting the data collection activities.
Table 6. Schedule of data collection activities
Activity |
Start Date |
End Date |
Sample selection of providers |
January 1, 2019 |
March 31, 2019 |
Site recruitment |
November 11 2019 |
January 31, 2020 |
Sample selection of clients |
November 11, 2019 |
January 31, 2020 |
Interviews and chart reviews |
|
|
Round 1 (10 sites) |
January 1, 2020 |
February 28, 2020 |
Round 2 (10 sites) |
March 1, 2020 |
April 30, 2020 |
Round 3 (10 sites) |
May 1, 2020 |
June 30, 2020 |
Mail thank you letter and honoraria/incentive payments |
Rolling basis as data collection is completed |
No later than June 30, 2020 |
Clean data and prepare final data file |
July 1, 2020 |
August 31, 2020 |
After receiving approval from the Office of Management and Budget to collect data for this study, which we expect will occur in November 2019, we will begin recruiting Type 1 providers from the sample drawn from the 2017 RSR (described in Chapter II). Because the Type 1 providers are our conduit to the Type 2 providers, provider recruitment will occur in two stages.
First, we will ask HAB to send each sampled Type 1 provider an email with an introductory letter from the bureau’s deputy director. We will follow-up with an e-mail to the provider asking to schedule a call so that we can explain the purpose of the study and assess the provider’s eligibility to participate. We will include a bulleted list of the participation requirements and a list of the clinical data elements to be collected so the provider can determine its own ability to participate. If we do not receive a response within one week, we will follow up with the provider by telephone.
If the Type 1 provider is willing to speak with us, we will schedule an initial 30–60 minute call within two weeks to describe the purpose and goals of the study, the benefits of participation, the data collection activities, the requirements of participation, and the timeline for interviews and data abstraction. We will invite the provider’s program director, a staff member familiar with its data systems, and other key decision makers to participate in the meeting, as appropriate. Before the meeting, we will share with the provider a PowerPoint presentation that we will use to guide the call, along with a study fact sheet that answers questions about the provider’s level of effort and benefits of participating.
We will also use the introductory calls to assess the sampled Type 1 providers’ eligibility to participate and to assess their willingness and ability to commit to the qualitative and quantitative data collection requirements. We will also ask the provider to estimate the number of non-OAHS clients it believes are not receiving regular HIV medical care. If a Type 1 provider is ineligible (for example, because it serves fewer than 25 non-OAHS clients, it does not know where its clients go for medical care, or none of its Type 2 partners serves more than 10 of its clients) or if it refuses to participate, we will record the reason for its ineligibility or refusal in our recruitment tracker, and we will draw a replacement Type 1 provider from the same stratum.
During the introductory call, we will inquire about the Type 1 provider’s willingness to help with our qualitative data collection effort. For this aspect of the study, we will request assistance with identifying and recruiting the appropriate staff and clients to interview about the service use and health outcomes of clients who do not receive directly RWHAP-funded OAHS. We will inform the provider that we would like to interview a RWHAP program manager, a case manager or other provider of non-OAHS medical or support services, a medical provider (if applicable), and up to four RWHAP clients.
During the introductory meeting, we will also discuss the quantitative data collection activities. If a Type 1 provider has the service use and clinical information we need and can share it with us, we will coordinate medical record abstractions at this Type 1 provider’s site. If the Type 1 provider cannot share the information with us but partners or has a relationship with Type 2 providers that meet the criteria described in Chapter II, we will consider the Type 1 provider eligible for a interviews involving the staff and clients, and we will initiate recruitment of the Type 2 providers for the qualitative (staff interviews) and quantitative (chart review and abstraction) aspects of the study.
We will also explain the benefits of provider participation during the recruitment call. Type II providers will be offered for an honorarium not to exceed $650. The honorarium amount will be based on the number of medical records abstracted ($10 per record, maximum of 50) and the number of staff participating in interviews ($50 each, maximum of 3). Type I providers will not be offered an honorarium because they receive federal funding under the RWHAP. Participating Type I and Type II providers will receive a summary of their clients’ abstracted data at the end of the study.
For the client interviews, we will determine with the provider the most appropriate strategy for recruiting the non-OAHS clients. Providers may prefer to identify and contact eligible clients and schedule the interviews themselves. We will give providers a fact sheet to share with the clients that explains the purpose of the project, interview topics, measures to protect client confidentiality, and logistics of the interview. Providers will ask clients to be available at a specific time on a specific date for a telephone interview. If the provider opts for on-site interviews, the clients will come to the clinic or service agency during the time of the site visit for an in-person interview. Alternatively, if on-site interviews are planned and the provider has enough non-OAHS clients, we can conduct “intercept” interviews in which clients who come in for care during the site visit. We will ask clients whether they would like to be interviewed, either before or after their appointment. To facilitate the interviews, we will offer clients a $25 gift card to compensate them for their time, share a fact sheet that Type 1 providers can use to explain our study to clients, and make ourselves available at times when it is most convenient for clients—for example, before or after the traditional workday.
During the 60-day notice period, the project team pilot-tested the provider and client interview guides for the non-outpatient ambulatory health services (OAHS) outcomes study. Pilot-testing was done in-person at the 2018 National Ryan White Conference, as well as by telephone, depending on the availability of the informant. We spoke with program administrators and a care manager from five RWHAP sites and held a focus group with three peer support staff from one RWHAP site. During the focus group, feedback was also elicited on both the provider and client interview guides. The purpose of pilot-testing was to elicit feedback on the clarity of the interview guides and the relevancy and appropriateness of the questions.
Provider guides. As a result of pilot testing the provider guides, the following revisions were made:
Added questions about clients who are out of care. Provider staff said that the guides assumed that all RWHAP clients were in care and receiving OAHS at another provider site. Because some non-OAHS clients might be out of care, some questions were rephrased to remove this assumption and a question added about the proportion of clients that are out of care.
Clarified terms. Terms such as “OAHS” and “Ryan White,” which some staff might not be familiar with, were clarified. Examples and probes were added to describe the support services that providers offer to link clients to care. Finally, the term “Program Manager” was replaced with “Program Leadership” so that we identify staff with a global perspective on the RWHAP and clinic partnerships.
Noted the need for criteria for service delivery staff. Provider staff also suggested we develop criteria for the service delivery staff to be interviewed. They said that doing so will help ensure that the selected staff are familiar with the general operations of the clinic, not just their own work and caseload. These criteria have been incorporated into our provider recruitment strategy.
Client guides. A number of revisions were made to the client guide based on feedback from peer support staff at one provider site:
Clarified terms. Peer support staff suggested minor changes to language to make the questions more relatable to RWHAP clients. For example, the peer support staff indicated that clients will likely know what “undetectable” means but might not understand the term “viral suppression.” They also said that referring to the individual providing HIV care as a “doctor” is more accessible to clients (though perhaps less precise) than “clinician.” Updates have been made to the client interview guide based on this feedback.
Shortened and rearranged protocol. Peer support staff indicated that the client protocol was too long. They also thought that breaks between sections interrupted the flow of the interview and had suggestions for grouping questions that would support a more conversational flow. In response to their feedback, transitions between sections have been removed, some questions have been regrouped, and duplicative questions have been removed.
Providing context for interview. Peer support staff suggested providing clients with an overview of services and terms (for example, “medical care”) to frame the interview. This has been incorporated in the client interview guide.
Burden. We asked providers to estimate how long they thought it would take to complete the provider interviews and for clients to complete the client interviews. Estimates were 60 minutes and 30 minutes respectively. These estimates were the basis for the burden estimates provided in Part A.
Barbara Carlson, M.A. in Statistics (Boston University)
Mathematica, Inc.
955 Massachusetts Avenue, Suite 801
Cambridge, MA 02138
(617) 674-8372
Fact Sheet for Type 1 RWHAP Nonmedical Non-OAHS Providers
Fact Sheet for Type 1 RWHAP Medical Non-OAHS Providers
Fact Sheet for Type 2 Non-OAHS Funded Medical Providers
Fact Sheet for RWHAP Clients
Introductory Letter from HRSA to Providers
Type 1 Provider Introductory Discussion PowerPoint Slides
Type 2 Provider Introductory Discussion PowerPoint Slides
Provider interview protocols
Scenario A
RWHAP provider: program manager
RWHAP provider: service delivery staff
Non-OAHS funded provider: medical director
Scenario B
RWHAP provider: program manager
RWHAP provider: case manager
RWHAP provider: clinician
Scenario C
RWHAP provider: program manager
RWHAP provider: case manager
RWHAP provider: clinician
Non-OAHS funded provider: medical director
Client interview protocol
1 We will also use this introductory telephone call to recruit eligible the providers for the study. Chapter III discusses our recruitment strategies.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | RWHAP OMB SUPPORTING STATEMENT PART B |
| Subject | OMB PART B |
| Author | MATHEMATICA |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |