Assessing
Care and Health Outcomes Among Ryan White HIV/AIDS Program (RWHAP)
Clients Who Do Not Receive RWHAP-Funded Outpatient Ambulatory Health
Services (OAHS)
OMB SUPPORTING STATEMENT
PART A
OMB Control No. 0906-XXXX
November
2019
Submitted by:
HIV/AIDS Bureau
Health Resources and Services Administration
U.S. Department of Health and Human Services
5600 Fishers Lane
Bldg.: 09N190B
Rockville, MD 20857
Project Officer: Robert Mills
CONTENTS
A.1. Circumstances making the collection of information necessary 1
A.2. Purpose and use of information collection 3
A.3. Use of improved information technology and burden reduction 5
A.4. Efforts to identify duplication and use of similar information 6
A.5. Impact on small businesses or other small entities 6
A.6. Consequences of not collecting the information or collecting the information less frequently 7
A.7. Special circumstances relating to the guidelines of 5 CFR 1320.5 7
A.8. Comments in response to the Federal Register notice/outside consultation 7
A.9. Explanation of any payment/gift to respondents 7
A.10. Assurance of confidentiality provided to respondents 8
A.11. Justification for sensitive questions 9
A.12. Estimates of annualized hour and cost burden 10
A.13. Estimates of other total annual cost burden to respondents or record keepers/capital costs 11
A.14. Annualized cost to federal government 11
A.15. Explanation for program changes or adjustments 12
A.16. Plans for tabulation, publication, and project time schedule 12
A.17. Display of OMB expiration date 14
A.18. Exceptions to certification for Paperwork Reduction Act submissions 15
The Ryan White HIV/AIDS Program (RWHAP) funds a comprehensive set of services for people living with HIV (PLWH) in the United States. Providers funded by the RWHAP are required to submit client-level data each year to the HIV/AIDS Bureau (HAB or the Bureau) in the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS). Despite these data requirements, there are gaps in the Bureau’s knowledge about service use and process and clinical outcomes, particularly for clients who connect with the RWHAP only for wraparound services while receiving medical care elsewhere (or not at all). Clinical data, including CD4 counts (a measure of health) and viral loads (a measure of infectiousness), are unavailable for clients who receive their HIV medical care from providers that are not directly funded by RWHAP for Outpatient Ambulatory Health Services (OAHS) (hereafter referred to as “non-RWHAP-funded OAHS providers”). These clients could have good outcomes related to retention in care, treatment adherence, and viral suppression that justify the use of RWHAP funds to provide only support services to them. Conversely, they might not be as engaged in medical care and treatment if they are connected to a non-RWHAP-funded OAHS providers, or they might not receive medical care at all, and subsequently have poorer health outcomes than clients who receive RWHAP-funded OAHS.
HRSA is proposing to conduct a research study called Assessing Care and Health Outcomes among Ryan White HIV/AIDS Program (RWHAP) Clients Who Do Not Receive RWHAP-Funded Outpatient Ambulatory Health Services (OAHS) to address this knowledge gap and collect information not available elsewhere to improve care and health outcomes for all RWHAP clients. Specifically, HRSA’s contractor, Mathematica, will collect qualitative and quantitative data from a nationally representative sample of 30 RWHAP grant recipients and subrecipients (hereafter referred to generally as “RWHAP providers”) on the service use and process and clinical outcomes of clients who do not receive OAHS directly funded by the RWHAP. Detailed information on the sample selection methodology is provider in Supporting Statement B, Section B.2.1.
The quantitative data, collected through medical record review, will offer insights on outcomes such as retention in care, initiation of antiretroviral therapies (ART), and viral suppression among the study population. The qualitative data, collected through interviews with clients and their providers, will help HRSA assess whether clients who receive only support services or non-OAHS medical services under the RWHAP are able to access care and treatment elsewhere, as well as any challenges they face obtaining those services. Understanding differences in access to care and health outcomes for clients who do not receive OAHS directly funded by the RWHAP will help HRSA identify opportunities to better coordinate and deliver medical care to all people living with HIV.
We are able to separate clients receiving RWHAP services into 2 categories: 1) those that received outpatient ambulatory services (OAHS) and thus we have their visit and viral load data and 2) those that do not receive outpatient ambulatory services but do receive other medical or support services. One of the purposes of this study is to estimate viral suppression among all RWHAP clients; as a result, we will be able to estimate the number of RWHAP clients who are not receiving OAHS anywhere – that is, they are out of care.
HRSA HAB could use this information to focus technical assistance efforts to help RWHAP recipients to re-engage the clients into medical care. People with HIV who are out of care are more likely to transmit the virus to HIV-negative partners; therefore it is critical to identify where there are people out of care.
If we find that clients are receiving OAHS in other care settings, HRSA HAB could use the study results to highlight best practices and resources then share those across the RWHAP community through other resources
This study will not affect our funding of providers. By statute, the RWHAP uses a formula-based funding mechanism. This study will not evaluate provider performance, it will identify gaps in the care delivery system and additional needs of people with HIV to engage and remain in HIV care. The majority of “providers” with clients for whom we don’t have clinical data are support services providers (for example, foodbanks); there is no link between funding for providers and this study.
HRSA HAB uses information from its evaluation studies to inform technical assistance projects and programmatic decisions. Through cooperative agreements (funding mechanism), HRSA HAB funds organizations to provide direct technical assistance to organizations for capacity building, implementation of interventions, clinical quality improvement, etc. Some examples include:
If this study found that clients receiving RWHAP certain types of services (for example foodbank services – continuing the example used above) were more likely to be in care (with non-RWHAP provider), adherent to medication, and virally suppressed, we could then consider various options for identifying foodbank best practices and funding technical assistance for other RWHAP organizations to implement food-based interventions which could then increase retention in care and viral suppression among their patient population. We could also consider a program letter encouraging RWHAP providers to expand their services or collaborate with food-based services for their clients.
If we found certain populations were more likely to be out of care, HRSA HAB could establish cooperative agreements for technical assistance focused on using evidence-informed interventions to re-engage the identified populations into care. The technical assistance could also help RWHAP recipients to partner with other agencies to ensure these clients are linked to care, remain engaged in care, and can adhere to antiretroviral therapy.
If we found specific policy barriers existed for clients who receive support services and are not engaged in HIV care, HRSA HAB would look at how current RWHAP policies are interpreted and implemented. If indicated, HRSA HAB would provide an updated policy clarification notice and training/technical assistance to address any barriers or gaps identified. Any updates to the RWHAP policies would adhere to legislative requirements.
HRSA is requesting approval from the Office of Management and Budget (OMB) to conduct these qualitative and quantitative data collection activities as part of the proposed study. While the study is two years in duration, it is anticipated that data collection will occur over a 4-month period, from January through April 2020. Although the study is two-years in duration, we are seeking clearance for a 3-year period in case we encounter delays in site recruitment and/or data collection.
The HRSA HAB administers RWHAP grants to eligible metropolitan areas and transitional grant areas (Part A), States and territories (Part B), and community-based organizations (Parts C and D). These grants enable the delivery of medical care and support services to PLWH who otherwise would not have access to these services. In 2017, the RWHAP funded more than 2,000 grant recipients and indirect subrecipients for at least one service, which were delivered to about 550,000 clients across the country.
Recipients and subrecipients funded to render direct client services must submit client-level data annually to HRSA as part of the RWHAP Services Report (RSR). The client-level data contain a record for each RWHAP-eligible client who received a service during the reporting period (the calendar year). The data elements that recipients and subrecipients must report vary by the services the client received during the year. Demographic and service utilization data are reported for all clients. Importantly, though, clinical data (including lab results, which are required for measuring viral suppression) are reported only for clients who received OAHS during the reporting period. Thus, the client-level data can paint a detailed picture of some characteristics of the population served by the RWHAP, but analysis of key health outcomes—such as viral suppression—is limited to the subset of clients who receive OAHS (representing approximately two-thirds of the clients served by the RWHAP).
Analysis of RSR client-level data has demonstrated continuing improvements in health outcomes for clients served by the RWHAP. Of the roughly 343,000 clients with viral load data in the RSR, nearly 86 percent were virally suppressed in 2017, compared with less than 70 percent in 2010.1 However, there are significant gaps in our knowledge about service use and process and clinical outcomes for some of its clients. Viral load (and medical service use) data are unavailable for RWHAP clients who receive medical care from a provider not directly funded by RWHAP for OAHS or who do not receive care and treatment at all.
There are three scenarios in which a RWHAP client could receive some services through the RWHAP but not receive OAHS directly funded through the RWHAP; we refer to these as Scenarios A–C below and in Table 1.1. Throughout this statement, we refer to the RWHAP-funded provider as the Type 1 provider and the non-RWHAP-funded OAHS providers as the Type 2 provider.
The Type 1 provider does not offer medical services. In this scenario, we assume that clients either receive their medical care from a Type 2 provider and their support services from the Type 1 provider or that they are not receiving medical care at all. We can observe this scenario in the RWHAP Services Report (RSR) by looking at RWHAP-funded providers who report delivering support services or non-OAHS core medical services, but do not report providing OAHS.
The Type 1 provider delivers medical care but is not directly funded for OAHS. In this scenario, we assume that most clients receive their medical care on-site (but the clinical data are not reported to us because the provider is not directly funded for OAHS) or that clients are not receiving medical care at all. We will use the provider type variable in the RSR to identify medical providers (such as hospital-based or community health centers) that do not report furnishing OAHS.
The Type 1 provider is directly funded for OAHS, but only some of its clients receive OAHS on-site. In this scenario, we assume that clients either receive their medical care from a Type 2 provider and their support services from the Type 1 provider or that they are not receiving medical care at all. For OAHS-funded providers with non-OAHS clients, we will assess the number of clients who did versus did not receive RWHAP-funded OAHS based on the “OAHS service visits delivered” client-level item in the RSR.
Table 1.1. Three scenarios under which RWHAP clients do not receive OAHS directly funded through the RWHAP
Provider Type |
Scenario A |
Scenario B |
Scenario C |
Type 1 |
RWHAP
provider |
RWHAP
provider |
RWHAP
provider |
Type 2 |
Non-RWHAP-funded OAHS providers (medical) |
Non-RWHAP-funded OAHS providers (medical) |
Legislative authorization for this study comes from Title XXVI of the Public Health Service Act as amended by the Ryan White HIV/AIDS Treatment Extension Act 2009 (Public Law 111-87). See Attachment A for the legislative authorization for this study.
We will conduct qualitative interviews at 30 provider sites, either in-person or by telephone, depending on the preference of the site. At each site, we expect to interview both Type 1 providers and its Type 2 non-RWHAP medical partner or partners. We will conduct two data collection activities: (1) semi-structured interviews with staff and clients for qualitative analysis and (2) medical record abstraction for quantitative analysis. In most cases, we will conduct staff interviews at both Type 1 and Type 2 providers and client interviews at only Type 1 providers. Whenever possible, we will seek quantitative data from Type 1 providers because they are funded by the RWHAP and our primary point of contact for this study. If the sampled Type 1 provider does not maintain the clinical data we need for this study, we will work with the Type 2 provider(s) to which the majority of their patients go for medical care to obtain the medical record data we need. We describe each data collection activity below.
The study team will conduct individual interviews with up to three to four staff at both of the Type 1 and Type 2 provider organizations. The purpose of the staff interviews is to better understand the services that RWHAP non-OAHS clients receive, the referral processes between RWHAP and non-OAHS funded providers, and their perceptions of quality of care on non-OAHS clients’ health outcomes. We plan to conduct interviews with three types of informants:
Provider program management/leadership staff. These interviews will focus on provider organizational structures, funding streams, care models, and partnerships and referral relationships with other providers.
Staff performing case management/direct service provision. These interviews will focus on the types of non-OAHS services that RWHAP clients receive, the mechanics of referral processes and information sharing across RWHAP and non-OAHS funded providers, and the challenges and benefits clients face accessing services in multiple setting types.
Medical directors/clinicians. At provider sites offering medical services, we will gain an understanding of collected information on these aspects of medical care models: initiation of antiretroviral therapy (ART), frequency of HIV care appointments, and strategies used to engage clients in care and encourage ART adherence.
Although each interview will have a slightly different focus based on the provider scenario (A–C) and the interviewee’s role in the provision of services, we will gather information from all informants on their perceptions of the quality of care and health outcomes for RWHAP non-OAHS clients and opportunities to improve care, coordination, and data sharing across providers for this population.
We will conduct individual interviews with up to four non-OAHS clients at each of the 30 sampled Type 1 providers. In interviews with non-OAHS clients, we will hear, in the clients’ own words, why they receive services from non-OAHS funded medical settings, the benefits or challenges they face receiving care across multiple settings, their medical and nonmedical service use, their health outcomes, and the supports that help them stay engaged in care, adherent to ART, and virally suppressed.
From each of the 30 selected provider sites, medical record data will be sought for all of the RWHAP non-OAHS clients for whom there is no clinical data in the RSR. If a provider has more than 50 non-OAHS clients, we will select a stratified random sample of 50 clients for a maximum of 1,500 chart reviews in total for this study. The purpose of the chart abstractions is to obtain service use and clinical information on clients who do not receive directly-funded OAHS through the RWHAP. The data elements to be extracted from medical records include (1) date of initial HIV diagnosis, (2) dates of outpatient medical visits, (3) dates and types of ART prescriptions, and (4) dates and results of viral load lab tests. We will limit the abstraction to records with dates from January through December 2018. In addition, we will obtain the clients’ encrypted unique identifier (eUCI), age, and gender from the 2017 RSR and merge this information onto the analytic file.
We will use the information collected in the abstraction tool to calculate the key outcomes of this study: (1) the percentage of non-OAHS clients who were retained in care, using a definition we developed, such as one visit during first nine months followed by second visit at least 90 days after first visit; (2) the percentage of non-OAHS clients for whom ART had been initiated; and (3) the percentage of clients who were virally suppressed. It will also allow us to assess the types of ART medications prescribed, relative to those covered under the AIDS Drug Assistance Program. Including the dates of each event will enable us to measure the average time from initial diagnosis to ART initiation and from ART initiation to viral suppression, critical measures of optimal HIV care and adherence. Finally, recording the actual lab results will enable us to assess the range of clinical outcomes, as well as the extent of virologic failure, persistent low-level viremia, and durable suppression.
In the event that some of the detailed information is not available in medical records, we will use a shorter version of the tool. In addition to the pre-populated eUCI, age, and gender of the client, the short version of the tool will include four key data elements: (1) count of the number of outpatient medical visits, (2) an indication of whether any HIV medications were prescribed, (3) an indication of whether any viral load lab tests were ordered, and (4) an indication of whether the viral load results were less than 200 copies/ml. The abstractors will use an Excel-based abstraction tool (full and short versions), which will also include instructions for abstracting the data elements. To the extent possible, the cells will be pre-formatted to reflect the data fields (such as dates) and may include a drop-down menu of acceptable response options (for example, ART drug codes).
The project team plans to reduce qualitative data collection burden through three strategies. First, we will conduct telephone interviews with provider staff and clients, unless the provider site requests on-site interviews. This will reduce burden on provider sites with regard to scheduling and space. Second, we will research each provider before the interview, allowing the interview to elicit information not available through other sources, such as the RSR and provider websites. We will create provider profiles on RWHAP funding sources, funded services, and client demographics to help inform the interviews without requesting duplicative information. Finally, we will limit the duration of the interviews. Staff interviews will take no more than 60 minutes to complete, and the client interviews will take no more than 30 minutes to complete.
The study team will not rely heavily on information technology for the qualitative data collection for this project. We will record interviews on audio, allowing respondents to speak at their own pace without jeopardizing comprehensive data collection. We will have the audio recordings of the provider and client interviews transcribed, and then code and analyze the transcribed data in NVivo. We will use the coded data to conduct a theme-based analysis based on the research questions. Additional details regarding the qualitative analysis can be found in Section A.16.1.
For the medical record abstractions, the study team will develop an Excel-based data entry tool to collect the data elements for all sampled clients. The macro-enabled Excel abstraction tool will require little or no installation requirements. To begin the chart review, the abstractor will open the site’s Excel document and enable the content, which will display the abstraction worksheet. The study team will auto-populate the study-specific de-identified client ID number (EUCI), client age, and client gender into the data entry tool. Some data fields will have drop-down menus to ensure uniformity of reporting and to reduce burden.
To reduce provider burden, the medical provider will decide if it is most efficient for them to do their own medical record abstraction or if they would prefer the study team perform remote or on-site abstraction. For providers that elect to perform their own abstraction, we will provide training to ensure consistency in data collection across provider sites. For providers that elect on-site chart review, we will provide a two-week advance notice of the client records we want to review so that the provider has adequate time to collect all hard-copy and electronic records before the abstractor’s arrival. For providers that elect remote chart review, an abstractor will work with them to identify the people with whom they can coordinate information technology and security protocols. This communication will determine the most appropriate means for establishing remote access while maintaining privacy and security of confidential information. We will ask providers to appoint a contact person to facilitate access to the medical records (electronic and hard copy) and answer abstractors’ questions regarding missing or incomplete data. Based on Mathematica’s experience with medical record chart abstraction on clinical quality measure development and testing work funded by the Centers for Medicare & Medicaid Services (CMS), we expect 70 percent of providers to perform their own abstractions, 20 percent to allow remote abstraction, and 10 percent to request on-site abstraction by members of the study team.
The information collection will not duplicate any other effort and will provide information unavailable from other sources. HRSA does not use other collection instruments to obtain similar data from its recipient or subrecipient grantees. The staff and client interviews and medical record abstractions will provide information that HRSA is unable to obtain through regular program records. As stated above, HRSA currently does not collect medical service use or clinical data for RWHAP clients who receive OAHS from a provider not directly funded for such services by the RWHAP. HRSA did consider collecting these data at the inception of its client-level data collection in 2008. However, through a vetting process with service providers across the country, we learned that it would be an undue burden for many service providers who are not funded by RWHAP for their OAHS services or who do not provide OAHS services to collect clinical data for their RWHAP clients.
Respondents for the provider staff interviews are program managers, medical directors, and case mangers or other frontline service providers. The sample of RWHAP providers selected for this study might include small businesses. We have attempted to minimize the impact on small businesses by designing interview protocols that should take no more than 30–60 minutes to complete and by limiting the sample size to the minimum number needed to produce generalizable population estimates. We may also choose to exclude providers from the study that serve 10 or fewer RWHAP clients, given the limited benefit of such an effort.
This is a one-time data collection effort. If the data are not collected, HRSA will be unable to measure and monitor the quality-of-care and health outcomes (such as retention in care and viral suppression rates) for the one-third of its clients who do not receive OAHS directly funded through the RWHAP. This would mean that the Federal agency would not be able to develop effective strategies for coordinating care between RWHAP and non-OAHS funded providers and ultimately improve the process and clinical outcomes for the study population in the future.
There are no special circumstances related to the proposed data collection. The request fully complies with the regulation.
A 60-day notice for this study was published in the Federal Register on April 8, 2019 Volume 84, No. 67, pp.13934–13935.
Public comment and responses. Not applicable
The evaluation team will provide a $25 gift card to respondents who participate in the semi-structured client interviews, to encourage participation and thank them for taking part. We will provide gift cards from a large retailer that can be accessed in-person or online (e.g., Target, Amazon, etc.) In two of our recent HIV studies, client incentives were not offered and response rates were 35 percent and 41 percent, respectively. With an incentive of $25, we hope that at least 65 percent of those invited to participate in interviews complete the task. We believe that a $25 gift card is commensurate with the 30 minutes of a client’s time needed to conduct the interview.
Because Type I providers receive federal funding through the RWHAP, we do not believe it is appropriate to offer them an honorarium for participating in this study (by assisting and medical record abstraction and/or participating in staff interviews). We consider these activities to be part of their responsibilities as a RWHAP grantee or subgrantee to improve the quality of care for their clients.
Because Type II providers are not funded through the RWHAP, we will offer them two types of honoraria for assisting us with data collection for this study. We will notify Type II providers about the honorarium during the initial recruitment telephone calls. Providing an honorarium to Type II providers is important to maximize participation and minimize recruitment costs. In two of our recent studies, similar in approach to our proposed study, provider incentives were not offered and response rates were only 35 percent and 41 percent.2 Although offering an honoraria does not guarantee an improvement in the response rate, we are hopeful that it will increase participation in our study and provide the agency with information on the effectiveness of offering honoraria for future data collection efforts.
Provider interviews. The first kind of honorarium offered will be for provider interviews. We anticipate that three staff members at each Type II site will participate in qualitative interviews. Each Type II site will receive a maximum of $150 for participation in this component of the study ($50 per interview). The honorarium will be offered to the facility rather than to the individual staff members participating in the interviews because interviews will occur during working hours.
Medical record abstraction. We will offer Type II providers that perform their own data abstractions an additional honorarium. Type II providers will abstract 10 data elements from medical records for between 10–50 patients (depending in the size of their RWHAP patient population). Type II sites that abstract between 10 and 25 records will receive an honorarium of $300. Those that abstract more than 25 records will receive a total of $500. The maximum honorarium for a Type II provider will be $650 if they abstract medical records for more than 25 patients ($500) and participate in staff interviews ($150).The total maximum honoraria payment across all Type II providers will be $19,500 (assuming 30 Type II providers).
HRSA has embedded protections for privacy and confidentiality in the study design. The proposed information collection will fully comply with all aspects of the Privacy Act. Individuals and practices will be ensured of the privacy of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). We will explain to all interview participants that the data they provide will be kept private to the extent allowed by law, reported at the aggregate level, and used for research purposes only. We will explain the four elements of consent to each participant so that they understand (1) the nature of the interview (subject matter and duration); (2) the privacy of the information he or she provides, as well as the privacy of his or her identity; (3) the voluntary nature of participation (including ability to discontinue participation at any time); and (4) any benefits, risks, or discomfort associated with participation. The data from the interviews and chart abstractions will not contain personally identifiable information (PII) that could be linked to specific individuals, nor will it put individuals at risk for criminal or civil liability or be damaging to the individual’s financial standing, employability, or reputation.
We will start by sending a list of the sampled clients’ encrypted unique client identifiers (eUCIs) used by us to deduplicate client records in the RSR to the Type 1 provider. We will ask the Type 1 provider to confirm that it can locate the list of clients in its records by the eUCI. If a Type 1 provider does not maintain the eUCIs in its records, we will offer assistance in generating these numbers based on its clients’ name and demographic characteristics. We will also ask the Type 1 provider to verify that the information needed is available in a format that is accessible to the abstraction team. If the Type 1 provider does not collect the information needed, we will ask it to send the list of clients to the associated Type 2 providers. Because the (non-RWHAP) Type 2 providers do not use eUCIs, we will ask the Type 1 provider to append the first and last names, age, and gender of the clients to the list. We will then ask the Type 2 provider to confirm that it can locate the clients in its system and that the necessary data elements are accessible. The list with clients’ name, age, and gender will remain at the provider sites and will only be accessible to the abstractors.
We will process and store all data on our password-protected local area network (LAN). We protect our LAN with several security mechanisms. We restrict access to private information stored on LAN directories to authorized project staff. In addition, we keep the network servers containing private information in a locked area. The data will not contain names or other personally identifying information. We will report interview responses at the aggregate level only, and we will not attribute any of the results to individual provider organizations, staff, or clients.
During medical record abstraction, we will collect confidential, patient-level data with the approval of the participating providers. Federal privacy regulations pursuant to the Health Insurance Portability and Accountability Act of 1996 (HIPAA) will govern this process. A Business Associate Agreement will be executed at each provider site (‘covered entity’). The HIPAA Privacy Rule permits disclosure of protected health information (PHI) by covered entities, without an individual’s authorization or permission, for public health activities including those aimed and preventing or controlling disease.3 Although client authorization or permission is not necessary to access client data, from the HIPAA perspective, we believe it is important that clients are notified about the study and the use of their medical data. Prior to the chart abstraction, Type I providers will be asked to post in a public place a Study Notification Sign in one or more well-traveled areas of their physical location (Appendix D). This will alert patients to the fact that their data may be abstracted as part of this study and that any information resulting from the study will be kept confidential, with findings shared in aggregate only.
All contractor staff assigned to work on this project sign confidentiality pledges as a term of employment. The confidentiality pledge requires that our staff maintain the confidentiality of all information collected. Attachment D provides a copy of our confidentiality pledge. All staff members associated with the study participate in annual trainings regarding applicable Federal laws and regulations governing data security and confidentiality. The study team will treat any data that are not de-identified, not aggregated, and/or not publicly available as confidential. Staff will read and sign the security and confidentiality agreement before they start work on this study. HRSA maintains electronic copies of the signed agreements as well as confidentiality and nondisclosure agreements.
Additionally, we have received institutional review board (IRB) clearance for this study.
The interview protocols that target provider staff have no sensitive questions. Interviews with HIV-positive clients that receive OAHS not directly funded by the RWHAP have sensitive questions related to their utilization of health care services and health status. These questions are essential for assessing the availability, quality, and outcomes of care for the study population. Given prior interview experience with this population, we believe that clients will be comfortable discussing questions related to their HIV care and health status in an interview with a researcher who is knowledgeable about the RWHAP and experienced with interviewing PLWH. However, as will be stated in the consent form, participation is entirely voluntary. Participants do not have to answer any questions that make them feel uncomfortable, and participants can stop the interview at any time. In addition, when providers recruit clients for the interview, they will communicate with them about the topics to be covered in the interview, so clients are not surprised by questions asked.
For all three scenarios, A–C, referenced earlier in this statement, we will conduct one interview with a Type 1 program manager and one interview with a Type 1 case manager or direct service provider. The number of interviews with a medical director or medical provider will depend on the scenario. For Scenarios A and B (where the Type 1 provider is not funded to provide OAHS), will conduct an additional interview with a medical director or medical provider. For Scenario A, these interviews will take place with the Type 2 non-OAHS funded medical provider. For Scenario B, these interviews will take place with the Type 1 RWHAP medical provider. For Scenario C (where the Type 1 provider is funded for OAHS but does not provide this service to some of its clients), will conduct two interviews with medical directors/providers: one at the Type 1 provider and one at the Type 2 provider. Each of these interviews will take at most one hour. Both types of providers will also need to participate in a 15-minute telephone screening to ensure they are eligible to participate in the study. In addition, we will conduct up to four interviews with clients at each of the Type 1 providers. These interviews will last 30 minutes each. Finally, either the Type 1 provider or the Type 2 provider (whichever one has access to the clinical information we need for this study) will need to perform medical record abstractions (or allow a Mathematica abstractor to do so) for 50 sampled patients. We estimate each abstraction will take 4.8 minutes for a total of four hours per provider site (4.8 minutes * 50 abstractions) or 120 hours across 30 provider sites. Table 12.1 summarizes the estimated burden hours by scenario and in total for this study.
Table 12.1. Estimates of total hour burden
Form name |
Number of respondents |
Number of responses per respondent |
Total responses |
Average burden per response (hours) |
Total burden (hours) |
Provider Interviews |
|
|
|
|
|
Program manager/ program leadership interview |
30 |
1 |
30 |
1 |
30 |
Case manager / service delivery staff interview |
30 |
1 |
30 |
1 |
30 |
Medical director / medical provider |
40 |
1 |
40 |
1 |
40 |
Client interview |
120 |
1 |
120 |
0.50 |
60 |
Medical record abstraction |
30 |
50 |
1,500 |
0.08 |
120 |
Telephone screeninga |
45 |
1 |
45 |
0.25 |
12 |
Total |
295 |
|
1,765 |
|
292 |
a We assume that we will conduct a telephone screening with all 30 Type 1 providers, plus half of the Type 2 providers. We have budgeted the cost of the screenings using the hourly rate of a program manager.
The calculations in Table 12.2 assume an hourly wage of $54.68 for medical and health services managers (used for the program manager interviews); $30.12 for social workers (used for the case manager or other frontline service provider interviews and for chart abstraction support); and $100 for physicians (used for the medical director or medical provider interviews).4,5,6 Hourly wages for these professional staff have been doubled to account for fringe benefits and employer overhead. In addition, the project team assumes an hourly wage of $24.98 for clients, using ‘all occupation’ data available from the Bureau of Labor Statistics.7 We will interview most clients immediately after a scheduled appointment, minimizing competition with any other wage-earning opportunities.
Table 12.2. Estimates of total cost burden
Data collection method |
Total burden (hours) |
Mean hourly wage |
Total respondent costs |
Provider Interviews |
|
|
|
Program manager / program leadership interview and screening |
42 |
$109.36 |
$4,593.12 |
Case manager / service delivery staff interview |
30 |
$60.24 |
$1807.20 |
Medical director or medical provider interview |
40 |
$200.00 |
$8,000.00 |
Client interview |
60 |
$24.98 |
$1498.80 |
Medical record abstraction |
120 |
$29.28 |
$3,513.60 |
Total |
250 |
|
$19,412.72 |
This is a one-time data collection effort. There are no capital or start-up costs or costs to record keepers. There are no direct costs to respondents other than the time to participate in the study.
The total cost of the study to the government is $1,170,962 ($38,290 in calendar [CY] 2018; $435,728 in CY2019; and $696,944 in CY2020). The total study cost was based on the project budget, which includes wages and fringe benefits for all project staff; all interview costs, including respondent payments; travel, telephone, computer charges, and other indirect overhead costs; and contract fees per year. Table 14.1 displays the project costs by year for both data collection activities and non-data collection activities.
Table 14.1. Annualized cost to federal government
Calendar year |
Semistructured interview costs |
Medical record abstraction costs |
Non-data collection costs |
Total costs |
2018 |
$8,090 |
$8,090 |
$22,110 |
$38,290 |
2019 |
$166,334 |
$210,872 |
$56,770 |
$435,728 |
2020 |
$198,894 |
$297,454 |
$202,348 |
$696,944 |
Total |
$373,318 |
$516,416 |
$281,228 |
$1,170,962 |
In addition, there will be the cost for a GS 14 (Step 7) at 15 percent time (approximately $20,625 per year) to monitor the project. The average annual total cost of the project is $606,106 and the total cost of the two-year project is $1,212,212.
Data collection for this study is new; there are no changes to our burden estimates.
Qualitative data analysis. We will have the audio recordings of the provider and client interviews transcribed, and then code and analyze the transcribed data in NVivo. After we complete about one-third of the site visits, the interview team will develop a high-level coding structure based on the themes that emerge from the initial interviews. We will align this preliminary code set with the research questions and share it with HRSA for review and approval. We will revise and update the codes as data collection continues to reflect new or divergent themes. Throughout the interview process, we will conduct coding to ensure timely completion of materials. We will train the coding team, systematically code the interviews, and review the coded data to promote inter-rater reliability. We will meet regularly throughout the data collection process to discuss emerging themes or questions about how to code areas of content that span multiple themes.
By using NVivo, the interview team can group data by theme and compare findings across Scenarios A, B, and C. For example, we might find that models of care differ between RWHAP-funded and non-OAHS funded medical providers. Clients who access care at a RWHAP-funded medical provider could receive more coordinated care than clients whose medical care is not co-located with their RWHAP-funded support service provider. We will also use the qualitative data to extend our understanding of the quantitative data. For example, we might find through medical record abstraction that clients who receive medical care from a non-OAHS funded medical provider have fewer medical visits than those who receive their medical care from a RWHAP-funded clinic. Through interviews, we can understand what drives this difference—for example, clinical guidelines established by the RWHAP-affiliated providers (separate from the actual clinical guidelines for HIV care and treatment) or challenges that clients accessing care at non-OAHS funded clinics face in keeping appointments. To conduct these analyses, the task leads will coordinate closely to discuss findings from our respective data sources.
Quantitative data analysis. We will conduct a descriptive analysis of quantitative data obtained from both the RSR and the medical record abstractions. Using the RSR, we will profile the characteristics of the RWHAP providers that do not provide OAHS directly funded by the RWHAP (location, organization type, number of clients served, and type of services provided). We will also profile the characteristics of RWHAP clients for whom HRSA does not have clinical data, including the number and percentage by demographic and risk group categories. For these clients, we will also calculate statistics on the number and percentage of clients using non-OAHS core medical and support services (by type of service). We will calculate this output for all study clients as well as the clients for the sampled sites, and compare the results with those for RWHAP clients who access OAHS under the program. We will test whether the differences between the two groups are statistically significant. We assume the size of the study population is about 144,000 clients (among a total of roughly 550,000 RWHAP clients).
In addition, we will use the service use and clinical data from the chart abstractions to calculate statistics for the process and clinical outcomes. These include number of medical visits per year (a measure of retention in care), receipt of ART (an indicator of optimal treatment for people diagnosed with HIV), and the distribution of clients by viral load (a measure of contagion). Specifically, we will assess the rate of viral suppression, durable suppression, virologic failure, and persistent low-level viremia. We will use the analysis weights generated from sampling to produce generalizable estimates of viral load suppression among the study population relative to those who access RWHAP-funded OAHS, and will test for the statistical significance of the difference.
It is possible that some non-OAHS clients are currently not receiving medical care from any provider. Although health outcomes of the out-of-care subpopulation fall outside the scope of this study, it is desirable to have an estimate of the size of this group so that the findings of the proposed study are placed within the context of the non-OAHS population more broadly. We will ask the sampled Type 1 RWHAP providers to estimate the proportion of their non-OAHS clients who they believe do not regularly see a clinician for their HIV medical care. Since we anticipate that the estimate may have low precision, we do not intend to use it for weighting. Rather, we will use this information to conduct sensitivity analyses of the study results. For example, we will estimate viral suppression rates among non-OAHS clients with and without the out-of-care population to see how sensitive the study results are to the inclusion of this special population. We will ask follow-up questions about the out-of-care subgroup of non-OAHS clients during the interviews with the provider staff.
Reporting. Mathematica will prepare and submit a draft report with the main findings from the data collection and analysis efforts. The report will include a discussion of the service use patterns and unmet needs among the RWHAP clients who do not receive their medical care from providers funded for OAHS; an estimation of the retention in care, treatment initiation, and viral suppression rates among this population; and recommendations for how to improve their process and clinical outcomes. In addition, the report will provide (1) an overview of the purpose and goals of the study; (2) the study’s research questions; (3) the methods used to sample RWHAP-funded providers that do not provide OAHS, and to collect and analyze the qualitative and quantitative data from these organizations; (4) tabular summaries of the qualitative and quantitative findings; and (5) discussion of the findings with recommendations.
The report will also include a three- to four-page executive summary designed for a nontechnical audience. We will work closely with our communications department to develop a brief, attractive, high-level summary that administrators, policymakers, and members of the public can use to quickly understand the purpose of the study, its findings, and policy implications.
In all reports, we will indicate that we used a nationally diverse convenience sample and we will provide both unweighted and weighted counts to inform the reader’s interpretation of the results.
Manuscripts. Our contractor for this study will summarize the results from the analyses in two journal manuscripts, each focusing on a different research question and audience. We might use the first manuscript to highlight the characteristics and service use patterns of clients who do not receive RWHAP-funded OAHS, the reasons why they do not, and any unmet needs they face, highlighting the interconnections between RWHAP and non-OAHS funded providers. The second manuscript could then provide a quantitative summary of the process and health outcomes (including the viral suppression rate) for this population (and its subgroups)—compared with those of clients who receive RWHAP-funded OAHS—and strategies for improving their access to medical care.
In all manuscripts, we will indicate that we used a nationally diverse convenience sample and we will provide both unweighted and weighted counts to inform the reader’s interpretation of the results.
Overview. We will conduct interviews at 30 sites to understand the use of health services and process and clinical outcomes among clients who do not receive medical care and treatment funded under the RWHAP. Tasks include (1) selecting and recruiting the sites; (2) scheduling interviews; (3) conducting the interviews and medical record abstractions; (4) analyzing the qualitative and quantitative data; (5) summarizing the findings in two peer-reviewed manuscripts; and (6) submitting the data files to HRSA. We will sample up to 50 clients per site for chart review, for a total of 1,500 medical record abstractions.
Analysis timeline. We will begin site recruitment in November 2019 and will continue until a total of 30 sites have been recruited for the study. We will complete interviews at the first 10 sites by late January 2020, another 10 by late February 2020, and the final 10 sites by late April 2020. We will submit the results and data files to HRSA by the end of June 2020.
Reporting timeline. Our contractor for this study will submit the final project report and executive summary to us by the end of July 2020. The contractor will provide an oral presentation at our headquarters in Rockville, Maryland in September 2020. Our contractor will summarize the results from the analyses in the form of two manuscripts for publication in a peer-reviewed journal, each focusing on a different research question and audience, and submit them to us in September 2020.
We will display the OMB number and expiration date on every document. Interviewers will be able to access the OMB number and expiration date at any point in the interview.
Data collection efforts for this study will conform to all provisions of the Paperwork Reduction Act. No exceptions are being sought.
1 Health Resources and Services Administration. “Ryan White HIV/AIDS Program Annual Client-Level Data Report 2017.” Available at https://hab.hrsa.gov/sites/default/files/hab/data/datareports/RWHAP-annual-client-level-data-report-2017.pdf.
2 Personal communication with Robert Mills, PhD, HRSA Project Officer.
3 https://www.hhs.gov/hipaa/for-professionals/special-topics/public-health/index.html
4 http://www.bls.gov/oes/current/oes119111.htm
5 https://www.bls.gov/oes/current/oes211029.htm
6 https://www.bls.gov/ooh/healthcare/physicians-and-surgeons.htm
7 https://www.bls.gov/oes/current/oes_nat.htm
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |