Supporting Statement A

Supporting Statement A

Enhancing Linkage of STI and HIV Surveillance Data in the Ryan White HIV/AIDS Program

OMB Control No. 0906-XXXX

New Information Collection Request

Terms of Clearance: None.

A. Justification

1. Circumstances Making the Collection of Information Necessary

The Health Resources and Services Administration (HRSA) is requesting approval from the Office of Management and Budget (OMB) for a new information collection titled “Enhancing Linkage of STI and HIV Surveillance Data in the Ryan White HIV/AIDS Program (RWHAP) Evaluation.” HRSA will complete a mixed-methods evaluation of this RWHAP Special Projects of National Significance (SPNS) demonstration project using quantitative and qualitative data obtained from approximately 41 representatives associated with each of the four participating RWHAP Part B recipients (state health departments) and three representatives associated with one RWHAP Part F recipient (academic institution) serving as a Technical Assistance Provider (TAP) to the RWHAP Part B recipients. The RWHAP, authorized under Title XXVI of the Public Health Service Act, as amended by the Ryan White HIV/AIDS Treatment Extension Act of 2009, funds and coordinates with cities, states, and local clinics/community-based organizations to deliver efficient and effective HIV care, treatment, and support to low-income people with HIV. The Department of Health and Human Services (HHS) HRSA administers funds for the RWHAP.

The HRSA RWHAP supports a comprehensive system of direct health care and support services for over half a million people with HIV¹. The HRSA RWHAP makes financial assistance available for the development, organization, coordination, and operation of more effective and cost-efficient systems for the delivery of essential core medical and support services to people with HIV. Funding priorities are determined by stakeholders at local and state levels, resulting in uniquely structured programs that address their jurisdictions’ critical gaps and needs. HRSA also works in partnership with RWHAP recipients at state and local levels to use innovative approaches for community engagement, needs assessment, planning processes, policy development, service delivery, clinical quality improvement, and workforce development activities that are needed to support a robust system of HIV care, support and treatment.

The U.S. Congress mandated that client-level data be collected under the Ryan White HIV/AIDS Treatment Modernization Act of 2006 and requires the submission of Annual Reports by the Secretary of Health and Human Services to the appropriate committees of Congress. Funded service providers are permitted to collect client level information and report de-identified data to HRSA HAB, as a public health authority, pursuant to 45 CFR 164.512(b). HRSA HAB is authorized by law to receive such information for the purpose of preventing or controlling disease, and the conduct of public health interventions. These data provide information about the grant recipients’ organization and staffing, the number of clients served, services provided, client demographics, clinical data of clients served and costs of providing services. However, the data needed to evaluate the effectiveness of the Enhancing STI Linkage RWHAP SPNS demonstration project is not available through existing HRSA HAB collection.

A persistent barrier to addressing HIV and STI infections simultaneously and jointly is the lack of data systems linking HIV and STI surveillance data. Aside from helping to address problems around coinfection, there are substantial opportunities - particularly for the RWHAP - associated with linking HIV and STI surveillance data, including, but not limited to, identifying people with HIV currently out of care and identifying people with STIs who could be tested for HIV and promptly linked to care. This matched STI/HIV surveillance data will subsequently be used to improve RWHAP clinic capacity to prioritize resources for linking and re-engaging people with HIV into care. Improving information systems tracking coinfection is essential to the HRSA HAB, jurisdictions’, and clinics’ efforts to end the HIV epidemic by 2030.

2. Purpose and Use of Information Collection

HRSA is required by statute to assess the quality of care provided by RWHAP recipients.

The HHS HIV care and treatment guidelines (e.g., Guidelines for the Use of Antiretroviral Agents in Adults and Adolescents Living with HIV; Guidelines for the Prevention and Treatment of Opportunistic Infections in HIV- Infected Adults and Adolescents; and Sexually Transmitted Diseases Treatment Guidelines, 2015) and U.S. Preventative Services Task Force (USPSTF) guidelines serve as the basis for assessing the quality of care within the RWHAP. Under the Ryan White HIV/AIDS Treatment Extension Act of 2009 (Public Law 111–87) the RWHAP Special Projects of National Significance (SPNS) Program is required to support the development of innovative models of HIV care and treatment to quickly respond to the emerging needs of clients served by the RWHAP. RWHAP SPNS advances knowledge and skills in the delivery of health care and support services to underserved populations with HIV. Through its demonstration projects, RWHAP SPNS evaluates the design, implementation, utilization, cost, and health-related outcomes of treatment models and promotes the dissemination and replication of successful interventions.² Information gleaned from the RWHAP SPNS Enhancing STI Linkage evaluation can be used to enhance and coordinate health departments’ responses to HIV and STI epidemics and affect change in HIV care continuum outcomes.

Specifically, information gathered through this evaluation will provide HRSA with an understanding of the extent and impact of the TA provided by the TAP, barriers and facilitators RWHAP jurisdictions face when doing this work, and opportunities for improvement and lessons learned. HRSA will use that information to guide decision-making and improve uptake of effective programs and practices among RWHAP jurisdictions.

The overarching aims of the Enhancing STI Linkage evaluation are to:

1. Describe the TA provided by the TAP under the Enhancing STI Linkage demonstration project, and assess the extent to which TA was provided as planned and met jurisdictions' needs.

2. Assess to what extent the Enhancing STI Linkage demonstration project affected the data linking processes within participating jurisdictions, i.e. organizational processes (staffing, data sharing agreements (DSAs), memoranda of understanding (MOUs), standard operating procedures (SOPs)) and technical processes.

3. Assess to what extent the Enhancing STI Linkage demonstration project had the intended impact on client and policy outcomes, i.e. use of linked HIV and STI surveillance data by health department staff and/or clinicians and policy stakeholders, enhancement of data-to-care procedures and activities, and HIV outcomes (linkage to and re-engagement in care).

4. Conduct a cost analysis among participating jurisdictions of linking HIV and STI surveillance data and using those linked data.

5. Disseminate key findings to HRSA and TAP to inform decision-making.

A mixed-methods evaluation will be conducted to address these aims. Evaluation data collection components will include: semi-structured interviews, data end-user survey, document review, aggregate statistics, and personnel time reporting data compiled from jurisdictions. Not all data collection activities will exceed the threshold of respondents that necessitate OMB approval, therefore, this application is seeking approval for the semi-structured interviews with both the jurisdiction TA recipients (n=12 respondents) and the policy stakeholders (n=12 respondents),

as well as the data end-user survey (n=105 respondents). OMB approval for the semi-structured interviews with the TAP (n=3 respondents), aggregate statistics (n=4 respondents), personnel time reporting (n=4 respondents), and document review data (n=4 respondents) is not being requested due to the number of respondents that will participate in those data collection. Although data are not being collected from more than nine respondents for the other data collection methods (i.e., TAP semi-structured interviews, aggregate statistics, personnel time reporting data, and document review), descriptions of these methods are provided to describe the entire scope of the project.

Exhibit 1. Data Collection Overview for Activities Requiring OMB Approval

	Jurisdiction TA Recipient SSI Respondents	Policy Stakeholder SSI Respondents	Data End-User Survey Respondents	Total Respondents per Demonstration Site
Demonstration Site A	3	3	35	41
Demonstration Site B	3	3	35	41
Demonstration Site C	3	3	35	41
Demonstration Site D	3	3	35	41

The Enhancing STI Linkage evaluation will assist HRSA to better understand the achievement and effectiveness of the Enhancing STI Linkage demonstration project. The tools included in this package allow researchers to collect process and outcome data to better understand the program. This is a mixed methods evaluation that will collect data over a 3-year period from the TAP and participating jurisdictions. This is an observational study with no comparison group. Exhibit 2 outlines the evaluation aims and questions that support the evaluation approach, and serves as the backbone of this evaluation. The data sources and collection method are listed in the third column. Listed in parentheses in this column is the party from which the data will be generated – i.e. the Evaluator (E), Technical Assistance Provider (T), and/or Jurisdiction (J). Data collection activities will include: semi-structured interviews, document review, data end-user survey, aggregate statistics, and cost data compiled from jurisdictions.

Exhibit 2. Enhancing STI Linkage Evaluation Questions

Evaluation Questions	Data type	Data sources and collection method (E=Evaluator; T=TAP; J=Jurisdiction)
Aim 1: Describe the TA provided by the TAP under this demonstration project, and assess the extent to which TA was provided as planned and met jurisdictions' needs.
1a. What changes did the TAP determine were required in each jurisdiction to create or improve data sharing and utilization across their STI and HIV surveillance programs?	Qualitative	Semi-structured interviews with TAP (E) Document review: Jurisdiction TA plans (T) Needs assessment (T) TAP call notes (E)
1b. What forms of programmatic TA did the TAP provide to each jurisdiction?	Quantitative Qualitative	Semi-structured interviews with jurisdictions (E) Document review: Jurisdiction TA plans (T) TAP call notes (E)
1c. How did jurisdictions perceive the TA plan and provided TA in terms of meeting their needs and helping them reach their goals?	Qualitative	Semi-structured interviews with jurisdictions (E) Document review: Needs assessment (T) Jurisdiction TA plans (T)
Aim 2: Assess to what extent the demonstration project impacted the data linking processes (organizational and technical) within participating jurisdictions.
2a. Did surveillance data system availability and/or the reliability and validity of included data elements change in each jurisdiction after implementation? If so, how? Were other or new relevant health data systems involved with the linking exercise?	Qualitative Quantitative	Semi-structured interviews with jurisdictions (E) Document review Jurisdiction data documentation, e.g. codebooks (J) Aggregate statistics (J)
2b. Did jurisdiction technical and technological capacity, linking frequency, sharing, and/or utilization of linked HIV and STI surveillance data improve after implementation? If so, how?	Qualitative Quantitative	Semi-structured interview with jurisdictions (E) Data end-user survey (E) Aggregate statistics (J)
2c. Were new or expanded DSAs/MOUs established during implementation? If so, how and why?	Quantitative – number of new MOUs established Qualitative – extent of expanded MOUs	Semi-structured interviews with jurisdictions (E) Document review: Jurisdiction MOUs (J)
Aim 3: Assess to what extent the demonstration project had the intended impact on client, data end-user, and policy outcomes.
3a. Did data end-users (providers, clinical managers, disease intervention specialists, linkage to care coordinators) in participating jurisdictions increase use of linked data for STI screening and treatment and other targeted services over the implementation period?	Quantitative	Data end-user survey (E)
3b. How was linking of STI and HIV surveillance data associated with outcomes for the HIV continuum of care, including linkage, retention, re-engagement in HIV care, and viral suppression?	Quantitative	Aggregate statistics of client health outcomes (J) Linkage to care Engagement in care Re-engagement in care Viral suppression
3c. How did health policy stakeholders use the linked data? Were there specific policies changed or health events uncovered (e.g. clusters or pandemics) due to the information provided by the linked data?	Qualitative	Semi-structured interviews with policy stakeholders (E)
Aim 4: Conduct a cost benefit analysis among participating jurisdictions of linking HIV and STI surveillance data and using those linked data.
4a. Can the costs of the linking, including the creation and maintenance of sharing agreements, data warehouse creation and maintenance, data ingestion and standardization, linking, making the data available, and training to use the data be quantified?	Quantitative	Costs for linking activities – quarterly cost reports from jurisdictions/TAP (J/T)
4b. Can the benefits from jurisdictional health policy improvements or improvements to clinical delivery be quantified, in the form of improved policy solutions and improved client health (e.g. viral suppression, retention to care, STI management and cure)?	Quantitative	Aggregate statistics of client health outcomes and benefits gleaned from semi-structured interviews with jurisdictions and policy stakeholders (E)

Data Collection Activities Requiring OMB Approval

Semi-Structured Interviews

Semi-structured interviews will be conducted with the TAP, jurisdiction TA recipients, and jurisdiction policy stakeholders. Each type of interview will capture different information, as described in Exhibit 3.

Exhibit 3. Semi-Structured Interview Descriptions

Interview Type	Purpose
Activities Requiring OMB approval (n >10 for each form)
Jurisdiction TA Recipient (Attachment A)	This interview will be conducted with up to 3 key points of contact (POCs) in each of the 4 participating jurisdictions each project year (32 total interviews). These interviews will gather information about organizational processes associated with data sharing and linkage (i.e., pain or friction points associated with the process of sharing data such as stigma and confidentiality issues; facilitators to linking HIV and STI surveillance data; MOUs, policies, and procedures in place; perceived impact of linked data on process of care and HIV outcomes) and technical processes of data sharing and linkage (i.e. common data linkage processes, procedures, and best practices); and perceptions of the TA provided by the TAP. Interviews with jurisdictions will also identify suggestions for improvements to the TA and lessons learned during implementation.
Jurisdiction Policy Stakeholder (Attachment B)	This interview will be conducted with up to 3 policy stakeholders in each of the 4 jurisdictions in years 2 and 3 of the project (24 total interviews). The purpose of these interviews is to gather information about if and how linked HIV and STI data is being used for policy and/or decision- making, and/or reasons for not doing so. This round of interviews with policy stakeholders will begin in year 2, after jurisdictions have conducted substantial work toward linking HIV and STI data. Interview participants will be health department officials, legislators, or other department heads that work closely with the health department (e.g. Department of Corrections, Department of Social Services) in each jurisdiction.
Activities Not Requiring OMB approval (n <10)
TAP (Attachment C)	This interview will be conducted with 3 TAP representatives (Principal Investigator, Data Integration Specialist, and Care Continuity Specialist) each project year (3 total interviews). The purpose of the TAP interview is to gather information about perceived jurisdiction readiness and qualities, barriers experienced by jurisdictions, the TA process and activities, and modifications made to the TA approach.

Data End-User Survey

A short, web-based survey of data end-users (health department staff such as disease intervention specialists and data managers and/or clinicians who provide care to RWHAP clients) will be conducted. The number of data end-users varies by jurisdiction, but it is expected that approximately 35 staff in each of the 4 jurisdictions will participate in this survey (140 total staff). The survey will assess the extent to which these staff receive linked data in reports, the usability and understandability of the reports, and current and planned use of the linked data to target resources and improve care for co-infected clients. The survey will be conducted twice – in years 2 and 3 - to capture changes in data end-users’ perceptions and use of linked data over that time period. The expected response rate is 75% (105) at each time point, therefore approximately 210 total survey responses are expected to be received. The end-user survey can be found in Attachment D.

Data Collection Activities Not Requiring OMB Approval

OMB approval is not sought for the below-referenced data collection activities because the estimated number of respondents for each of these activities is less than 10 respondents.

Aggregate Statistics

Aggregate statistics for specific measures to support the evaluation will be collected, focusing on data system contents, linking results, and ultimate system and patient outcomes. These data will be collected from participating jurisdictions quarterly. These will include basic statistics – percentages, counts, means, medians, minimum, maximum, and standard deviations. Aggregate statistics measures are aligned to applicable HRSA HAB performance measures and to TA focus areas. Measures on STI and HIV surveillance will be collected, including additional measures on the HIV care continuum, outreach to those not engaged in care, screening and linkage to care of the partners of people with HIV for STIs and HIV in alignment with the statement of work. Finally, statistics about linking will also be collected, as well as activities to be conducted after linkages are made (e.g., identifying comorbidities/coinfection; outreach to those who become disengaged from care). The aggregate statistics template can be found in Attachment E.

Personnel Time Reports

A monthly time reporting form will be collected from each jurisdiction to inform the cost analysis. The time reporting form captures information on the hours spent by jurisdiction staff to conduct activities related to developing and implementing a process for linking HIV and STI data and using linked data to conduct outreach and follow up activities. The personnel time report template can be found in Attachment F.

Document Review

Several different types of documents developed by and/or administered by the TAP and jurisdictions will be collected as a secondary data source to inform the evaluation. Documents from the TAP will include programmatic TA documents between the TAP and jurisdictions, meeting notes, and needs assessment results. Documents from the jurisdictions will include data use and sharing agreements, data governance models, memoranda of understanding, and standard operating procedures.

This evaluation, like any study, has limitations due to the types of planned data collection.

These limitations may result in over or understatement of the conclusions. One potential limitation is that the semi-structured interviews and data end-user survey will gather self-report data. Self-reported data are at risk of threats to reliability and validity if participants do not understand the questions, provide a desired answer, or provide inaccurate or misleading information. Another limitation is that the data quality of the aggregate statistics and time reporting data collected may vary by jurisdiction. Some participating jurisdictions have better, cleaner data to work with and better data capacity than others. A third limitation relates to the document review - documentation of certain TA processes and activities may be better documented than others, which will impact the conclusions drawn when reviewing the documents. These limitations will be considered, and evaluation findings will reflect the context of these limitations. Essentially, results will be generalizable to only those jurisdictions that participate in this project. However, findings may translate and be informative to other, similar RWHAP jurisdictions.

3. Use of Improved Information Technology and Burden Reduction

All efforts have been made to minimize respondent burden, while obtaining the essential information needed to answer the evaluation questions. The use of web-based data submission methods decreases respondent burden as compared to that required for alternate methods, such as paper-based, by allowing direct transmission of data. This data collection effort includes primary (e.g., surveys, semi-structured interviews) and secondary (i.e., existing, extant) data achieved through a combination of electronic methods as described below.

Data Collection Activities Requiring OMB Approval

Semi-Structured Interviews

Semi-structured interviews will be conducted via videoconference. The policy stakeholders will be invited to participate in a brief, 15-30-minute interview. Informed consent will be obtained from each participant prior to all interviews, if required by the IRB. All interviews will be guided by an interview guide, and notes will be taken electronically during each interview. Interviews will be audio recorded so that audio files can be referenced if needed.

To further minimize burden, the semi-structured interview guides have been designed to ensure that the discussion is limited and the questions are well organized, flow well together, and are easy to understand and answer. Interviews will be scheduled at a date and time that is convenient for the interviewee.

Data End-User Survey

The web survey will be programmed using Confirmit software. Confirmit allows for the design and programming of input screens that visually guide respondents through the survey instrument and encourage accurate data entry. Beta testing and quality checks will be performed to ensure skip patterns and data are accurately captured. All survey programming, testing and refinement will be conducted within a secure Analytic Computing Environment (ACE3).

Survey administration will use a three-stage recruitment email approach whereby an initial email is sent to all data end-users introducing the evaluation and alerting potential respondents to the forthcoming survey. The second email will contain a live link to the web-survey, and approximately one week later a reminder email will be sent to all non-respondents. The TAP will be engaged to help facilitate outreach to jurisdictions ahead of the survey launch to ensure the maximum response rate possible. For example, a webinar describing the forthcoming survey and encouraging jurisdictions could be held to promote participation among their data end-users. Incentives will not be provided for completion of the web-survey. Therefore, the survey will be brief - approximately 10 questions for respondents to answer. The survey will be circulated over an eight-week period and survey response reports will be run every few days to assess response rates and promptly troubleshoot issues that arise during the survey response period.

Data Collection Activities Not Requiring OMB Approval

Semi-Structured Interviews

Semi-structured interviews will be conducted with the TAP via videoconference and are expected to last 60 minutes. The same informed consent and interview procedures described for the jurisdiction and policy stakeholder semi-structured interviews will be followed for the TAP interviews.

Aggregate Statistics, Personnel Time Reports, Document Review

Quarterly reports of aggregate statistics and monthly reports of cost data will be collected from jurisdictions electronically by the TAP and uploaded to Box File Transfer Protocol (FTP), a secure, file-sharing database used by the TAP. The contractor will then download those data to Huddle (the contractor’s secure file-sharing database) and will securely transfer them from Huddle to ACE3. For the document review, the TAP and participating jurisdictions will be uploading various documents to Box and an identical file transfer process from Huddle to ACE3 will occur. Data will not be shared with HRSA using Box. Any data that needs to be shared with HRSA will be transmitted securely via the file transfer protocol (FTP) and will be maintained on secure servers.

4. Efforts to Identify Duplication and Use of Similar Information

This evaluation collects information unique to Enhancing STI Linkage that is otherwise not available to HRSA HAB. With an eye toward minimizing duplication and burden, data collected from each instrument is non-duplicative and complementary to the other evaluation components and program monitoring tools. The overall evaluation strategy utilizes five sources of data: (1) semi-structured interviews; (2) data end-user survey; (3) aggregate statistics; (4) personnel time reports; and (5) existing documents. All of these data sources will require collection of information not already available within this request for OMB approval.

Because no data collection, federal or otherwise, contains comprehensive, high quality data representative of the RWHAP program to answer the key evaluation questions of this project, HRSA proposes the current information collection. This information collection will be the only comprehensive source of qualitative and aggregate quantitative information that will adequately address and meet HRSA’s data collection needs and objectives.

5. Impact on Small Businesses or Other Small Entities

Information collection will not have a significant impact on small entities. Data will be collected from state health departments, which do not constitute small businesses.

6. Consequences of Collecting the Information Less Frequently

Frequency of data collection is held to the minimum necessary to meet evaluation objectives and achieve evaluation aims. Plans for data collection activities were developed in conjunction with the TAP, and efforts were made to consider and balance data collection burden with the ability to achieve evaluation aims. For example, the evaluation design includes reviewing existing documents to glean needed information. Additionally, aggregate statistics measures were aligned to those jurisdictions that already report to HRSA and the Centers for Disease Control and Prevention (CDC) including: infection and comorbidity rates, rates of linking new HIV-positive individuals to care, and retention in HIV care. These data collection strategies reduce the need for primary data collection. However, to achieve the evaluation objectives, the interviews and surveys described will need to be conducted. Data collection tools and frequency of use are shown in Exhibit 4.

Exhibit 4. Enhancing STI Linkage Evaluation Data Collection Tools & Frequency

Instrument	Data Collection Method	Frequency of Data Collection	Maximum Number of Data Collections	Attachment Number
Activities Requiring OMB Approval (n >10 for each form)
Jurisdiction TA Recipient Semi-Structured Interview Guide	Jurisdiction interviews will be recorded and transcribed; stored on secure drive	Yearly	3 times: Years 1-3	Attachment A
Policy Stakeholder Semi-Structured Interview Guide	Policy Stakeholders interviews will be recorded and transcribed; stored on secure drive	Yearly	2 times: Years 2-3	Attachment B
Data End-User Survey	Data End-Users complete electronically via Confirmit	Yearly	2 times: Years 2-3	Attachment D
Activities Not Requiring OMB Approval (n <10 for each form)
TAP Semi-Structured Interview Guide a	TAP interviews will be recorded and transcribed; stored on secure drive	Yearly	3 times: Years 1-3	Attachment C
Aggregate Statistics a	Jurisdictions will upload forms to Box FTP	Quarterly	6 times: Years 2-3	Attachment E
Personnel Time Reports a	Jurisdictions will upload forms to Box FTP	Monthly	18 times: Years 2-3	Attachment F
Document Review a	Documents requested from TAP and Jurisdiction TA Recipients, who upload the documents to Box FTP	Quarterly	6 times: Years 2-3	Attachment not applicable

^a These data collection activities will involve data collection from four or fewer respondents and OMB approval is not being requested for these activities.

Less frequent data collection will hinder the ability to describe how TA activities evolve as well as how barriers change over the course of the project. Additionally, less frequent data collection will hamper the ability to assess how jurisdictions’ processes and outcomes change over the course of the project.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

The request fully complies with the regulation.

8. Comments in Response to the Federal Register Notice/Outside Consultation

Section 8A:

A 60-day Federal Register Notice was published on August 20, 2020, vol. 85, No. 162; pp.51454-51455. There were no public comments.

Section 8B:

HRSA contractor consulted with Janet Myers, PhD, Professor of Medicine, University of California, San Francisco for feedback on the evaluation design, approach, and data collection plans. Additionally, input was obtained from the TAP - Georgetown University - which included Auntre Hamp, Med, MPH, LPC, Research Assistant Professor, School of Medicine and Ann Marie Stringer, Data Integration Specialist.

9. Explanation of any Payment/Gift to Respondents

Respondents will not receive any payments or gifts.

10. Assurance of Confidentiality Provided to Respondents

Confidentiality, integrity and availability of study data will be protected in the following ways. First, access to data will be restricted to authorized personnel by assigning permission to access specific project folders which are maintained behind multiple firewalls. Second, anti-spam, anti-malware, and anti-intrusion software will be used to guard against data breaches. Third, secure external access to information systems will be ensured by using a Virtual Private Network (VPN) and an encrypted secure file transfer portal.

All data, including PII will be stored and accessed only in access-restricted folders on ACE3. ACE3 is FISMA Moderate and FIPS 140 2 compliant, and requires DUO multi-factor authentication to access. Any data transferred will occur through one of two secure data transfer applications, Huddle or Box. Huddle and Box both provide secure, encrypted transmission of PII to and from study partners, and are FedRAMP certified and FIPS 140-2 compliant. Only authorized study team members will have access to the Huddle and Box project folders through an approval process.

In summary, as part of the data collection effort:

• For the interviews and survey of data end-users,

  • Individual level data will be collected;

  • Personally identifiable information (PII) will be collected from survey and stakeholder interview participants; these PII will include full names and email addresses only

  • All PII will be stored on ACE3 separately from survey responses

  • Data reported will be blinded, and aggregated at the jurisdiction level

  • Any data provided to the client will be de-identified and coarsened

  • Any data that needs to be shared with HRSA will not contain PII and will be transmitted securely via the file transfer protocol (FTP) and maintained on secure servers.

• For the aggregate statistics and personnel time reports:

  • No direct identifiers will be collected;

  • Protected Health Information (PHI) will not be collected (though participating sites will create aggregate files using person-level, PHI-containing datasets before transferring aggregate-only data)

  • Data will be aggregated by the jurisdiction into subgroups large enough so no individual may be identified

  • Data will not be shared with HRSA using Box. Any data that needs to be shared with HRSA will be transmitted securely via the file transfer protocol (FTP) and maintained on secure servers.

The following approaches will be used to collect, transfer and store project data:

Semi-Structured Interviews: Interviewee contact information will be collected from the TAP, and securely uploaded to Box. The contact information shared will be immediately retrieved and downloaded from Box to ACE3, and then deleted from Box. Interviews will be arranged and recorded via WebEx. Notes will be taken by in a Word document saved within ACE3, and stored there. Recordings will be uploaded to ACE3, via Huddle, immediately following the end of the interview and deleted from the WebEx platform.

Data End-User Survey: A census list of appropriate data end-users’ contact information (email addresses and full names) will be requested and obtained from participating jurisdictions and securely uploaded to Box. Survey invitations will be sent via Confirmit, from within ACE3’s secure environment. All data will be collected and stored on ACE3 for the duration of the contract. Email addresses and names will not be merged with survey responses.

Personnel Time Report Data: A blank time reporting template will be shared with participating jurisdictions via Box. Jurisdictions will then populate and return the form - on a monthly basis -via Box. All cost data will be directly saved to and remain on ACE3 for the duration of the contract. When cost data include personnel costs, we will request job classification descriptions rather information that identifies individuals.

Aggregate Statistics: A blank aggregate statistics reporting template will be shared with participating jurisdictions via Box. Jurisdictions will then populate and return the form - on a quarterly basis - via Box. GU will ensure Box sub-folders are blinded such that participating jurisdictions cannot view one another’s folders or uploaded materials. All aggregate statistics data will be directly saved to and remain on ACE3 for the duration of the contract.

Data will be kept private to the extent allowed by law. Key client data destruction requirements associated with tasks will be adhered to, and data will be stored/collected relevant to the current project as detailed below:

1. General Security Requirements. The Contractor (and/or any subcontractor) shall comply with information security and privacy requirements, Enterprise Performance Life Cycle (EPLC) processes, HHS Enterprise Architecture requirements to ensure information is appropriately protected from initiation to expiration of the contract. All information systems development or enhancement tasks supported by the contractor shall follow the HRSA EPLC framework and methodology (https://sharepoint.hrsa.gov/oo/oit/dcppm/pmo/Shared Documents/z_Old PMO Archives/Documents.aspx) and in accordance with the HHS Contract Closeout Guide (2012).

2. Sanitization of Government Files and Information. As part of contract closeout and at expiration of the contract, the Contractor (and/or any subcontractor) shall provide all required documentation to the CO and/or COR to certify that, at the government’s direction, all electronic and paper records are appropriately disposed of and all devices and media are sanitized in accordance with NIST SP 800-88, Guidelines for Media Sanitization.

3. Notification. The Contractor (and/or any subcontractor) shall notify the CO and/or COR and system ISSO within 10 days before an employee stops working under this contract.

11. Justification for Sensitive Questions

The information reported by respondents for the Enhancing STI Linkage evaluation does not ask for sensitive personal information or include questions of a sensitive nature. The focus of the Enhancing STI Linkage evaluation data collection is to determine the effectiveness of the training and TA provided to the RWHAP jurisdictions. Respondents will mostly provide information about their organizations and organizational activities. Information collected in the interviews and web-survey will ask individuals about their experiences and use of data, but these questions are not designed to collect overly personal or sensitive information.

12. Estimates of Annualized Hour and Cost Burden

This section provides annualized burden hours for each Enhancing STI Linkage evaluation data collection activity described in this OMB package. Total burden hours for this entire OMB package is 80 hours and $3,822.38, shown in Exhibit 5. Data collection for Year 1 will only involve semi-structured interviews and those annualized burden hours are shown in Exhibit 6. Years 2 and 3 will be inclusive of additional data collection activities and Exhibit 7 reflects the annualized burden hours for those years. Time estimates are based on experience with similar instruments in other studies of comparable organizations.

Exhibit 5. Total Data Collection Burden Hours

Study Name	Total Burden Hours	Total Wage Burden
Enhancing STI Linkage Evaluation	80	$3,822.38

12A. Estimated Annualized Burden Hours

In year one of the Enhancing STI Linkage evaluation, individual, 60-minute semi-structured interviews will be completed with up to eight jurisdiction TA recipients (1-2 per jurisdiction). Because only one data collection activity will be implemented in year one, the total estimated burden hour for this year is 8 hours.

Exhibit 6. Annualized Burden Hours Estimate – Year 1

Type of Respondent		Form Name	No. of Respondents	No. Responses per Respondent	Total Responses	Average Burden per Response (in hours)	Total Burden Hours
	Activities Requiring OMB Approval (n >10 for each form in Years 2 and 3)
Jurisdiction TA Recipient		Jurisdiction TA Recipient Semi-Structured Interview Guide	8	1	8	1	8
		Total	8		8		8

In years two and three of the Enhancing STI Linkage evaluation, additional data collection activities will be employed. These include two types of semi-structured interviews with the the jurisdiction TA recipients (twelve 60-minute, individual interviews per year), and the policy stakeholders (twelve 30-minute individual interviews per year). A 10-minute, electronic data end-user survey will be collected two times per year from 105 expected respondents across the four jurisdictions. The combined total burden hours for years 2 and 3 is 72 hours.

Exhibit 7. Annualized Burden Hours Estimate – Years 2 and 3

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Total Responses	Average Burden per Response (in hours)	Total Burden Hours
	Activities Requiring OMB Approval (n >10 for each form)
Jurisdiction TA Recipient	Jurisdiction TA Recipient Semi-Structured Interview Guide	12	2	24	1	24
Policy Stakeholder	Policy Stakeholder Semi-Structured Interview Guide	12	2	24	0.5	12
Data End-User	Data End-User Survey	105	2	210	0.17	36
	Total	129		258		72

12B. Estimated Annualized Burden Costs

Exhibit 8 depicts $409.84 as the annualized cost burden for year 1 of the Enhancing STI Linkage evaluation. Exhibit 9 presents $3,412.44 as the combined annualized cost burden for years 2 and 3 of the Enhancing STI Linkage evaluation. There are no direct costs to respondents other than their time to complete the data collection instruments. Hourly wage rates are based on the mean hourly wage rates as reported in the 2019 Occupational Employment (OES) by the Bureau of Labor Statistics (BLS) found at https://www.bls.gov/oes/current/oes_nat.htm#b29-0000.htm.

Exhibit 8. Annualized Cost Burden – Year 1

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Administrative Services Manager a	8	$51.23	$409.84
Total	8		$409.84

^a Jurisdiction TA Recipients were classified as the administrative services manager respondent type and the identified hourly wage is based on the mean hourly wage for management analysts as reported in the 2019 OES by the BLS found at https://www.bls.gov/oes/current/oes_nat.htm#b29-0000.htm.b.

Exhibit 9. Annualized Cost Burden – Years 2 – 3

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Administrative Services Manager a	36	$51.23	$1,844.28
Operations Research Analyst b	36	$43.56	$1,568.16
Total	72		$3,412.44

^a Jurisdiction TA Recipients and Policy Stakeholders were classified as the administrative services manager respondent type and the identified hourly wage is based on the mean hourly wage for administrative service managers as reported in the 2019 OES by the BLS found at https://www.bls.gov/oes/current/oes_nat.htm#b29-0000.htm.b.

^bData End-Users were classified as the operations research analyst respondent type and the identified hourly wage is based on the mean hourly wage for operations research analysts as reported in the 2019 OES by the BLS found at https://www.bls.gov/oes/current/oes_nat.htm#b29-0000.htm.b.

13. Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/Capital Costs

Other than their time, there is no cost to respondents.

14. Annualized Cost to Federal Government

The total cost of this fixed-price contract that supports this three-year information collection is $1,814,168, annualized to $604,723. This includes the labor costs to design the evaluation, develop the data collection instruments, conduct pilot data collection, scale up the full evaluation, and to analyze the survey, interview, document review, aggregate statistics, and cost data. Additionally, there will be the cost for a GS 14 (Step 6) at 5% time (approximately $5,424) time to monitor the project. The average annual total cost of the project is $610,147 and the total cost of the three-year project is $1,830,441.

15. Explanation for Program Changes or Adjustments

This is a new information collection.

16. Plans for Tabulation, Publication, and Project Time Schedule

A mixed-methods evaluation of the data collected will be conducted. At the end of each year (and again at the end of the evaluation itself), summative reports, project briefs and briefing materials appropriate for and actionable to stakeholders involved will be developed. All final materials will be made Section 508-compliant prior to dissemination. The contractor will also collaborate with HRSA HAB to develop materials that would be of interest to HRSA HAB’s website (TargetHIV), the AIDS Education and Training Center National Coordinator Resource Center website, and other venues to be determined during the period of performance that will fit within the project’s allocated budget. This also includes publications in peer-reviewed journals, as appropriate. Exhibit 10 represents a timeline for data collection and reporting benchmarks for the Enhancing STI Linkage evaluation.

Exhibit 10. Time Schedule for Enhancing STI Linkage Data Collection

Activity/Deliverable		Target Timeline
Preparation for Qualitative Analysis and Instrument Refinement		April – July 2020
Semi-Structured Interviews	TAP Interviews	August 2020; February – March 2021; February – March 2022
	Jurisdiction Interviews	July 2020; March – May 2021; March – May 2022
	Policy Stakeholder Interviews	January – May 2021; January – May 2022
Document Review		September 2020; March – May 2021; March – May 2022
Data-End User Survey		May – August 2021; February – May 2022
Aggregate Statistics		September 2020 – July 2022
Personnel Time Reporting		September 2020 – July 2022

17. Reason(s) Display of OMB Expiration Date is Inappropriate

The OMB number and Expiration date will be displayed on every page of every form/instrument.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

1 Health Resources and Services Administration. Ryan White HIV/AIDS Program Annual Client-Level Data Report 2016. http://hab.hrsa.gov/data/data-reports. Published December 2018. Accessed September 9, 2019.

2 https://hab.hrsa.gov/about-ryan-white-hivaids-program/part-f-special-projects-national-significance-spns-program

3