Supporting Statement A

06681 Mathematica Policy Research

Supporting Statement A

Justification for the

Healthy Weight Collaborative implementation study

A Justification

1. Circumstances Making the Collection of Information Necessary

Supported through the Public Health and Prevention Fund created by the Affordable Care Act of 2010 (see Appendix A, Affordable Care Act of 2010 [Sections 4002 and 10401], the U.S. Department of Health and Human Services’ Health Resources and Services Administration (HRSA) awarded $5 million to the National Initiative for Children’s Healthcare Quality (NICHQ) in fiscal year (FY) 2010 through March 31, 2013, to create and manage a prevention center for healthy weight to address obesity in children and families, which created the Collaborate for Healthy Weight initiative. Collaborate for Healthy Weight is a national initiative by HRSA to bring together primary care providers, public health professionals, and leaders of community-based organizations to use quality improvement methods to address the obesity epidemic in communities across the country.

As part of Collaborate for Healthy Weight, in collaboration with HRSA and a coalition of partner organizations, NICHQ has created the Healthy Weight Collaborative (HWC) to identify model approaches linking clinical services, public health functions, and other community services to address childhood and family obesity. The HWC aims to prevent and treat childhood obesity by working with 50 community teams to identify, test, and evaluate a national HWC change package of evidence-based program and policy interventions. Each community team represents multiple sectors, including primary care, public health, and other community sectors, such as education. The HWC is modeled after the Breakthrough Series methodology, a quality improvement approach pioneered by the Institute for Healthcare Improvement (IHI), in which learning sessions (that is, meetings) are interspersed with action periods (that is, times to test interventions in local environments).

The HWC is being implemented in two consecutive program implementation phases, each with 3 learning sessions and action periods. These program implementation phases represent activities conducted by community teams that were supported by a cooperative agreement and did not include any evaluation activities. The first program implementation phase (July 2011 to July 2012) included 10 community teams selected from 10 regions of the country, in which the teams attended two in-person learning sessions conducted at the beginning and end of the phase, and one online learning session conducted mid-term. Between meetings, the community teams worked on team projects, participated in monthly webinars other action calls and coaching sessions, used quality improvement methods to improve their strategies, and collected and submitted performance measurement data identified by HRSA, NICHQ and team representatives to the HWC.

Quality improvement methodology involves conducting small tests of change. Community teams use a Plan-Do-Study-Act framework to test and implement changes, in which they Plan the test, Do the test, Study the results to build new knowledge, and Act based on what is learned from testing. In June 2012, the 10 program implementation Phase 1 teams were invited to remain involved in the second program implementation phase of the project as unpaid volunteers, by serving as mentors to Phase 2 teams, and by continuing to implement their HWC strategies and report on their team’s HWC performance indicators. Of the 10 teams, four volunteered to continue in Phase 2 of program implementation. How they will sustain their HWC strategies and how they will contribute to Phase 2 of program implementation, are parts of the project’s implementation study.

The second program implementation phase of the HWC (March 2012 to March 2013) will include about 40 additional community teams recruited from across the country that will attend three series of online learning sessions, at the beginning, middle, and end of Phase 2 of program implementation. In addition, the Phase 2 program implementation teams will participate in monthly action calls and in group coaching and technical assistance sessions between the three learning series. The Phase 2 community teams conducting program implementation activities are expected to implement the same HWC change package of strategies used in Phase 1 program implementation, and are expected to collect and report on an expanded list of HWC performance indicators, tracking their progress.

In addition to community team input, three groups of NICHQ staff also play key roles in the project’s design and implementation. First, the NICHQ leadership directs the HWC project’s design and implementation; the leadership includes NICHQ’s associate project director, physician champion, and senior project managers of Quality Improvement and Technology, as well as the HWC project’s Quality Improvement advisor. In addition, the project has contracted with a team of technical experts to serve as HWC faculty, who facilitate the project’s learning sessions, webinars, and team coaching sessions. The NICHQ faculty includes staff from nationally-recognized childhood obesity prevention initiatives, including the Let’s Go program in Maine, and staff from the Nemours Foundation in Delaware. Last, a group of NICHQ project managers work closely with the community teams, serving as project liaisons and responding to team questions and requests for assistance. In Phase 2 program implementation, the program managers are more involved in coaching and providing technical assistance to the teams, conducting group coaching calls to clusters of teams. To obtain a complete picture of the successes and challenges of the HWC project, it will be important to gather information from these three NICHQ groups as well as from the project’s community teams.

HRSA is seeking OMB clearance to gather information from the HWC community teams and NICHQ leaders, staff, and faculty in order to conduct an implementation study of Phase 2 of program implementation of the HWC project. The goal of the implementation study is to document the ongoing development and refinement of the HWC project’s “change package,” in Phase 2 of program implementation of the project, as well as the adaptation of the project’s learning collaborative process in a virtual format with no in-person meetings. The study will also document the participation of the 40 Phase 2 community teams, their experiences, and their results. HRSA has hired Mathematica Policy Research, to conduct the implementation study.

2. Purpose and Use of Information Collection

The Phase 2 implementation study will provide recommendations on how to improve the design and implementation of the HWC project, including how to improve the scope of the HWC project’s change package; the project’s implementation process (that is, the learning sessions, community team activities, coaching calls and webinars, and the iLab); and the content of the project’s performance measures. The implementation study will also generate suggestions for how the NICHQ faculty and staff can help community teams optimize their implementation of the HWC change package to meet the HWC initiative’s goals.

The HWC implementation study involves several key data collection activities in Phase 2 that will shed light on how effective the program is at creating linkages across sectors and addressing childhood obesity, and on ways in which the second phase of the program can be improved to make it more effective. The study team will collect and analyze several kinds of qualitative data: interviews with community teams; monthly observations of HWC learning sessions, webinars, group coaching calls, and other project activities; and reviews of project documents posted on NICHQ’s iLab, including learning series agendas and presentations, community team work plans, progress reports, and HWC performance measures.

Site Visits with Continuing Phase 1 HWC Community Teams. Between February and April 2013, the implementation study team will conduct one round of site visits with 4 of the original 10 Phase 1 community teams that are continuing to participate in the HWC project in Phase 2. These site visits will enable the implementation study team to gather information about the continuation and sustainability of the teams’ HWC activities in Phase 2, the teams’ involvement in Phase 2 activities, and their feedback on the Phase 2 implementation process and Phase 2 performance measures. The four teams will be selected based on their continued participation in the project.

For each of these four community teams, a two-person study team (a researcher and an analyst) will conduct a site visit, which will include individual interviews with the team’s lead and the team’s data manager, and a group interview with the rest of the community team (six members). The interview with the team’s lead will include questions about the team’s Phase 2 goals, work plan, structure, funding, and any challenges sustaining and spreading the team’s HWC activities. The interview with the team’s data manager will include questions about the team’s collection and reporting of the expanded set of Phase 2 HWC performance indicators, the team’s collection of other quality improvement indicators, and any challenges the team experienced meeting the project’s expanded Phase 2 data requirements. Topics for the group interview will cover the team’s ongoing engagement in the project as Phase 2 mentors; sustained community-level results of the team’s ongoing efforts in Phase 2; and the team’s feedback and recommendations for how to improve the HWC’s project’s key components, including the change package, format and content of the virtual learning series, the iLab functionality, and the value of collecting and reporting the expanded set HWC performance measures. (See Appendix C: Phase 1 Community Team Sustainability Site Visit Protocols.)

Site Visits with Phase 2 HWC community teams. Between February and March 2013, the implementation study team will also conduct one round of site visits with 7 of the approximately 40 Phase 2 community teams. These visits will enable the implementation study team to gather information on the Phase 2 teams’ experiences with the HWC project, including their feedback on the project’s change package, implementation process, and HWC performance measures. The 7 teams will be selected to meet several criteria, including their project’s geographic location, target population(s), and content of their work plans.

For each of the seven Phase 2 teams, a two-person team (a researcher and an analyst) will conduct a site visit, which will include individual interviews with the team’s coordinator and the team’s data manager, and a group interview with the rest of the community team (6 members). The interview with the team’s lead will include questions about the team’s goals, work plan, structure, funding, and any challenges participating in the project. The interview with the team’s data manager will include questions about the team’s collection and reporting of the 13 Phase 2 performance indicators, the team’s collection of other quality improvement indicators, and any challenges the team is experiencing meeting the project’s data requirements. Topics for the group interview will cover the team’s engagement in Phase 2; the team’s participation in HWC events and activities; community-level results of the team’s Phase 2 efforts; and the team’s feedback and recommendations for how to improve key project components, including the change package, the virtual learning session format and content, the functionality of the iLab, the value of the project’s coaching and technical assistance from the program managers, and faculty, and the adequacy of the funding and other supports provided. (See Appendix D, Phase 2 Community Team Site Visit Protocols.)

Group interviews with NICHQ leaders, faculty members, and project managers. To gain a broader understanding of the HWC, we also seek to learn about the experiences of three other groups involved in the project: the NICHQ leadership, HWC faculty members, and HWC project managers. We will conduct one round of individual interviews with the leaders and project managers, and a group interview with the faculty between February and April, 2013. They will be able to provide insights into the implementation of Phase 2 of the HWC and identify lessons learned regarding the HWC project’s design and operation as a virtual LC. Topics of these interviews will cover the quality of the implementation of HWC events and activities, the engagement of the community teams in the HWC activities, and community-level results of the teams’ efforts. We will also use the discussions to gather the NICHQ groups’ own reflections and suggestions for how to improve the project’s key components, including the change package, the format and content of the Phase 2 learning series, , the functionality of the iLab, the value of the project’s performance measures, and the adequacy of the funding and other supports provided. (See Appendix E).

Other non-burden baring data collection activities. HRSA is not seeking clearance for these non-burden baring activities and describes them only for the sake of completing the picture of HWC data collection. The Mathematica implementation study team will also conduct other data collection activities that do not pose a burden on any respondents, but will provide important contextual information on the project. These activities include observations of the virtual HWC learning sessions that will document the content, process, and effectiveness of the HWC learning sessions and provide background information on the activities and progress of the approximately 40 teams participating in the Phase 2 activities. The implementation study team members will act as silent observers, not active participants in these events, in order to minimize any potential intrusion or disruption of these activities.

The Mathematica study team will use a standard observation template to record these observations (see Appendix F: Learning Session Observation Template). On a monthly basis, or as needed, Mathematica team members will also (1) observe HWC monthly training webinars to understand ongoing team development activities and opportunities provided by NICHQ and interactions among community teams; (2) review documentation of iLab activity to track sharing of documents and other resources among teams and with NICHQ; and (3) abstract monthly project performance measurement data submitted by the community teams to the iLab.

3. Use of Improved Information Technology and Burden Reduction

Because all aspects of the project’s Phase 2 data collection (the individual and group interviews, observations of learning series webinars, and review of electronic and paper project documents) will involve the use of qualitative data collection methods, there is not an opportunity to use information technology (IT) methods to reduce respondents’ reporting burden; HRSA will not use IT to collect this information. The interview data will be collected orally using the attached site visit and interview guides. This process will be supported by digital recordings that the study team will use to ensure the accuracy of the interview notes. Although this does not affect respondent burden per se, the study team will analyze the interview notes using Atlas.ti, a software system used for the qualitative analysis of large bodies of textual data.

4. Efforts to Identify Duplication and Use of Similar Information

The individual and group interviews are intended to collect information for a new HRSA project; the information collected does not duplicate any other HWC implementation study activities. The interview guides have been developed specifically for the HWC implementation study. The information collected through the individual and group interviews will provide a unique and important opportunity to understand the experiences of the community teams, and the NICHQ leadership, the faculty, and project managers involved in Phase 2 of the HWC project.

5. Impact on Small Businesses or Other Small Entities

The HWC community teams are composed of physicians, academics, program managers, and agency administrators from several community sectors including: primary care clinics, local public health department and city planning agencies, and community-based organizations including school districts, YMCAs, and community health coalitions. Team members can also be managers or administrators of child care centers, nonprofit organizations, and hospitals. NICHQ staff and faculty are not considered to be employees of small businesses or other small entities.

A small administrative burden will be placed on small organizations when their personnel will be invited to participate in the one-time site visit interviews. These interviews will be conducted in person with the team lead, data manager, and other community team members. To minimize burden, the duration of any interview will be limited to 90 minutes and the times and locations of the interviews will be convenient for all respondents.

6. Consequence of Collecting the Information Less Frequently

The interviews will be conducted with the community teams on a one-time basis; that is, the teams interviewed will not be interviewed again. Data collected from observation of Phase 2 learning sessions and iLab activities, and the Phase 2 performance measurement data submitted by HWC teams to the iLab, will not cause any additional burden on HWC participants. There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with 5 CFR 1320.5. There are no special circumstances.

8. Comments in Response to the Federal Register Notice/Outside Consultation

Public comments and responses. There were no public comments to the 60-day Federal Register Notice published on July 27^th (Nol. 77, No. 145). We have reserved Appendix G as a placeholder for any comments based on the 30-day Federal Register Notice.

Consultation outside the agency. In preparing this Office of Management and Budget (OMB) package, HRSA team pilot tested the community team interview protocol with one of the Phase 1 community teams in March 2012, consisting of seven respondents. The implementation study team used this experience to revise the content and reduce the length of the Phase 2 interview guides (see attached appendices). In addition, the study team reviewed several site visit interview protocols previously used in other similar studies under contract with HRSA and used that information to guide the general format and length of the HWC protocols.

9 Explanation of Any Payment/Gift to Respondents

No payments will be made to community team members or NICHQ leaders, faculty, or project managers for their participation in the interviews.

10. Assurance of Confidentiality Provided to Respondents

HRSA will take several steps to assure respondents that the information they provide through Phase 2 interviews, iLab and learning session activities, and submission of HWC performance measures will be kept private to the extent allowed by law and will be used for informational purposes only. Before each individual or group interview, the interviewer will read a confidentiality statement assuring the respondents that none of their comments will be attributable to them individually or to their organization, that their participation is voluntary, and that their decision to participate will have no impact on their HWC funding. Before the site visit interviews, Mathematica will also send the team leads an email requesting their team’s participation and containing the same set of assurances. In addition, interview participants will be asked to sign written consent forms that repeat the assurances listed above (see Appendix B).

After each interview, interviewers will summarize discussion notes in Microsoft Word files that will be saved on a secure server. Notes from the Phase 2 learning sessions, observations of iLab activities, and performance measurement data submitted by HWC teams will also be stored on the secure server. The notes will not identify speakers by name and the moderator will not use names in the notes. All implementation study team staff sign a confidentiality pledge as a term of employment; the confidentiality pledge requires that staff maintain the confidentiality of all information collected. The study team protects its local area network (LAN) with several security mechanisms available through the network operating system. Access to private information stored on LAN directories is restricted to authorized project staff by means of IDs and passwords. In addition, network servers containing private information are kept in a locked area. At the end of the project, tape recordings from the interviews will be destroyed.

11. Justification for Sensitive Questions

The Phase 2 interview guides do not contain any sensitive questions. Respondents will not be asked about their personal information; rather, they will be asked for information about their professional background, project goals and objectives, involvement in Phase 2 implementation activities, and perceived outcomes of the HWC project. The interviews will cover the respondent’s engagement in Phase 2 of the project; participation in specific HWC events and activities; community-level results of team efforts; and feedback and recommendations for how to improve key project components, including the HWC change package, the format and content of the virtual learning series, , the functionality of the iLab, the value of collecting and reporting the project’s performance measures, and the adequacy of the funding and other supports provided to the Phase 1 and Phase 2 teams.

12. Estimates of Annualized Hour and Cost Burden

Tables A.1 and A.2 present our estimates of respondent burden for participating in the Phase 2 interviews. The tables show the expected number of respondents, the hours per response, and the annual hour and total cost burden for the data collected. A total of 103 respondents will be included in the Phase 2 individual and group interviews:

1. Community teams. HRSA will conduct site visits with four continuing Phase 1 community teams and site visits with seven Phase 2 community teams. Each site visit is expected to include interviews with eight people: one team lead, one team data manager, and six team members. Each team lead interview will last an average of 1.5 hours. Each data manager interview will last an average of 30 minutes. Each group interview with the other team members is expected to last an average of 1.5 hours. Community teams are composed of a wide range of professions, encompassing multiple sectors: primary care, public health, community-based organizations, and schools. We estimate the median wage for each of these three groups, based on the U.S. Department of Labor, Bureau of Labor Statistics “2010 National Compensation Survey,” to be $28.12, $27.86, and $27.86 per hour for each sector, respectively. Each of the 11 community teams will be visited only once.

2. NICHQ leadership. We will conduct individual interviews with up to four NICHQ leadership staff. The each interview is expected to last up to one hour. We estimate the median wage, based on the Department of Labor, Bureau of Labor Statistics “2011 National Compensation Survey,” to be $37.45 per hour for anticipated members of the NICHQ leadership team.

3. HWC faculty. We will conduct one group interview with up to six HWC faculty members. The group interview is expected to last 1.5 hours. We estimate the median wage, based on the Department of Labor, Bureau of Labor Statistics “2011 National Compensation Survey,” to be $27.86 per hour for anticipated members of the HWC faculty.

4. HWC project managers. We will conduct individual interviews with up to five HWC project managers. The interview is expected to last up to one hour. We estimate the median wage, based on the Department of Labor, Bureau of Labor Statistics “2011 National Compensation Survey,” to be $27.86 per hour for anticipated members of the HWC faculty.

No other data collection activities involve participant burden.

Table A.1. Estimated Annualized Burden Hours

Type of Respondent	Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours
Phase 1 Team Lead	4	1	1.5	6.0
Phase 1 Team Data Manager	4	1	.5	2.0
Phase 1 Team Group Interview	24	1	1.5	36.0
Phase 2 Team Lead	7	1	1.5	10.5
Phase 2 Team Data Manager	7	1	.5	3.5
Phase 2 Team Group Interview	42	1	1.5	63.0
NICHQ Leaders	4	1	1.0	4.0
NICHQ Project Managers	5	1	1.0	5.0
NICHQ Faculty	6	1	1.0	6.0
Total	103	--	--	136.0

Table A.2. Estimated Annualized Burden Costs for Phase 1

Type of Respondent	Total Burden Hours	Hourly Wage Rate ($)	Total Respondent Costs ($)
Community Teams
Primary care	42	28.12	1,181.04
Public health	42	27.86	1,170.12
Community-based organization	41	27.86	1,142.26
NICHQ Leadership	4	36.89	147.56
HWC Faculty	9	27.86	250.74
HWC Project Managers	5	27.86	139.30
Total	136	-	4,031.02

13. Estimates of Other Total Annual Cost Burden to Respondents

There are no capital and start-up costs to respondents associated with this data collection. The interviews are a one-time data collection effort.

14. Annualized Cost to the Federal Government

The overall implementation study of Phases 1 and 2 will take place over a 4-year period. The total cost of the overall implementation study to the government is $640,658, which includes the amount awarded via contract to Mathematica ($595,058) and HRSA staff time/resources ($45,600). The total implementation study cost was based on the budget developed by Mathematica that calculated wages and hours for all staff, all mailing costs, telephone charges, and overhead costs per contract year along with the Government staff costs. The annualized contrast cost has been determined to be $160,164.50 per year by dividing the total funded amount by four years. This OMB review is for Phase 2 of the implementation study.

15. Change in Burden

Not applicable. This is a new data collection activity.

16. Plans for Tabulation and Publication and Project Time Schedule

The Phase 2 HWC primary data collection will begin upon receiving OMB clearance. Phase 2 data collection will be conducted between February and April 2013. Data will be collected only once and the Phase 2 field period is expected to be 16 weeks. The study team will deliver a final report summarizing and synthesizing findings from the interviews and other data sources. The final HWC project implementation study report will include a qualitative analysis of the Phase 2 interviews with the community teams, the NICHQ interviews, a descriptive analysis of the Phase 2 teams’ performance measurement data submitted to NICHQ’s iLab, a content analysis of team documents submitted to the iLab, and a descriptive analysis of findings from NICHQ’s Phase 2 collaboration and feedback surveys of community team members. The report will summarize key findings, provide feedback on the overall quality of the implementation of the HWC project, describe the accomplishments of the participating teams, and recommend ways in which the HWC model can be improved in future learning collaboratives.

Table A.3. Project Time Schedule

Deliverable	Due Date
Select/Recruit Sites	December 2012 - February 2013
Interviews	February - April 2013
Final Report	June – August 2013

17. Reason(s) Display of OMB Expiration Date Is Inappropriate

There are no exceptions to the certification; the expiration date will be displayed on the letters that will be sent to respondents before the individual and group interviews.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Phase 2 data collection efforts for the interviews will conform to all provisions of the Paperwork Reduction Act.

6