0990-HPUS 2014 FINAL OMB package_Revised 6.10.15

Healthy People is a national health promotion and disease prevention initiative. The Healthy People initiative has provided a comprehensive set of data-driven, national disease prevention and health promotion objectives with 10-year targets aimed at improving the health of all Americans since 1979. Healthy People 2020 (HP2020) is the fourth iteration of the Healthy People initiative. Its overarching goals are: to attain high-quality, longer lives free of preventable disease, disability, injury, and premature death; to achieve health equity, eliminate disparities, and improve the health of all groups; to create social and physical environments that promote good health for all; and to promote quality of life, healthy development, and health behaviors across all life stages. HP2020 consists of over 1200 objectives organized under 42 topic areas.

In light of the tremendous collective energy that goes into developing the initiative, it is important to assess how the key target audiences are using Healthy People, and identify barriers to the initiative’s success. The goal of this assessment is to create a comprehensive picture of how HP2020 contributes to state, local and tribal disease prevention and health promotion planning. Additionally, this study will examine two new populations of Healthy People users: individuals who have attended a Healthy People webinar and members of the Healthy People Consortium. These respondents will provide new information regarding how Healthy People is used by different types of organizations. HHS is eager to document the utilization of HP2020, and to seek input from key users on how the next iteration of the initiative, Healthy People 2030 (HP2030), could be improved to encourage greater involvement. Since the last iteration of HP2010, HHS has transformed the initiative from a print to a web-based platform, implementing a number of online tools and activities. This study will determine which elements of HP2020 are most utilized by organizations. Finally, the study will identify barriers to implementation and use at a point in time when HHS could take action to facilitate or support use in the forthcoming HP2030. The main research questions include:

Are organizations aware of HP2020, and if so, how are the organizations using the initiative? Has the use of HP2020 changed since the 2008 user study?
Do organizations monitor progress towards HP2020 objectives and targets?
What components of HP2020 are most useful to users?
What key issues should be considered in framing the next iteration of health promotion and disease prevention objectives for the nation?
What are the reasons that organizations are not using HP2020?
How has Healthy People webinar participation influenced organizations’ use of HP2020?
What are the organizational characteristics of users and non-users of HP2020, and has this changed since the 2005 and 2008 user studies?

HHS is seeking OMB approval to conduct a short survey using a self-administered questionnaire of state, local, and tribal organizations; Healthy People Consortium organizations; and Healthy People webinar attendees. The survey will be administered via a web-based platform. The sample size for all respondent groups is 1,102.

This collection of data is authorized by Section 301 of the U.S. Public Health Service Act (42 U.S.C.241). A copy of this legislation can be found in Appendix 1.

Purpose and Use of Information Collection

To better inform HP2010 and HP2020, NORC conducted several studies, including a 2008 Healthy People Users Study, 2005 Assessment of the Uses and Users of Healthy People 2010, and HealthierUS. There is currently limited information documenting the uses and users of the updated web-based version of HP2020. This study seeks to investigate characteristics of organizations that use and do not use HP2020 to generate information about improving the utility of the initiative and the web-based resources. Additionally, longitudinal comparisons can be made between the previous assessments and this study in terms of the organizational characteristics of Healthy People users. HHS will use the information gleaned from this study to make crucial planning decisions in light of its work on the next decade’s health objectives, as well as highlight effective strategies that can assist the community in working towards the nation’s disease prevention and health promotion goals.

HHS used the results of the 2005 Assessment and 2008 study to plan for HP2020 and to strengthen current outreach and assistance work with state, local, and tribal entities. For example, the 2008 study found that 58 percent of organizations believed examples of how others are using Healthy People would improve their ability to further implement the initiative. In the past several years, HHS has developed the “Who’s Leading the Leading Health Indicators?” series, which showcases states, communities, or organizations that are addressing the Leading Health Indicators in innovative ways. The study will assess if respondents use the Leading Health Indicators, and whether they view them as a valuable element of Healthy People. As the first assessment of the web-based version of HP2020, this iteration of the study will provide HHS with valuable information on the online platform. Results from the 2008 study also provided data for reporting on ODPHP’s annual performance and GPRA measures and established related trend data.

Having data on the use of HP2020 from a broad sample of public health entities allows HHS to take a more strategic approach to the design, dissemination, and implementation of HP2030. The data collected during this study will be useful to OMB and HHS in continuing to monitor ODPHP’s progress toward reaching its GPRA measure target. Finally, the results of the current study will be useful to policymakers as they clarify which aspects of the HP2020 are useful to constituent groups and perhaps identify areas for augmentation or policy development.

Use of Improved Information Technology and Burden Reduction

In addition to a formal mailed invitation letter, all respondents will receive an email inviting them to participate in the survey via a secure website. This letter will contain the survey URL as well as the respondent’s unique Personal Identification Number (PIN). The web survey will be programmed with skip logic and error checks thereby ensuring accurate response and reducing respondent burden. A postcard reminder will be sent to any non-respondents one week after the initial mailing, highlighting the convenience of the online completion option. A second web letter will be sent to all non-respondents approximately two weeks after the postcard, again emphasizing completion over the web. Any outstanding non-respondents at five weeks after the initial mailing will be contacted using computer-assisted telephone interviewing (CATI) to confirm receipt of the web invitation letters, and to inquire whether the respondent would like to complete the survey online or by telephone. If the respondent has lost or misplaced the letter or does not know his/her PIN, NORC can provide this information to the respondent via phone or an automated email. If the respondent opts to complete the survey by telephone, the interviewer will access the respondent’s case online and enter responses directly into the online survey.

Efforts to Identify Duplication and Use of Similar Information

NORC conducted a literature review, and the search did not identify any systematic evaluation of types of users and uses of HP2020 or HP2010 other than the 2008 Healthy People Users Study and 2005 Assessment of the Uses and Users of Healthy People 2010, conducted by NORC. Other literature indicates a commitment to the goals of HP2020 from a diverse set of organizations, but does not provide more than scattered descriptions of organizational efforts toward a HP2020 objective.

NORC’s 2005 User Study and 2008 Assessment of the Use and Users of Healthy People 2010 (OMB No. 0990-0276 and OMB No. 0990-0329, respectively) established the groundwork for the current study. The 2008 study found that 90 percent of respondent organizations were aware of HP2010, compared to 83 percent of responding organizations in the 2005 assessment. In addition, in 2008, 77 percent of the organizations aware of HP2010 reported using it in their agency, an increase over the 71 percent of organizations that reported using HP2010 in the 2005 assessment. These studies have provided HHS with valuable information about the uses and users of Healthy People and the results of these studies has informed the development of the current study to ensure HHS obtains important new information.

For example, in 2008, we sampled two distinct groups of state-level respondents, Healthy People State Coordinators and Directors of Chronic Disease Programs, to determine whether the knowledge of the Healthy People initiative is role-based or organization-based at the state level. The 2008 study found that 100 percent of Healthy People State Coordinators and 98 percent of State Chronic Disease Directors were aware of Healthy People. The current study will again look at Healthy People State Coordinators, but will target Senior Deputy Directors at the state health agencies instead of State Chronic Disease Directors. This approach will determine whether senior leadership at the state level health departments are aware of and use Healthy People, in addition to Healthy People State Coordinators, who would be expected to respond affirmatively to being aware of and using Healthy People.

In addition, the current study expands the respondent groups to include two new samples, Healthy People Consortium organizations and Healthy People webinar attendees. Healthy People Consortium members are a motivated group of agencies and organizations nationwide that are committed to achieving HP2020 goals and objectives. Webinar attendees include individuals who have demonstrated an interest in Healthy People and represent a diverse group of organizations and professions. The addition of these two new respondent groups will provide HHS with information about awareness and usage of HP2020 at organizations other than state, local, and tribal health departments.

Impact on Small Businesses or Other Small Entities

It is possible that some participating organizations will be small entities; however, the burden to complete this survey is low and therefore the impact will be minimal. This study will not unduly affect small businesses or small entities.

Consequences of Collecting the Information Less Frequently

The design of this study requires only one data collection activity per respondent. Without collecting this data, HHS will not have access to a comprehensive assessment of the level and types of involvement from the target audiences of HP2020. The federal government will find enormous benefit in having information available that will answer the questions about how, where, and for whom their public health initiative is being used. Additionally, without this data collection, HHS will not have an enumeration of the activities planned by these key target audiences to assess progress towards HP2020 goals and objectives at the end of the decade and input from these groups on activities related to HP2030. This iteration of the study will be the first to examine the current outreach activities and online tools available on healthypeople.gov, which will be valuable for planning future improvements of the platform. Finally, this data collection will allow the continuation of ODPHP’s GPRA measure based on the percentage of states currently using HP2020.

There are no legal obstacles to reduce the burden of collection.

Special Circumstances Relating to the Guidelines of 5 CRF 1320.5

This request complies with the information collection guidelines of 5 CFR 1320.5(d)(2).

Federal Register Notice and Efforts to Consult Outside the Agency

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on March 16, 2015. For Federal Register information, see the Office of the Secretary Certification Form. In addition, we have consulted with the Association of State and Territorial Health Officials (ASTHO) and the National Association of County and City Health Officials (NACCHO) related to survey development and sampling of local health organizations. A 60-day Federal Register Notice was published in the Federal Register on Monday, March 16, 2015,80-FR 13573; pp 13573 -13574. There were no public comments

NORC at the University of Chicago staff consulted include (full contact details for these individuals can be found in Section B.5 of this document):

Caitlin Oppenheimer, MPH

Steven Pedlow, MS

Catharine Fromknecht

Stephanie Poland, MA

Megan Heffernan, MPH

The NORC Institutional Review Board

NACCHO representatives consulted include:

Carolyn Leep, MS, MPH

Julia Joh Elligers, PhD, MPH

ASTHO representatives consulted include:

Katie Sellers, DrPH

Explanation of Any Payment or Gift to Respondent

There will be no payments or gifts to respondents.

Assurance of Confidentiality Provided to Respondents

Data will be treated in a confidential manner, unless otherwise compelled by law. Personal identification information (i.e., respondent names) will not be collected in the survey instrument and the unit of sampling is the organization, not the individual. Although the individual will be asked to report his/her organization name, this information will be used solely by NORC to categorize and summarize types of respondents for comparison purposes during the analysis phase of the project. Specific information linking organization name to particular survey responses will not be included in any information viewed by ODPHP, or any other HHS officials. Further, the study’s briefs and report will not identify any specific organizations. All potentially identifying information will be destroyed at the study’s conclusion.

Justification for Sensitive Questions

The surveys will not include any questions of a sensitive or personal nature. Respondents will be asked to answer from the perspective of their organization about particular aspects of the government programs, as well as the respondents’ opinions of different aspects of HP2020. The questions are designed to solicit information solely regarding uses of the initiative in a professional/worksite setting.

Estimates of Annualized Burden Hours and Cost

Burden Hours

In Exhibit 1, we provide estimates of the collection burden on participants from each of the seven samples for this effort. Study participants from each sample will participate in data collection one time only, responding via a web-based questionnaire. The data collection instrument is the same for the state, local, and tribal health organizations, as well as the Healthy People Consortium members. The survey instrument for Healthy People webinar attendees is slightly different given the variety of organizations that will be represented and the need to ask specific questions related to the webinars. Hour burden estimates were derived using a time estimation tool, and will be verified during the pilot/pretesting of the survey instrument.

Exhibit 1. Estimated Burden Hours

Type of Respondent	# of Respondents	No. Responses per Respondent	Average Burden Per Response (Hours)	Total Burden Hours
Healthy People State Coordinators (Frame A)	59	1	18/60	18^
*Senior Deputy Directors (Frame A)**	57	1	18/60	17^
Local Health Organizations (Frame B)	375	1	18/60	113^
Tribal Health Organizations (Frame C)	100	1	18/60	30
Tribal Area Health Boards (Frame D)	11	1	18/60	3^
Healthy People Consortium Organizations (Frame E)	250	1	18/60	75
Healthy People Webinar Attendees (Frame F)	250	1	18/60	75
TOTAL	1,102			331^

^Numbers have been rounded.

Burden Cost (Average Hourly Rate)

Exhibit 2. Estimated Burden Cost

Type of Respondent	Total Burden Hours	Average Hourly Wage Rate	Total Hour Cost
Healthy People State Coordinators¹ (Frame A)	18^	$33.74	$607^
Senior Deputy Directors² *(Frame A)**	17^	$57.11	$971^
Local Health Organizations¹ (Frame B)	113^	$33.74	$3,813^
Tribal Health Organizations¹ (Frame C)	30	$33.74	$1,012^
Tribal Area Health Boards¹ (Frame D)	3^	$33.74	$101^
Healthy People Consortium Organizations³(Frame E)	75	$34.50	$2,587.5
Healthy People Webinar Attendees³(Frame F)	75	$34.50	$2,587.5
TOTAL	331^		$11,679^

¹ Based on hourly wage for medical and health services managers in state government from the Department of Labor (DOL) National Compensation Survey (http://www.bls.gov/ncs/ocs/sp/nctb1349.pdf)

²Based on hourly wage for administrative services managers in state government from the Department of Labor (DOL) National Compensation Survey (http://www.bls.gov/ncs/ocs/sp/nctb1349.pdf)

³Based on hourly wage for social and community service managers in state government from the Department of Labor (DOL) National Compensation Survey (http://www.bls.gov/ncs/ocs/sp/nctb1349.pdf)

^Numbers have been rounded.

Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

Data collection for this study will not result in any additional capital, start-up, maintenance, or purchase costs to respondents or record keepers. Therefore, there is no burden to respondents other than that discussed in the previous section.

Annualized Cost to the Government

All costs for conducting the Healthy People User’s Study are included in the contract between the Department of Health and Human Services and NORC under contract number HHSP233201400366G. The total estimated cost is $400,833.96 over an eighteen-month period to conduct the surveys, analyze and present findings, and write a final report. This is an annualized cost of $267,222.64.

Explanation for Program Changes or Adjustments

This is a new collection of data.

Plans for Tabulation and Publication and Project Time Schedule

The data collected in this survey will be analyzed and interpreted to produce preliminary and final briefings as well as a final study report to HHS. NORC will deliver the final report to HHS in hardcopy and a print-ready electronic format. Publication of findings on the internet is at HHS’s discretion. The remainder of this section discusses data sources and the analytic techniques that will be employed. Information will be collected over a three- to four-month period following OMB approval. Exhibit 3 provides a schedule of data collection, analysis, and reporting following OMB approval.

Exhibit 3. Timetable for Data Collection, Analysis, and Publication

Activity	Expected Date of Completion
Survey sent to respondents and data collected	1-4 months following OMB approval
Data analysis	4-5 months following OMB approval
Preliminary briefing and preparation of draft report	6-7 months following OMB approval
Final report	7-8 months following OMB approval
Final briefing	9 months following OMB approval

Data Sources

This assessment includes one self-administered web survey, which will be sent to members of state, local, and tribal health organizations, in addition to Healthy People webinar attendees and Consortium members. Each individual will be asked to complete the one-time survey, expected to take no longer than 18 minutes. Results will be summarized within and across organization type.

The survey is designed to ascertain how state, local, and tribal health organizations, webinar attendees and Consortium organizations use HP2020. The survey also seeks to understand how organizations perceive the utility of HP2020. The questionnaire consists of four main sections, which are outlined below. The webinar attendees will be asked slightly different questions than the other sample groups, due to some inherent differences in their roles at their organization. Specifically, we predict webinar attendees may use HP2020 in their work, but may not be able to speak for their organization’s use. The questions will attempt to identify who the webinar attendees are and what aspects of HP2020 they utilize.

Uses of Healthy People 2020. Captures data about whether the organization uses HP2020, how they use the initiative, awareness and use of online tools and outreach activities, the impact of HP2020, how the organization monitors progress, value of the Leading Health Indicators, and factors that enable or hinder its use within the organization.
Looking Forward to 2030. Captures data about the next iteration of Healthy People and ways to improve the initiative for 2030.
Non-users of Healthy People 2020. Captures data from respondents that report their organization does not use HP2020 on why they do not use the initiative, barriers to use, and ascertains general perceptions about the initiative.
Demographics. Captures data about organizational characteristics such as type, size, and health priorities of organization, as well as the job title of the respondent.

The survey instrument for state, local, tribal and Consortium organizations is included as Attachment 1. The survey instrument for webinar attendees is included as Attachment 2.

Tabulations and Statistical Analysis

This section details the tabulations and statistical analyses that will be conducted for this study. This study will use both univariate and, where possible, multivariate techniques to analyze the data.

Data analysis will focus on identifying results of the established key research questions. In addition to answering this core set of questions, the analysis will also compare the groups and determine the extent to which certain characteristics of the organization seem to be related to the extent of awareness, the extent of use, the nature of use, and the kinds of barriers experienced. Exhibit 4 lists the key research questions and sub-questions.

Exhibit 4: Key Research Questions

Awareness: Are organizations aware of HP2020, and if so, how are the organizations using the initiative? Has the use of HP2020 changed since the 2008 user study?

Is the organization aware of HP2020?
Has the organization incorporated HP2020 into its planning of health activities? If so, how did it do this?
How does the organization use HP2020?
What is the impact of HP2020 on the work of the organization?
Are organizations aware of the Leading Health Indicators? Are the Leading Health Indicators a valuable element of HP2020?

Monitoring Progress: How do organizations monitor progress towards HP2020 objectives and targets?

Does the organization assess progress towards HP2020 goals?
What data sources does the organization use to assess progress towards the HP2020 objectives?

3. Useful Elements: What components of HP2020 are most useful to users?

Does the organization use HP2020 as a source of data for benchmarking or evaluation?
Which HP2020 tools and activities are users aware of? Which do they use?
Are there additional resources that would be useful to organizations?

4. Looking Forward: What key issues should be considered in framing the next iteration of health promotion and disease prevention objectives for the nation?

How can HHS improve the next iteration of national health objectives to be more useful to organizations?
Should the scope of issues covered in the next iteration of Healthy People be narrower or broader than HP2020?
Would a reorganization (e.g., by health risks/ determinants, by disease areas, by life stages) of objectives be helpful to organizations?
How involved should organizations be in framing the next iteration of Healthy People?

Non-users: What are the reasons that organizations are not using HP2020?

What barriers to using HP2020 exist at the organization?
What aspects of the HP2020 pose obstacles or challenges to using it at the organization?
What changes to this initiative would increase its usefulness?

WEBINAR PARTICIPATION: How has webinar participation influenced the organizations use of HP2020?

Were webinar attendees aware of HP2020 before attending the webinar?
Has participants’ usage of HP2020 changed since attending webinar(s)?
Does usage of HP2020 differ based on the type of webinar (LHI, Progress Reviews, Spotlights) the participant attended?

Demographics: What are the organizational characteristics of users and non-users of HP2020, and has this changed since the 2005 and 2008 user studies?

What is the type, size, and location of the organization?

Both descriptive and inferential statistics, such as the standard t-test, chi-square test, and multiple comparison procedures will be utilized in the analysis. Standard errors will also be provided for these estimates. Non-parametric statistical techniques may also be used to analyze the data, including the chi-square test for cross tabulations, the Wilcoxon rank-sum (Mann-Whitney) two-sample test, and the Komolgorov-Smirnov test for equality of distributions. Nonsampling errors arising from unit and item nonresponse will be dealt with through weighting and imputation where appropriate.

The remainder of this section presents specific analyses that will be conducted to answer the research questions for each initiative.

1) Are organizations aware of HP2020, and if so, how are the organizations using the initiative? Has the use of HP2020 changed since the 2008 user study?

Ascertaining the awareness level of the initiatives is a main goal of the assessment. The main statistical technique used in analyses will examine the proportion of respondents that indicate awareness of the initiatives. Simple univariate statistics will examine the data overall, and chi-square tests of association or student’s t-tests will be used to compare data among and between respondent groups.

By comparing responses between different kinds of organizations (using data obtained from the demographics section of the survey), it will be possible to identify characteristics of organizations that require additional outreach. Univariate statistics will be used to assess awareness of HP2020 in each of the seven respondent groups.

There are many ways that organizations could use HP2020, and HHS has anecdotal evidence from many organizations as well as the results of the 2008 User Study. This survey will provide an opportunity to further document utilization of HP2020 in a uniform manner. Several questions on the survey relate to gaining information about how organizations utilize the program. Initial questions will establish whether the organization uses HP2020, and subsequent questions seek to catalog how it is being used. Wherever possible, answer options have been narrowed as possible responses in order to minimize the burden on the respondent. Additional questions that will be assessed relate to how users interact with the program (e.g., through the publications or website), the frequency with which HP2020 is used as a resource, and how organizations use the initiative to measure health outcomes.

Descriptive statistics will be used to identify how the program is being used, and chi-square tests will be used to determine if HP2020 is used similarly across respondent categories and organizational characteristics (i.e., comparisons across the seven respondent groups may be made as well as different organizational sizes within a respondent category). We will also use logistic regression, where appropriate, to determine if organizational characteristics are associated with the likelihood of using the initiative in specific ways.

We will also make comparisons between the results of this User Study, the 2008 User Study, and the 2005 Assessment. Most of these analyses will exclude the new groups of state Senior Deputy Directors, Consortium members, and webinar attendees. Descriptive statistics will be used to compare how the program usage has changed, and chi-square tests and t-tests will be used to determine if these differences are statistically significant. If appropriate, logistic regression and other advanced statistical tools will be used to better understand the changes between the two data sets.

2) How do organizations monitor progress towards HP2020 objectives and targets?

HHS is interested in determining whether organizations are measuring progress toward HP2020 targets. Descriptive statistics will be used to identify how often progress is being measured, and chi-square tests will be used to determine if some respondent categories and organizational characteristics result in different rates of measurement.

3) What components of HP2020 are most useful to users? Specifically, what tools and activities are most widely used?

HP2020 has several tools and activities, including, but not limited to: Data, implementation stories, tools for program planning, Healthy People webinars, and Healthy People communication. HHS is interested in learning which activities users are aware of, and which they have used. Simple descriptive statistics will be used to identify which tools and activities users are aware of and using.

Analyses will also assess the impact of HP2020 on the work of the organization. The question asks respondents to rate, on a scale of 1-5, the impact of HP2020 on its organization. Mean scores will be computed and compared among different organizational characteristics using the student’s t-test, which assumes normally distributed data. An alternative non-parametric test (with no accompanying normality assumption) that will be used is the Wilcoxon signed rank test. Analyses will be conducted to examine possible correlation between overall opinion of the program and utilization of the program.

4) What key issues should be considered in framing the next iteration of health promotion and disease prevention objectives for the nation?

HHS is particularly interested in gaining information from on-the-ground users as to how Healthy People can be improved for the next iteration of Healthy People, HP2030. By increasing the usefulness and utility of the next Healthy People iteration to the state, local, and tribal entities, HHS can increase the usage of the program to improve the health of the nation. Descriptive statistics will be used to summarize the feedback as much as possible.

5) What are the reasons that organizations are not using HP2020?

Gaining insight into how HHS can reduce barriers to utilization of HP2020 and encourage greater participation and action toward HP2020 targets is a key objective for this project. Each of the seven respondent groups is a key target user of the initiative. For organizations that indicate they do not use HP2020, this project provides the opportunity to understand why organizations do not utilize the program in anticipated ways. Descriptive statistics will be used to explore possible program and organizational causes for non-use of the initiative. Among users of HP2020, descriptive statistics will assess reasons that prevent them from expanding their use of the initiative.

The limited number of open-ended questions that seek suggestions for improving the program will be examined and categorized where possible.

6) How has Healthy People webinar participation influenced organizations’ use of HP2020?

For the first time, webinar attendees will be asked about Healthy People. HHS is interested in whether they were aware of Healthy People before attending, and how HP2020 has been used since their attendance. Descriptive statistics, including t-tests and chi-square tests, will be used to compare participant sub-groups and webinar types in the answers given. Non-parametric tests will be considered as well.

7) What are the organizational characteristics of users and non-users of HP2020, and has this changed since the 2005 and 2008 User Studies?

To determine the characteristics associated with users and non-users of HP2020, chi-square tests of association between organizational characteristics and use of the initiatives will be conducted.

Reason(s) Display of OMB Expiration Date is Inappropriate

HHS does not seek this exemption. All data collection materials will display the OMB expiration details.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement.

Collection of Information Employing Statistical Methods

Surveys are being administered to collect information about how seven groups of target users use and perceive HP2020. These groups include Healthy People State Coordinators, State Deputy Directors, local health organizations, tribal health organizations, tribal Area Health Boards, Healthy People webinar attendees, and Healthy People Consortium members. The surveys will also collect information from non-users of HP2020 to determine the factors that prevent target groups from using the initiative in their organizations.

The results will be generalizable to the respondent universe, which consists of government entities that interact with HHS and their constituents to improve the health of the populations they serve.

Respondent Universe and Sampling Methods

The sample will include 1,102 organizations from the 50 states, the District of Columbia, U.S. territories, and Native American tribes. The unit of analysis for the sample will be the organization, meaning that no more than one survey will be sent to each organization, although this sample treats Healthy People State Coordinators as separate organizations from state Senior Deputy Directors and the tribal Area Health Boards as separate organizations from the individual tribal health organizations. The project will census state health departments (Healthy People State Coordinators and Senior Deputy Directors separately) and the tribal Area Health Boards, and sample local and tribal health organizations. Separate samples of organizations from the Healthy People Consortium as well as Healthy People webinar attendees will also be carried out. These samples of public health officials will be able to provide the type of data necessary to evaluate HP2020.

The sample frames will be constructed from multiple sources and will result in seven separate lists: Healthy People State Coordinators, State Deputy Directors, local health organizations, tribal health organizations, tribal Area Health Boards, Healthy People webinar attendees, and Healthy People Consortium members. A list of the 59 Healthy People State Coordinators will serve as the primary contacts for the states (sample frame A). The list of state Senior Deputy Directors will serve as the second set of state-level contacts (sample frame A*). The list of approximately 3,000 members of the National Association of County and City Health Officials (NACCHO) will serve as the sample frame for the local officials (sample frame B). The list of 566 federally recognized Tribes will serve as the sample list for tribal health organizations (sample frame C). The list of 11 tribal Area Health Boards will serve as the second set of tribal contacts (sample frame D). The Area Health Boards align with the 12 physical areas of the United States designated by the Indian Health Service (IHS).¹ One IHS physical area does not have an Area Health Board contact listed. The complete list of 2,400 Healthy People Consortium members will serve as the set of Consortium organization contacts (sample frame E). Webinar attendees from all 2014 Healthy People webinars, including Leading Health Indicator, Spotlight on Health, and Progress Review webinars, will be included in the sample list for webinar attendees (sample frame F).

These frames will be used to draw samples that satisfy the study’s goals. The proposed sample design satisfies two key requirements. First, all organizations from frames A and A* and all tribal Area Health Boards from frame D will be included with certainty. Second, the design will draw samples that produce nationally representative estimates for urban and rural organizations in group B, and nationally representative estimates by tribal size and region in group C, if possible. For the Healthy People Consortium and webinar attendee samples, we aim to be as representative of these groups as possible.

Our total sample of 1,102 organizations will consist of all 116 organizations from frames A (Healthy People State Coordinators) and A* (State Senior Deputy Directors), all of the 11 organizations in frame D (Tribal Area Health Boards) plus 975 more sampled from frames B (Local Health Organizations), C (Tribal Health Organizations), E (Healthy People Consortium Organizations), and F (Healthy People Webinar Attendees). We will include in the sample 375 organizations from frame B, 100 organizations from frame C, and 250 organizations from both frames E and F. If the same organization is selected in both the webinar and Consortium samples, the Consortium sample will take priority. We would reselect a new respondent to replace the organization in the Webinar sample.

Exhibit 5 shows the sizes of the frames provided, as well as the sample sizes and expected response rates and respondent sizes.

Exhibit 5: Sample Statistics

	Population	Sample	Expected Response Rate	Total Expected Respondent (n)
Healthy People State Coordinators (Frame A)	59	59	85%	50^
*State Senior Deputy Directors (Frame A)**	57	57	75%	43^
Local Health Organizations (Frame B)	~3,000	375	70%	262^
Tribal Health Organizations (Frame C)	566	100	75%	75
Tribal Area Health Boards (Frame D)	11	11	75%	8^
Healthy People Consortium Organizations (Frame E)	~2,400	250	40%	100
Healthy People Webinar Attendees (Frame F)	10,103^^	250	40%	100
TOTAL	~16,196	1,102	58.5%	638^

^Numbers have been rounded.

^^8261 unique attendees.

Selection Methods

For sample frame B and C we will use systematic samples with equal probability of selection (within frame) and implicit stratification. The only difference is the variables that will be used for implicit stratification. Implicit stratification involves sorting the frame on certain variables so that the sample drawn is representative on that variable. For example, assume that 44 percent of local health organizations are in a rural setting and 56 percent are in urban settings. By sorting on urban-rural status and then drawing a systematic sample, the resulting sample will be very close to including 44 percent of organizations in a rural setting.

We will sort on multiple variables so that samples will be representative on more than one dimension. The variability in sample size percentages will increase for variables that appear later in the sorting. Serpentine sorting will be used when sorting on multiple variables to maximize the effect of the stratification. Serpentine sorting involves sorting by an order that is alternately increasing or decreasing. For example, serpentine sorting on urban/rural status and region could result in this sort order: Rural Northeast, Rural Midwest, Rural South, Rural West, Urban West, Urban South, Urban Midwest, and Urban Northeast. This sort order successfully keeps the two West strata together.

It should be noted that the level of precision for subgroup estimates may not be sufficient to make meaningful comparisons between frames. To account for this imprecision, we will employ strategic collapsing of strata in estimation to create estimates with a higher level of precision. For example, the urban groups and rural groups may be collapsed to form nationally representative estimates of urban and rural areas.

Local Health Organizations

The NACCHO list frame consists of approximately 3,000 records. However, we will remove any “inappropriate” records (e.g., tribal records) so that our sampling frame contains only local health organizations. Inappropriate records to be deleted include duplicate records, records without title or agency name, as well as other inappropriate records such as public health consultants, foundations, special interest groups (for hand gun violence, for example), students, professors, etc.

Since it is desired to have a representative sample with respect to urban and rural organizations, we will sort the file first on urban/rural status. Using the zip code from the file, we will map each organization to the state and county in which it resides. We will then determine if this county is inside a Census defined Metropolitan Statistical Area (MSA) or not. The Census Bureau defines MSAs as the counties that involve economic activity related to a central city. If the county is in an MSA, we will count this organization as “urban.” Otherwise, we will classify the organization as “rural.” Suburban organizations will be classified as “urban.”

Tribal Organizations

The target respondent is the lead tribal health representative, meaning the person within the Tribe who has the authority and responsibility for disease prevention and health promotion activities. We will select a sample of 100 tribal entities from the 566 federally recognized Tribes. We will contact the Tribal Leader, asking for contact information for the appropriate respondent who can speak to the health promotion activities of the Tribe. If data related to the approximate size of the Indian population are available, we will divide the tribes into small (< 2,500 Indian population), medium (2,500 – 10,000), and large (> 10,000). The tribes with unknown population size will be placed into a fourth category.

To ensure a representative mix of small, medium, and large tribes, we will sort the file on size, if possible. Additionally, to the extent that tribal entities are geographically diverse, we will draw the sample to be as representative as possible by sorting on the Bureau of Indian Affairs regions. If we cannot obtain the tribal size data, we will cross reference the current list with data from the 2008 sample frame to stratify the data as best as possible.

Healthy People Consortium

NORC will select a sample of 250 organizations from the 2,400 members of the Healthy People Consortium so that the information collected is sufficient to learn about this sub-population. We will conduct advance locating to identify individuals at Consortium organizations that would be most appropriate to complete the survey. If we are unable to determine an appropriate contact person, the organization will not be included in the sample population. Consortium organizations will be classified by organization type (i.e. academic institution, businesses, professional association) and sorted by these categories to provide a representative sample of Consortium organizations. With an expected 40 percent response rate, we will complete 100 surveys.

Healthy People Webinar Attendees

NORC will sample 250 webinar attendees, with an expected 40 percent response rate, resulting in 100 completed surveys. Our sample population will include the universe of webinar attendees from 2014, including Leading Health Indicator webinars, Spotlight on Health webinars, and Progress Review webinars. The number of attendees selected from each webinar will be proportional to the size of the webinar. Participants will have a probability of being selected proportional to the number of webinars they attended. Before sample selection, we will sort the sample list by webinar, organization type, and company to limit the number of individuals selected from the same organization. If two participants are selected from the same organization (or department of a larger organization), then we will replace one of the repeats. Additionally, we will remove any individuals from the sample list who participated on the webinar for less than 20 minutes.

Procedures for the Collection of Information

The sample will include 1,102 organizations from state, local, and tribal organizations. The unit of analysis for the survey will be the organization, so that no organization will be asked to complete more than one survey, although this sample treats Healthy People State Coordinators as separate organizations from state Senior Deputy Directors. Fielding of the survey will entail mailing survey invitation letters to the key staff member at each organization. The letter will contain background information on the survey, the link to the survey URL, and the respondent’s unique Personal Identification Number, or PIN, which is used to access the survey. A postcard mailing will be sent to respondents one week after the initial mailing, and a second invitation letter will be mailed to the non-respondents. Approximately five weeks after the initial web invitation mailing, a phone call will be made to those who have not responded. The phone call will also provide an opportunity for the researchers to provide web access information to the respondent if she/he has lost or misplaced this information, or to access the respondent’s case online and enter his or her responses over the telephone.

Because the key staff member for the Healthy People Consortium or tribal organization samples may be unknown, NORC will conduct advance locating prior to sending the initial survey invitation letters. For the tribal organizations, in addition to a phone call, we will mail an advance letter to Tribal Leaders (Attachment 9). This advance letter would introduce the survey, explain its purpose, and request the name of someone who could complete the survey. NORC will employ a similar process for the Healthy People Consortium sample, focusing on advance locating phone calls, in order to identify the best point of contact. This process will ensure that any communication related to this survey is addressed to the appropriate person within the organization.

Project investigators will use an electronic receipt control system using case ID numbers to track the individual mailings and to record address updates. This system can also track those that indicate a refusal to participate via a returned letter.

All data will be collected via a Computer-Assisted Web Interview (CAWI) or Computer-Assisted Telephone Interview (CATI) instrument. Both instruments will be programmed with skip logic and any error checks, as appropriate, to reduce respondent error. The primary emphasis will be on web completion; however, the respondent will have the option to start in one mode and complete in another.

Methods to Maximize Response Rates and Deal with Nonresponse

The investigators will use a number of proven methods to maximize participation in the study. First, the instrument itself is designed to maximize response rates. The style of the survey is inviting and user friendly, with a maximum of 25 questions for the state, local, tribal, and Consortium organizations, and a maximum of 28 questions for webinar attendees. The instructions for the survey are straightforward, and the skip patterns will be programmed into the online survey. During the OMB review period, the questionnaire will be pilot tested with nine respondents from the sampling frame, and questions will be amended to reflect suggested improvements from these respondents. In addition to the web invitation letter, each respondent will receive a cover letter encouraging participation in the survey. The cover letters (Attachments 3, 4, 5, 6, 7 and 8) will convey the importance of the survey to the ODPHP and HHS. The cover letters will also indicate that the respondent will not be identified to any government agency. A postcard reminder will be sent to any non-respondents one week after the initial mailing, highlighting the convenience of the online completion option. A second web invitation letter will be sent to remaining non-responders two weeks after the pre-mailing. This mailing will again highlight the importance of the study and the convenience of responding online. Any outstanding non-respondents at five weeks after the initial mailing will be contacted using computer-assisted telephone interviewing to confirm receipt of the invitation letters, and to inquire whether the respondent would like to complete the survey online or by telephone. If the respondent has lost or misplaced the letter or his/her PIN, NORC can provide this information to the respondent over the phone or via an automated email. If the respondent opts to complete the survey by telephone, the interviewer will access the respondent’s case online and enter responses directly into the online survey.

In the 2008 Healthy People User Study, similar procedures were used with the same respondent population with good success. The response rate for the Healthy People State Coordinators group was 85 percent. The local sample had a response of 71 percent. The tribal sample had a response of 50 percent for the tribal health organizations, and 75 percent for the tribal Area Health Boards. Overall, the response to the survey was 70 percent. Given a greater familiarity with web-based surveys, we believe high response rates will be achieved.

Tests of Procedures of Methods to be Undertaken

A pilot test of the survey was conducted during the OMB comment period with consortium and webinar participants. The following changes were made in response to suggestions from the pilot respondents:

For Question 4 (users in main survey); Question 10 (non-users in main survey); Question 5 (organizational users in webinar survey); Question 7 (non-organizational users in webinar survey); Question 12 (non-users in webinar survey)
- Add “Not Applicable” answer choice
- Change “For collaboration/outreach” category to “For collaboration/outreach or education”
- Move answer choice “To support applications for grants or other funding” from “For setting internal priorities” category to “Other uses”
In Question 15 (users in main survey); Question 17 (organizational users in webinar survey); Question 12 (non-organizational users in webinar survey)
- Add additional clarifying text for “data” answer choice.
In Question 16 (users in main survey); Question 18 (organizational users in webinar survey); Question 13 (non-organizational users in webinar survey)
- Add “Examples of evaluation instruments or tools/templates from other organizations” as an answer choice

Individuals Consulted of Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following individuals contributed to the questionnaire and study design and will be involved in the interpretation and analysis of findings:

Caitlin Oppenheimer, MPH

Vice President, Public Health Department