Surveys are being administered to collect information about how seven groups of target users use and perceive HP2020. These groups include Healthy People State Coordinators, State Deputy Directors, local health organizations, tribal health organizations, tribal Area Health Boards, Healthy People webinar attendees, and Healthy People Consortium members. The surveys will also collect information from non-users of HP2020 to determine the factors that prevent target groups from using the initiative in their organizations.
The results will be generalizable to the respondent universe, which consists of government entities that interact with HHS and their constituents to improve the health of the populations they serve.
The sample will include 1,102 organizations from the 50 states, the District of Columbia, U.S. territories, and Native American tribes. The unit of analysis for the sample will be the organization, meaning that no more than one survey will be sent to each organization, although this sample treats Healthy People State Coordinators as separate organizations from state Senior Deputy Directors and the tribal Area Health Boards as separate organizations from the individual tribal health organizations. The project will census state health departments (Healthy People State Coordinators and Senior Deputy Directors separately) and the tribal Area Health Boards, and sample local and tribal health organizations. Separate samples of organizations from the Healthy People Consortium as well as Healthy People webinar attendees will also be carried out. These samples of public health officials will be able to provide the type of data necessary to evaluate HP2020.
The sample frames will be constructed from multiple sources and will result in seven separate lists: Healthy People State Coordinators, State Deputy Directors, local health organizations, tribal health organizations, tribal Area Health Boards, Healthy People webinar attendees, and Healthy People Consortium members. A list of the 59 Healthy People State Coordinators will serve as the primary contacts for the states (sample frame A). The list of state Senior Deputy Directors will serve as the second set of state-level contacts (sample frame A*). The list of approximately 3,000 members of the National Association of County and City Health Officials (NACCHO) will serve as the sample frame for the local officials (sample frame B). The list of 566 federally recognized Tribes will serve as the sample list for tribal health organizations (sample frame C). The list of 11 tribal Area Health Boards will serve as the second set of tribal contacts (sample frame D). The Area Health Boards align with the 12 physical areas of the United States designated by the Indian Health Service (IHS).1 One IHS physical area does not have an Area Health Board contact listed. The complete list of 2,400 Healthy People Consortium members will serve as the set of Consortium organization contacts (sample frame E). Webinar attendees from all 2014 Healthy People webinars, including Leading Health Indicator, Spotlight on Health, and Progress Review webinars, will be included in the sample list for webinar attendees (sample frame F).
These frames will be used to draw samples that satisfy the study’s goals. The proposed sample design satisfies two key requirements. First, all organizations from frames A and A* and all tribal Area Health Boards from frame D will be included with certainty. Second, the design will draw samples that produce nationally representative estimates for urban and rural organizations in group B, and nationally representative estimates by tribal size and region in group C, if possible. For the Healthy People Consortium and webinar attendee samples, we aim to be as representative of these groups as possible.
Our total sample of 1,102 organizations will consist of all 116 organizations from frames A (Healthy People State Coordinators) and A* (State Senior Deputy Directors), all of the 11 organizations in frame D (Tribal Area Health Boards) plus 975 more sampled from frames B (Local Health Organizations), C (Tribal Health Organizations), E (Healthy People Consortium Organizations), and F (Healthy People Webinar Attendees). We will include in the sample 375 organizations from frame B, 100 organizations from frame C, and 250 organizations from both frames E and F. If the same organization is selected in both the webinar and Consortium samples, the Consortium sample will take priority. We would reselect a new respondent to replace the organization in the Webinar sample.
Exhibit 5 shows the sizes of the frames provided, as well as the sample sizes and expected response rates and respondent sizes.
Exhibit 5: Sample Statistics
|
Population |
Sample |
Expected Response Rate |
Total Expected Respondent (n) |
Healthy
People
State Coordinators |
59 |
59 |
85% |
50^ |
State Senior Deputy Directors (Frame A*) |
57 |
57 |
75% |
43^ |
Local Health Organizations (Frame B) |
~3,000 |
375 |
70% |
262^ |
Tribal Health Organizations (Frame C) |
566 |
100 |
75% |
75 |
Tribal Area Health Boards (Frame D) |
11 |
11 |
75% |
8^ |
Healthy People Consortium Organizations (Frame E) |
~2,400 |
250 |
40% |
100 |
Healthy People Webinar Attendees (Frame F) |
10,103^^ |
250 |
40% |
100 |
TOTAL |
~16,196 |
1,102 |
58.5% |
638^ |
^Numbers have been rounded.
^^8261 unique attendees.
Selection Methods
For sample frame B and C we will use systematic samples with equal probability of selection (within frame) and implicit stratification. The only difference is the variables that will be used for implicit stratification. Implicit stratification involves sorting the frame on certain variables so that the sample drawn is representative on that variable. For example, assume that 44 percent of local health organizations are in a rural setting and 56 percent are in urban settings. By sorting on urban-rural status and then drawing a systematic sample, the resulting sample will be very close to including 44 percent of organizations in a rural setting.
We will sort on multiple variables so that samples will be representative on more than one dimension. The variability in sample size percentages will increase for variables that appear later in the sorting. Serpentine sorting will be used when sorting on multiple variables to maximize the effect of the stratification. Serpentine sorting involves sorting by an order that is alternately increasing or decreasing. For example, serpentine sorting on urban/rural status and region could result in this sort order: Rural Northeast, Rural Midwest, Rural South, Rural West, Urban West, Urban South, Urban Midwest, and Urban Northeast. This sort order successfully keeps the two West strata together.
It should be noted that the level of precision for subgroup estimates may not be sufficient to make meaningful comparisons between frames. To account for this imprecision, we will employ strategic collapsing of strata in estimation to create estimates with a higher level of precision. For example, the urban groups and rural groups may be collapsed to form nationally representative estimates of urban and rural areas.
Local Health Organizations
The NACCHO list frame consists of approximately 3,000 records. However, we will remove any “inappropriate” records (e.g., tribal records) so that our sampling frame contains only local health organizations. Inappropriate records to be deleted include duplicate records, records without title or agency name, as well as other inappropriate records such as public health consultants, foundations, special interest groups (for hand gun violence, for example), students, professors, etc.
Since it is desired to have a representative sample with respect to urban and rural organizations, we will sort the file first on urban/rural status. Using the zip code from the file, we will map each organization to the state and county in which it resides. We will then determine if this county is inside a Census defined Metropolitan Statistical Area (MSA) or not. The Census Bureau defines MSAs as the counties that involve economic activity related to a central city. If the county is in an MSA, we will count this organization as “urban.” Otherwise, we will classify the organization as “rural.” Suburban organizations will be classified as “urban.”
Tribal Organizations
The target respondent is the lead tribal health representative, meaning the person within the Tribe who has the authority and responsibility for disease prevention and health promotion activities. We will select a sample of 100 tribal entities from the 566 federally recognized Tribes. We will contact the Tribal Leader, asking for contact information for the appropriate respondent who can speak to the health promotion activities of the Tribe. If data related to the approximate size of the Indian population are available, we will divide the tribes into small (< 2,500 Indian population), medium (2,500 – 10,000), and large (> 10,000). The tribes with unknown population size will be placed into a fourth category.
To ensure a representative mix of small, medium, and large tribes, we will sort the file on size, if possible. Additionally, to the extent that tribal entities are geographically diverse, we will draw the sample to be as representative as possible by sorting on the Bureau of Indian Affairs regions. If we cannot obtain the tribal size data, we will cross reference the current list with data from the 2008 sample frame to stratify the data as best as possible.
Healthy People Consortium
NORC will select a sample of 250 organizations from the 2,400 members of the Healthy People Consortium so that the information collected is sufficient to learn about this sub-population. We will conduct advance locating to identify individuals at Consortium organizations that would be most appropriate to complete the survey. If we are unable to determine an appropriate contact person, the organization will not be included in the sample population. Consortium organizations will be classified by organization type (i.e. academic institution, businesses, professional association) and sorted by these categories to provide a representative sample of Consortium organizations. With an expected 40 percent response rate, we will complete 100 surveys.
Healthy People Webinar Attendees
NORC will sample 250 webinar attendees, with an expected 40 percent response rate, resulting in 100 completed surveys. Our sample population will include the universe of webinar attendees from 2014, including Leading Health Indicator webinars, Spotlight on Health webinars, and Progress Review webinars. The number of attendees selected from each webinar will be proportional to the size of the webinar. Participants will have a probability of being selected proportional to the number of webinars they attended. Before sample selection, we will sort the sample list by webinar, organization type, and company to limit the number of individuals selected from the same organization. If two participants are selected from the same organization (or department of a larger organization), then we will replace one of the repeats. Additionally, we will remove any individuals from the sample list who participated on the webinar for less than 20 minutes.
The sample will include 1,102 organizations from state, local, and tribal organizations. The unit of analysis for the survey will be the organization, so that no organization will be asked to complete more than one survey, although this sample treats Healthy People State Coordinators as separate organizations from state Senior Deputy Directors. Fielding of the survey will entail mailing survey invitation letters to the key staff member at each organization. The letter will contain background information on the survey, the link to the survey URL, and the respondent’s unique Personal Identification Number, or PIN, which is used to access the survey. A postcard mailing will be sent to respondents one week after the initial mailing, and a second invitation letter will be mailed to the non-respondents. Approximately five weeks after the initial web invitation mailing, a phone call will be made to those who have not responded. The phone call will also provide an opportunity for the researchers to provide web access information to the respondent if she/he has lost or misplaced this information, or to access the respondent’s case online and enter his or her responses over the telephone.
Because the key staff member for the Healthy People Consortium or tribal organization samples may be unknown, NORC will conduct advance locating prior to sending the initial survey invitation letters. For the tribal organizations, in addition to a phone call, we will mail an advance letter to Tribal Leaders (Attachment 9). This advance letter would introduce the survey, explain its purpose, and request the name of someone who could complete the survey. NORC will employ a similar process for the Healthy People Consortium sample, focusing on advance locating phone calls, in order to identify the best point of contact. This process will ensure that any communication related to this survey is addressed to the appropriate person within the organization.
Project investigators will use an electronic receipt control system using case ID numbers to track the individual mailings and to record address updates. This system can also track those that indicate a refusal to participate via a returned letter.
All data will be collected via a Computer-Assisted Web Interview (CAWI) or Computer-Assisted Telephone Interview (CATI) instrument. Both instruments will be programmed with skip logic and any error checks, as appropriate, to reduce respondent error. The primary emphasis will be on web completion; however, the respondent will have the option to start in one mode and complete in another.
The investigators will use a number of proven methods to maximize participation in the study. First, the instrument itself is designed to maximize response rates. The style of the survey is inviting and user friendly, with a maximum of 25 questions for the state, local, tribal, and Consortium organizations, and a maximum of 28 questions for webinar attendees. The instructions for the survey are straightforward, and the skip patterns will be programmed into the online survey. During the OMB review period, the questionnaire will be pilot tested with nine respondents from the sampling frame, and questions will be amended to reflect suggested improvements from these respondents. In addition to the web invitation letter, each respondent will receive a cover letter encouraging participation in the survey. The cover letters (Attachments 3, 4, 5, 6, 7 and 8) will convey the importance of the survey to the ODPHP and HHS. The cover letters will also indicate that the respondent will not be identified to any government agency. A postcard reminder will be sent to any non-respondents one week after the initial mailing, highlighting the convenience of the online completion option. A second web invitation letter will be sent to remaining non-responders two weeks after the pre-mailing. This mailing will again highlight the importance of the study and the convenience of responding online. Any outstanding non-respondents at five weeks after the initial mailing will be contacted using computer-assisted telephone interviewing to confirm receipt of the invitation letters, and to inquire whether the respondent would like to complete the survey online or by telephone. If the respondent has lost or misplaced the letter or his/her PIN, NORC can provide this information to the respondent over the phone or via an automated email. If the respondent opts to complete the survey by telephone, the interviewer will access the respondent’s case online and enter responses directly into the online survey.
In the 2008 Healthy People User Study, similar procedures were used with the same respondent population with good success. The response rate for the Healthy People State Coordinators group was 85 percent. The local sample had a response of 71 percent. The tribal sample had a response of 50 percent for the tribal health organizations, and 75 percent for the tribal Area Health Boards. Overall, the response to the survey was 70 percent. Given a greater familiarity with web-based surveys, we believe high response rates will be achieved.
A pilot test of the survey was conducted during the OMB comment period with consortium and webinar participants. The following changes were made in response to suggestions from the pilot respondents:
For Question 4 (users in main survey); Question 10 (non-users in main survey); Question 5 (organizational users in webinar survey); Question 7 (non-organizational users in webinar survey); Question 12 (non-users in webinar survey)
Add “Not Applicable” answer choice
Change “For collaboration/outreach” category to “For collaboration/outreach or education”
Move answer choice “To support applications for grants or other funding” from “For setting internal priorities” category to “Other uses”
In Question 15 (users in main survey); Question 17 (organizational users in webinar survey); Question 12 (non-organizational users in webinar survey)
Add additional clarifying text for “data” answer choice.
In Question 16 (users in main survey); Question 18 (organizational users in webinar survey); Question 13 (non-organizational users in webinar survey)
Add “Examples of evaluation instruments or tools/templates from other organizations” as an answer choice
The following individuals contributed to the questionnaire and study design and will be involved in the interpretation and analysis of findings:
Caitlin Oppenheimer, MPH
Vice President, Public Health Department
NORC at the University of Chicago
4350 East-West Highway, Suite 800
Bethesda, MD 20814
301-634-9322
Steven Pedlow, MS
Senior Statistician II, Statistics and Methodology Department
NORC at the University of Chicago
55
East Monroe Street
Chicago, IL 60603
312-759-4084
Stephanie Poland, MA
Survey Director, Public Health Department
NORC at the University of Chicago
55
East Monroe Street
Chicago, IL 60603
312-759-4261
Catharine Fromknecht
Senior Research Analyst, Public Health Department
NORC at the University of Chicago
4350 East-West Highway, Suite 800
Bethesda, MD 20814
301-634-9384
Megan Heffernan, MPH
Research Analyst, Public Health Department
NORC at the University of Chicago
4350 East-West Highway, Suite 800
Bethesda, MD 20814
301-634-9412
The government project officer for this study is:
Allison Roper, LICSW
Public Health Advisor
Office of Disease Prevention and Health Promotion
Office
of the Secretary, U.S. Department of Health and Human Services
1101
Wootton Parkway, Suite LL-100
Rockville, MD 20852
(240) 453-8263
Appendix 1
TITLE III—GENERAL POWERS AND DUTIES OF PUBLIC
HEALTH SERVICE
PART A—RESEARCH AND INVESTIGATION
IN GENERAL
SEC. 301. ø241¿ (a) The Secretary shall conduct in the Service,
and encourage, cooperate with, and render assistance to other appropriate
public authorities, scientific institutions, and scientists in
the conduct of, and promote the coordination of, research, investigations,
experiments, demonstrations, and studies relating to the
causes, diagnosis, treatment, control, and prevention of physical
and mental diseases and impairments of man, including water purification,
sewage treatment, and pollution of lakes and streams. In
carrying out the foregoing the Secretary is authorized to—
(1) collect and make available through publications and
other appropriate means, information as to, and the practical
application of, such research and other activities;
(2) make available research facilities of the Service to appropriate
public authorities, and to health officials and scientists
engaged in special study;
(3) make grants-in-aid to universities, hospitals, laboratories,
and other public or private institutions, and to individuals
for such research projects as are recommended by the advisory
council to the entity of the Department supporting such
projects and make, upon recommendation of the advisory council
to the appropriate entity of the Department, grants-in-aid
to public or nonprofit universities, hospitals, laboratories, and
other institutions for the general support of their research;
(4) secure from time to time and for such periods as he
deems advisable, the assistance and advice of experts, scholars,
and consultants from the United States or abroad;
(5) for purposes of study, admit and treat at institutions,
hospitals, and stations of the Service, persons not otherwise eligible
for such treatment;
(6) make available, to health officials, scientists, and appropriate
public and other nonprofit institutions and organizations,
technical advice and assistance on the application of statistical
methods to experiments, studies, and surveys in health
and medical fields;
(7) enter into contracts, including contracts for research in
accordance with and subject to the provisions of law applicable
to contracts entered into by the military departments under
title 10, United States Code, sections 2353 and 2354, except
that determination, approval, and certification required thereby
shall be by the Secretary of Health, Education, and Welfare;
and
Sec. 301 PUBLIC HEALTH SERVICE ACT 68
(8) adopt, upon recommendations of the advisory councils
to the appropriate entities of the Department or, with respect
to mental health, the National Advisory Mental Health Council,
such additional means as the Secretary considers necessary
or appropriate to carry out the purposes of this section.
The Secretary may make available to individuals and entities, for
biomedical and behavioral research, substances and living organisms.
Such substances and organisms shall be made available
under such terms and conditions (including payment for them) as
the Secretary determines appropriate.
(b)(1) The Secretary shall conduct and may support through
grants and contracts studies and testing of substances for carcinogenicity,
teratogenicity, mutagenicity, and other harmful biological
effects. In carrying out this paragraph, the Secretary shall consult
with entities of the Federal Government, outside of the Department
of Health, Education, and Welfare, engaged in comparable activities.
The Secretary, upon request of such an entity and under appropriate
arrangements for the payment of expenses, may conduct
for such entity studies and testing of substances for carcinogenicity,
teratogenicity, mutagenicity, and other harmful biological effects.
(2)(A) The Secretary shall establish a comprehensive program
of research into the biological effects of low-level ionizing radiation
under which program the Secretary shall conduct such research
and may support such research by others through grants and contracts.
(B) The Secretary shall conduct a comprehensive review of
Federal programs of research on the biological effects of ionizing
radiation.
(3) The Secretary shall conduct and may support through
grants and contracts research and studies on human nutrition,
with particular emphasis on the role of nutrition in the prevention
and treatment of disease and on the maintenance and promotion
of health, and programs for the dissemination of information respecting
human nutrition to health professionals and the public. In
carrying out activities under this paragraph, the Secretary shall
provide for the coordination of such of these activities as are performed
by the different divisions within the Department of Health,
Education, and Welfare and shall consult with entities of the Federal
Government, outside of the Department of Health, Education,
and Welfare, engaged in comparable activities. The Secretary, upon
request of such an entity and under appropriate arrangements for
the payment of expenses, may conduct and support such activities
for such entity.
(4) The Secretary shall publish a biennial report which
contains—
(A) a list of all substances (i) which either are known to
be carcinogens or may reasonably be anticipated to be carcinogens
and (ii) to which a significant number of persons residing
in the United States are exposed;
(B) information concerning the nature of such exposure
and the estimated number of persons exposed to such substances;
(C) a statement identifying (i) each substance contained in
the list under subparagraph (A) for which no effluent, ambient,
69 PUBLIC HEALTH SERVICE ACT Sec. 302
or exposure standard has been established by a Federal agency,
and (ii) for each effluent, ambient, or exposure standard established
by a Federal agency with respect to a substance contained
in the list under subparagraph (A), the extent to which,
on the basis of available medical, scientific, or other data, such
standard, and the implementation of such standard by the
agency, decreases the risk to public health from exposure to
the substance; and
(D) a description of (i) each request received during the
year involved—
(I) from a Federal agency outside the Department of
Health, Education, and Welfare for the Secretary, or
(II) from an entity within the Department of Health,
Education, and Welfare to any other entity within the Department,
to conduct research into, or testing for, the carcinogenicity of
substances or to provide information described in clause (ii) of
subparagraph (C), and (ii) how the Secretary and each such
other entity, respectively, have responded to each such request.
(5) The authority of the Secretary to enter into any contract for
the conduct of any study, testing, program, research, or review, or
assessment under this subsection shall be effective for any fiscal
year only to such extent or in such amounts as are provided in advance
in Appropriation Acts.
(c) The Secretary may conduct biomedical research, directly or
through grants or contracts, for the identification, control, treatment,
and prevention of diseases (including tropical diseases)
which do not occur to a significant extent in the United States.
(d) The Secretary may authorize persons engaged in biomedical,
behavioral, clinical, or other research (including research
on mental health, including research on the use and effect of alcohol
and other psychoactive drugs) to protect the privacy of individuals
who are the subject of such research by withholding from all
persons not connected with the conduct of such research the names
or other identifying characteristics of such individuals. Persons so
authorized to protect the privacy of such individuals may not be
compelled in any Federal, State, or local civil, criminal, administrative,
legislative, or other proceedings to identify such individuals.
1 Indian Health Service. Locations. Available at: http://www.ihs.gov/locations/
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Funn, Sherrette (OS/ASA/OCIO/OEA) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |