Surveys are being administered to collect information about how four groups of target-users, state (two groups), local, and tribal health departments, use and perceive Healthy People. The surveys will also collect information from non-users to determine the factors that prevent target groups from using the initiative in their organizations.
The results will be generalizable to the respondent universe, which consists of government entities that interact with HHS and their constituents to improve the health of the populations they serve.
The sample will include 502 organizations from the 50 states, the District of Columbia, and Native American tribes. The unit of analysis for the sample will be the organization, meaning that no more than one survey will be sent to each organization, although this sample treats state Healthy People State Coordinators as separate organizations from state Directors of Chronic Disease Programs. The project will census state health departments (Healthy People State Coordinators and Directors of Chronic Disease Programs separately), and sample local and tribal health organizations. This sample of public health officials will be able to provide the type of data necessary to evaluate the Healthy People initiative.
The sample frame will be constructed from multiple sources and will result in four separate lists for Healthy People State Coordinators, state Directors of Chronic Disease Programs, local health departments, and tribal nations. A list of the 51 State Healthy People Coordinators will serve as the primary contacts for the states (sample frame A). The list of state chronic disease directors will serve as the second set of state-level contacts (sample frame A*). The list of approximately 3,000 members of the National Association of County and City Health Officials (NACCHO) will serve as the sample frame for the local officials (sample frame B), and the tribal officials will be selected from a list of approximately 400 tribal health leaders provided by the Indian Health Service (sample frame C).
These frames will be used to draw samples that satisfy the study’s goals. The proposed sample design satisfies two key requirements. First, all 51 organizations from frames A and A-1 will be included with certainty. Second, the design will draw samples that produce nationally representative estimates for urban and rural organizations in group B, and nationally representative estimates by tribal size and region in group C.
Our total sample of 502 organizations will consist of all 102 organizations from frames A (State Healthy People Coordinators) and A* (State Chronic Disease Program Directors), plus 400 more sampled from frames B (Local Health Organizations) and C (Tribal Health Organizations). We will include in the sample 300 organizations from frame B and 100 organizations from frame C.
Exhibit 5 shows the sizes of the frames provided, as well as the sample sizes and expected response rates and respondent sizes.
Exhibit 5: Sample Statistics
|
Population |
Sample |
Expected Response Rate |
Total Expected Respondent (n) |
State Healthy
People Coordinators |
51 |
51 |
90% |
46 |
State Chronic Disease Program
Directors |
51 |
51 |
80% |
41 |
Local Health Organizations (Frame B) |
~3,000 |
300 |
80% |
240 |
Tribal Health Organizations (Frame C) |
~400 |
100 |
75% |
75 |
TOTAL |
~3,500 |
502 |
80% |
402 |
Selection Methods
For sample frame B and C we will use systematic samples with equal probability of selection (within frame) and implicit stratification. The only difference is on which variables will be used for implicit stratification. Implicit stratification involves sorting the frame on certain variables so that the sample drawn is representative on that variable. For example, assume that 44% of local health organizations are in a rural setting and 56% are in urban settings. By sorting on urban-rural status and then drawing a systematic sample, the resulting sample will be very close to including 44% of organization in a rural setting.
We will sort on multiple variables so that samples will be representative on more than one dimension. The variability in sample size percentages will increase for variables that appear later in the sorting. Serpentine sorting will be used when sorting on multiple variables to maximize the effect of the stratification. Serpentine sorting involves sorting by an order that is alternately increasing or decreasing. For example, serpentine sorting on urban/rural status and region could result in this sort order: Rural Northeast, Rural Midwest, Rural South, Rural West, Urban West, Urban South, Urban Midwest, and Urban Northeast. This sort order successfully keeps the two West strata together.
It should be noted that the level of precision for subgroup estimates may not be sufficient to make meaningful comparisons between frames. To account for this imprecision, we will employ strategic collapsing of strata in estimation to create estimates with a higher level of precision. For example, the urban groups and rural groups may be collapsed to form nationally representative estimates of urban and rural areas.
Local Health Organizations
The NACCHO list frame consists of approximately 3,000 records. However, we will remove any “inappropriate” records (e.g., “tribal” records) so that our sampling frame contains only local health organizations. Inappropriate records to be deleted include duplicate records, records without title or agency name, as well as other inappropriate records such as public health consultants, foundations, special interest groups (for hand gun violence, for example), students, professors, etc.
Since it is desired to have a representative sample with respect to urban and rural organizations, we will sort the file first on urban/rural status. Using the zip code from the file, we will map each organization to the state and county in which it resides. We will then determine if this county is inside a Census defined Metropolitan Statistical Area (MSA) or not. The Census Bureau defines MSAs as the counties that involve economic activity related to a central city. If the county is in an MSA, we will count this organization as “urban.” Otherwise, we will classify the organization as “rural.” Suburban organizations will be classified as “urban.”
Tribal Organizations
The target respondent is the lead tribal health representative, meaning the person within the tribe who has the authority and responsibility for disease prevention and health promotion activities. The tribal list frame consists of approximately 450 records. This file will also contain a code for the approximate size of Indian population excepting urban Indian health agencies and a few other organizations. The tribes are divided into small (< 2,500 Indian population), medium (2,500 – 10,000), and large (> 10,000). The tribal health agencies with unknown population size will be placed into a fourth category.
To ensure a representative mix of small, medium, and large tribes, we will sort the file on this size code. To the extent that tribal organizations are geographically diverse, we will draw the sample to be as representative as possible by sorting on Census Region.
The sample will include 502 organizations from state, local, and tribal organizations. The unit of analysis for the survey will be the organization, so that no organization will be asked to complete more than one survey, although this sample treats state Healthy People State Coordinators as separate organizations from state Directors of Chronic Disease Programs. Fielding of the survey will entail mailing surveys, along with a cover letter, to the key staff member at each organization. A self-addressed stamped envelope will be included with each survey so survey respondents can return the survey directly to the researchers. Respondents will also be offered the option to complete the survey online via the internet. A postcard mailing will be sent to respondents two weeks after the initial mailing, and a phone call will be made to those who have not responded after four weeks. The phone call will also provide an opportunity for the researchers to remail questionnaires that have been lost or misplaced, or to access the respondent’s case online and enter his or her responses over the telephone.
Project investigators will use an electronic receipt control system using case ID numbers to track the initial questionnaire mailing, address updates, remailing of questionnaires, mailing of postcard reminders, and complete and incomplete questionnaire returns. Reports from the system will identify the sample members which require prompting for completion of the survey.
All data from the completed questionnaires will be keyed (data entered) to create the analytic data file. Ten percent of the questionnaires will be randomly selected for keying a second time (double entry). The accuracy of the data entry process will be verified by comparing the data from the first entry with the data from the second entry. The double keying verification process will allow researchers to report the rate of accuracy to the Project Officer. The questionnaires will be processed in two batches. Data entry of the second and final batch will be completed within two weeks of the close of data collection.
The investigators will use a number of proven methods to maximize participation in the study. First, the instrument itself is designed to maximize response rates. The style of the survey is inviting and user friendly, with a maximum of 38 questions. The instructions for the survey are straightforward, and there are a limited number of skip patterns. The questionnaire will be pilot tested with six respondents from the sampling frame, and questions will be amended to reflect suggested improvements from these respondents. In addition to the questionnaire, each respondent will receive a cover letter encouraging participation in the survey. The cover letters (Attachments 2 and 3) will convey the importance of the survey to the ODPHP and the ASPE. The cover letters will also indicate that the respondent will not be identified to any government agency. The survey will be sent to respondents as a hardcopy SAQ with the option of completing the survey online via the internet. Completed hardcopy surveys may be returned via the included self-addressed, stamped envelope or by fax transmittal. It is anticipated that respondents will choose the option of least personal burden, thereby reducing the overall burden of the study. A postcard reminder will be sent to any non-respondents two weeks after the initial mailing, highlighting the convenience of the online completion option. Any outstanding non-respondents at four weeks after the initial mailing will be contacted using computer-assisted telephone interviewing to confirm that the hardcopy SAQ was received, and to inquire whether the respondent would like to complete the survey online or by telephone. If the respondent has lost or misplaced the hardcopy SAQ and indicates a preference for hardcopy completion, NORC will fax and/or mail the respondent a new hardcopy SAQ. If the respondent opts to complete the survey by telephone, the interviewer will access the respondent’s case online and enter responses directly into the online survey.
In the 2005 Assessment of the Users Healthy People 2010 and HealthierUS and, these same procedures were used with the same respondent population with good success. The response rate for the state Healthy People Coordinators group was 86%, the local sample has a response of 76%, and the tribal sample had a response of 73%. Overall the response to the survey was 78%. Given a greater familiarity with web-based surveys and the increased attention on the Healthy People program due to planning activities for 2020, we believe high response rates will be achieved. NORC is also conducting advance outreach work with IHS to facilitate higher response rates from tribal organization and will work with HHS to publicize the survey at regional stakeholder meetings.
A pilot test of six individuals will be conducted during the initial OMB review period.
The following individuals contributed to the questionnaire and study design and will be involved in the interpretation and analysis of findings:
Daniel Gaylin, MPA
Executive Vice President
NORC at the University of Chicago
4350 East-West Highway, Suite 800
Bethesda, MD 20814
301-634-9417
Caitlin Oppenheimer, MPH
Principal Research Scientist
Department of Health Policy and Evaluation
NORC at the University of Chicago
4350 East-West Highway, Suite 800
Bethesda, MD 20814
301-634-9322
Angela DeBello, MA
Associate
Director
Department of Public Health and Epidemiology
NORC at the University of Chicago
55
East Monroe Street
Chicago, IL 60603
312-759-4069
Steven Pedlow, MS
Senior Survey Statistician
Department of Statistics and Methodology
NORC at the University of Chicago
55
East Monroe Street
Chicago, IL 60603
312-759-4084
The government project officer for this study is:
Wilma Tilson, MPH
Health Policy Analyst
US Department of Health and Human Services
Office of Assistant Secretary of Planning and Evaluation
Office of Health Policy
200 Independence Ave, SW
Room 447 D
Washington, DC 20201
| File Type | application/msword |
| Author | DHHS |
| Last Modified By | DHHS |
| File Modified | 2008-05-16 |
| File Created | 2008-05-16 |