Trends in U.S. Public's Awareness

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

General Population

The potential respondent universe for the proposed information collection includes all adults age 18 and older who reside in telephone households in the continental U.S.—from which, NORC will draw a nationally representative sample. This study will use a stratified, random digit dial (RDD) sample. To produce reliable estimates for racial and ethnic minorities, the project team will oversample geographic areas that have large concentrations of racial and ethnic minority populations. Such oversampling will be achieved by completing the following steps: (1) the racial and ethnic composition of each telephone exchange will be estimated by matching the exchange to block group level census data; (2) the RDD frame of telephone numbers will be stratified by telephone exchange based on the racial and ethnic composition of each exchange; (3) the racial and ethnic composition of each stratum will be estimated from exchange level information; (4) the stratum sample size/allocation will be determined based on the required number of interviews per racial/ethnic group and the tradeoff between cost and variance with the minority strata being sampled at a higher rate relative to their share in the sampling frame; and, (5) the sample will be selected systematically and independently from each stratum.

In order to meet the targeted number of completed surveys, the project team will use targeted list samples to supplement the RDD sample. For instance, should the RDD sample not produce a sufficient number of surveys completed by Asians or Hispanics, the team will use list frames to sample Asians and Hispanics more efficiently. Such list frames are compiled based on Asian and Hispanic surnames. Estimates from the RDD sample and the list sample can be combined to derive the composite estimate. NORC plans to work with GENESYS Sampling Systems (the same company which was used by KFF/PSRAI to draw the sample in 1999) to implement the proposed RDD sample design.

The project team will determine the optimal allocation of the sample based on our past experience, and on information provided by the sample vendor. We will determine the sample size and the stratum allocation by considering a series of expected outcome rates, including resolution rate, working residential number rate, screener completion rate, eligibility rate, and interview completion rate. Some of these rates will differ by stratum and by race/ethnicity, increasing design effects, which reduces the effective sample size. The potential for increases in design effects is another important consideration in sample allocation. Within each stratum, telephone numbers will be selected systematically, with equal probabilities, from working phone banks that contain one or more residential listings. See below exhibit for observed (1999) and Expected (2008) Survey Interviews, by Strata:

Exhibit 4: Observed (1999) and Expected (2008) Survey Interviews, by Strata

R/E Group (Strata)	1999	2008*	Difference
White**	1,479	1,200	(279)
Black	1,189	1,200	11
Hispanic	983	1,200	217
Asian/Pacific Islander	***	500	-
Physicians	***	360	-
Not coded (missing)	233	0	-
Total Completes	3,884	4,460	576

Source: Americans’ Perception of Racial Disparities in Health Care, Princeton Research Associates, The Kaiser Family Foundation: Methodology. September, 1999.

*Figures in this column represent the total number of expected or targeted number of completed interviews.

**Refers to Whites non-Hispanics

***AA/PI subjects were not included in the 1999 sample, and the survey was not administered to physicians in 1999.

Practicing Physicians

In addition to the survey of U.S. households, the instrument will be fielded to a national probability sample of U.S. practicing physicians. This administration will be used to obtain a baseline estimate of the overall physician population given that this instrument has not been fielded in this population. The survey is a pilot effort that will lay the foundation for additional work while at the same time providing data that will be of immediate interest. Although the survey will provide estimates that are representative of all US physicians providing patient care and will give unbiased estimates, it will not allow subgroup analysis such as of Black or Hispanic physicians, or physicians practicing in localities with high minority populations. We believe that the focus on all practicing physicians will provide the most accurate picture of current perceptions of disparities and will allow the project team to best answer one of the core research questions regarding the comparability of attitudes about racial disparities in the overall physician population and the general population.

A random sample of approximately 700 physicians from the AMA Masterfile will be purchased from one of the vendors authorized by the AMA to develop and deliver these sampling frames. Based on the project team’s experience using this sampling frame, it is anticipated that approximately 100 cases will be ineligible (e.g., the doctor may have retired, died, or is no longer involved in direct patient care). Also, a response rate of 60 percent is expected, which will yield about 360 completed cases. Given that a simple random sample will be used, sampling weights will not be required.

2. Information Collection Procedures

General Population

As stated in B1., the instrument will be fielded to a random sample of U.S. households, oversampling for African American, Hispanic, and Asian American/ Pacific Islander groups. The mode of data collection will be a telephone survey. Interviews will be conducted by experienced NORC interviewers who will receive training specific for the planned data collection. In addition to the computerized interview and related procedures, interviewers will be given scripts for contacting, consenting and re-contacting respondents (Attachment J). In addition, interviewers will be provided a list of Frequently Asked Questions (Attachment Q) to refer to during the telephone calls. NORC will utilize software providing enhanced call scheduling capabilities that support intelligent calling rules. Not only can these rules guarantee a minimum number of call attempts to reach a household within the required data collection period, but they can also reference both case-level call history and questionnaire embedded sample management data to distribute future call attempts to new days and times and finalize sample that has reached the maximum call attempts. NORC’s Computer Assisted Telephone Interviewing (CATI) system also features an exclusive state-of-the-art auto-dialer that has been specifically customized for social science surveys with the chief goal of maximizing response rates while containing costs.

Interviewers will make a minimum of 10 attempts to complete an interview at every sampled telephone number. The calls will be staggered over different times of the day and days of the week to maximize the chances of making a contact with a potential respondent. All interview break-offs and refusals will be re-contacted at least once in order to attempt to convert them to completed interviews. All of those with an initial refusal will be re-contacted at least once and offered an incentive of $15 to complete the interview.¹

In each contacted household, interviewers will ask to speak with the “resident eighteen and older who most recently had a birthday.” If that person is not at home an appointment will be scheduled.

Practicing Physicians

Below are brief descriptions of the various steps and methods that will be required to field the instrument to physicians. NORC will first mail a pre-notification letter to all sampled U.S. practicing physicians (Attachment M-1). Approximately 10 days following the pre-notification letter, the project team will mail the initial questionnaire to all sampled respondents. This mailing will utilize all updated address information resulting from returned pre-notification letters. Integrated into each questionnaire will be a cover letter (Attachment M-2) and instruction pages. The personalized cover letter will describe the purpose of the study and request participation. NORC proposes that the letter encourage cooperation by making sure respondents have the most convenient means available to respond. We will offer a choice to respond via phone, mail or a secure, dedicated fax-line. Included in the initial mailing will be a pre-paid incentive of $150. Follow-up mailings (Attachment M-3) will be sent to respondents whose questionnaires have not been received within one week of distribution. These second versions will be identical to the initial mailings, with the exception of the cover letter, which will be revised to acknowledge the earlier mailing and express gratitude to those who have already responded. If, after two weeks we have not received a completed survey; we will begin the telephone prompting effort (Attachment M-4). These calls will serve to boost the response rate achieved from the original mailing.

Telephone interviewers will prompt providers who have not yet returned their completed surveys despite receiving the initial and follow-up packages via U.S. mail. They will gain cooperation and offer options of re-mailing the questionnaire, faxing or conducting a telephone interview. An important task for the interviewer will be the management of resistance from gatekeepers, such as office managers, to achieve contact with sampled physicians.

Sample size and power analyses were performed to determine the degree of accuracy and level of confidence in which inferences can be made from the proposed survey sample to the universe of respondents. See Attachment N.

3. Ways to maximize response rates

The project team will be adhering to various survey administration methodologies and procedures that are designed to achieve the highest possible response rate. As mentioned in Section A15., the project team will be making significant reductions to the number of instrument items. A crosswalk of revisions made to the 1999 KFF/PSRAI Survey, which includes a brief description of the revisions as well as the rationale for the modification, appears in Attachments E-1 and E-2. The items from the 1999 questionnaire which were excluded from the revised instrument can be found in Attachment J, and lists of new items added to the revised questionnaires are contained in Attachments O-1 and O-2.

The hardcopy questionnaire will be designed to minimize the burden on the respondent by using a layout that is attractive in appearance and easy to complete. For instance, the questionnaire will be printed in booklet format; it will have a vertical flow of questions and sections of questions based on content will be created (see Attachment P). In addition, respondent will be given a choice of data submission method, either via mail, secure fax or telephone.

In addition to reducing respondent burden, the project team intends to provide incentives to survey participants. NORC project staff members have implemented experiments to study the effect of incentives both in the general population (Berk, et al., 1987) as well as on physician surveys (Berk, et al., 1993) and the results of our work as well as others (Berry and Kanouse, 1987) have generally suggested that incentives on physician surveys are quite effective. As such, the project team will follow the protocol followed in the 1999 KFF/PSRAI administration of conducting follow-up calls to non-responders and offering an incentive of $15 to complete the interview. For the administration to physicians, the project team will include a pre-paid incentive of $150 in the initial mailing to physicians.

General Population

The sample will be released for interviewing in replicates, which are random subsamples of the overall sample, to facilitate sample management at the NORC call center. Releasing the sample by replicates ensures that only enough telephone numbers are screened to achieve the target sample size. To the extent possible, NORC will follow the same calling protocol used in the KFF/PSRAI study to reduce bias due to design changes. For example, at least ten attempts will be made to complete an interview at every sampled telephone number. To maximize the chances of making a contact with a potential respondent, the calls will be spread over various times of the day and days of the week. All interview break-offs and refusals will be re-contacted at least once to attempt to convert them to completed interviews.

Practicing Physicians

Pre-notification Letter. The pre-notification letter (Attachment M-1) will be printed on OMH letterhead and personalized with respondent name, address and appropriate salutation. We propose to use business quality window envelopes showing OMH logo.

Initial Questionnaire Mailing. Approximately 10 days following the pre-notification letter, the project team will mail the initial questionnaire to all sampled respondents. This mailing will utilize all updated address information resulting from returned pre-notification letters. Integrated into each questionnaire will be a cover letter and instruction pages. The personalized cover letter will describe the purpose of the study and request participation (Attachment M-2). NORC proposes that the letter encourage cooperation by making sure respondents have the most convenient means available to respond. We will offer a choice to respond via phone, mail or a secure, dedicated fax-line. Included in the initial mailing will be a pre-paid incentive of $150.

Second Questionnaire Mailing. Follow-up mailings (Attachment M-3) will be sent to respondents whose questionnaires have not been received within one week of distribution. These second versions will be identical to the initial mailings, with the exception of the cover letter, which will be revised to acknowledge the earlier mailing and express gratitude to those who have already responded.

Telephone Prompting. If, after two weeks we have not received a completed survey; we will begin the telephone prompting effort (Attachment M-4). These calls will serve to boost the response rate achieved from the original mailing. Telephone interviewers will be responsible for conducting the following activities:

• Telephone prompting of providers who have not yet returned their completed surveys despite receiving the initial and follow-up packages via U.S. mail.

• Gaining cooperation and offering options of re-mailing the questionnaire, faxing or conducting a telephone interview.

• Managing resistance from gatekeepers, such as office managers, to achieve contact with sampled physicians.

4. Test of Procedure

A pilot test of the data collection instruments and study design will be conducted during the OMB initial OMB review period, using nine (9) adults over 18 years of age and nine (9) practicing physicians. Any changes to the instruments or study procedures will be detailed in the final clearance package.

5. Statistical Consultants

The project was awarded to the National Opinion Research Center (NORC) at the University of Chicago in 2007. The main NORC personnel on this contract are:

Oscar Espinosa, M.A.

Project Director

National Opinion Research Center, at the University of Chicago

(301) 634-9344

Alma Kuby, Ph.D.

Senior Survey Director, II

National Opinion Research Center, at the University of Chicago

(312) 759-4007

Bernard Dugoni, Ph.D.

Senior Survey Methodologist

National Opinion Research Center, at the University of Chicago

(773) 256-6193

Contractor personnel will implement the sample design, conduct data collection, handle data receipt/editing/keying, produce the data file, conduct statistical analysis and develop a survey report. OMH will provide direction and review functions to the contractor. Data collection is planned for September through mid-December of 2008.

REFERENCES

Berk ML, Mathiowetz NA, Ward EP, White AA. The effect of prepaid and promised incentives: results of a controlled experiment. Journal of Official Statistics, 3(4): 449-457, 1987.

Berk ML, Edwards WS, Gay NL. The use of a prepaid incentive to convert nonresponders on a survey of physicians. Evaluation and the Health Professions, 16(2): 239-45, 1993.

Berry, H. and Kanouse, D.E. Physician response to a mailed survey. Public Opinion Quarterly, 54: 102-114, 1987.

Gunn, W. and I. Rhodes. Physician Response Rates to a Telephone Survey:  Effects of Monetary Incentives on Response Level.  Public Opinion Quarterly, 45:109-115. 1981.

Halpern, SD, Ubel, PA, Berlin, JA, Asch, DC. Randomized Trial of $5 Versus $10 monetary incentives, envelope size, and candy to increase physician response rates to mailed questionnaire. Medical Care 40 (9):834-39, 2002.

1 In the 1999 survey a total of 270 of 2,318 refusals (12%) were converted to completes, using this method.

7