Supporting Statement B for:
The National Survey of Physician Attitudes Regarding the Care of Cancer Survivors (SPARCCS) (NCI)
October 1, 2008
(Anticipated date submission will be sent to HHS)
Paul Han, M.D.,
Task Order Monitor
National Institutes of Health
National Cancer Institute
Division of Cancer Control and Population Sciences
Applied Research Program
Health Services and Economics Branch
EPN 4005; 6130 Executive
Boulevard
Bethesda, MD 20892-7344
Phone: 301-402-3362
Fax: 301-435-3710
E-mail: potoskya@mail.nih.gov
Table of Contents
B. Collections of information employing statistical methods 1
B.1 Respondent Universe and Sampling Methods 1
B.2. Procedures for the Collection of Information 3
B.3 Methods to Maximize Response Rates and Deal with Non-response 11
B.4 Test of Procedures or Methods to be Undertaken 14
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting
and/or Analyzing Data 15
References 16
ATTACHMENTS
1 Medical Oncologist Instrument
2 Primary Care Physician Instrument
3 Questionnaire design consultants
4 Statement Regarding the Privacy Act Systems of Records
5 Westat & NCI IRB Approvals
6 NCI Cover Letter, Letters of Support & FAQs
7 Non-Response Follow-Up Letters
8 Primary Care Physician Script
9 Medical Oncologist Script
10 Follow-up call script
11 FAX and email
B. Collections of Information Employing Statistical
Methods
The primary populations of interest for this study are non-federal, office-based physicians belonging to two primary care specialties (General Internal Medicine doctors (IMs) or Family Practitioners (FPs)) who provide clinical care for at least 20 percent of their professional time, and non-federal oncology specialists who provide clinical care for at least 20 percent time. Another PCP specialty, OB/GYNs, will also be surveyed as part of this study in order to obtain some basic information for this specialty regarding the issues addressed in the questionnaire. However, OB/GYNs are of secondary interest analytically and will be undersampled.
The contractor for the National Survey of Physician Attitudes Regarding the Care of Cancer Survivors (SPARCCS) completed the Cancer Screening Study (CSS) (OMB No. 0925-0562) in 2007 for NCI, for which data were obtained for the three primary care specialties of interest in SPARCCS. Table B.1-1 provides information related to the estimated response and eligibility rates as well as the estimated population sizes for the three PCP specialties based on the CSS data.
Table B.1 - 1 Sample and Population Distributions and Response Rates from the Cancer Screening Study
Physician Type |
Number of Eligible Physicians Responding |
Estimated Response Rate |
Estimated Population Size |
Estimated Eligibility Rate (Pop Size to Frame Size) |
Family Practitioner |
924 |
71.8 |
61,308 |
82.7 |
Internal Medicine |
797 |
67.1 |
54,403 |
67.9 |
Obstetrics/Gynecology |
638 |
66.3 |
26,417 |
79.0 |
TOTAL |
2,359 |
69.0 |
142,128 |
76.4 |
For the purposes of this survey medical oncologists are defined as those physicians in the physician specialties of “Medical Oncologist” and “Hematologist/Oncologist” on the AMA frame. This group will generally be identified as oncologists in the discussion here. Of analytic interest is the partitioning of oncologists into two groups, those characterized as “academically-based” and those characterized as “community-based”. As previously mentioned in Section A16, academically-based physicians are defined as physicians who provide clinical care within a university-affiliated system (item 22 for PCPs: item 23 for oncologists), or who teach medical students (item 20, for PCPs: item 21 for oncologists). Community physicians are defined as all other physicians. Based on previous studies, it is expected that “community-based” oncologists will make up roughly 60 to 65 percent of all oncologists (Helft et al found that 62% of a sample of oncologists listed “community setting”, and 38% listed “University / Academic Setting” as their “practice environment,” respectively). From the CSS data it has been estimated that roughly a third of the FPs and IMs are “academically-based”.
In developing the sample design, two general analytic objectives were considered of primary importance:
Establishing estimates for FPs, IMs, and oncologists as well as FPs and IMs together as the primary PCP group of interest; and
Making comparisons between physicians of the following specialty types and groupings: FPs vs. IMs; PCPs (FPs plus IMs) vs. oncologists; within oncologists, Academically-based vs. Community-based; and Academically-based vs. Community-based regardless of specialty.
The targeted number of physicians participating in SPARCCS is: 500 FPs, 500 IMs, 100 OB/GYNs, and 1,100 oncologists. Thus, 1,100 PCPs (including the OB/GYNs) and 1,100 oncologists are the overall targeted number of participants, when considering PCPs and cancer specialists as two distinct analytic groups. The numbers of physicians associated with these various groups on the AMA file in May, 2008 are: FPs, 73,031; IMs, 76,550; OB/GYN, 33,193 (31,008 OB/GYNs and 2,185 gynecologists) and oncologists, 10,460 (5,205 hematology/oncology and 5,255 oncology). Osteopaths appearing on the AMA file are to be included in the study.
We expect a response rate of at least 70 percent, consistent with Westat’s previous experience with the data collection procedures described in Section B.3 below.
Target Population and Sample Frame
As discussed previously, the target populations for this study are practicing medical oncologists and PCPs: specifically, non-federal physicians who are medical oncologists, hematologist/oncologists, FPs, general internists (IMs), and OB/GYNs (those physicians classified as gynecologists rather than OB/GYNs will also be eligible for sampling within the OB/GYN stratum) with patient care accounting for at least 20 percent of their professional time.
As previously mentioned, we plan to use the American Medical Association (AMA) master file to construct the sample frame. The AMA file provides the most complete coverage available of physicians of all available sample frames and has been used in numerous studies, including the National Ambulatory Medical Care Survey (NAMCS) carried out by the National Center for Health Statistics, the 1999-2000 National Survey of Colorectal Cancer Screening Practices (OMB No. 0925-0468), the CSS, and most recently the National Survey of Energy Balance Related Care Among Primary Care Physicians (OMB No. 0925-0583). In constructing the sample frame, we will exclude physicians with addresses outside the continental U.S. as well as physicians who do not meet survey eligibility requirements based on data appearing on the lists (not in active practice; deceased; not among the targeted physician specialties; federal; classified as research, administration or teaching as the primary professional activity--as opposed to office-based practice). Notably, as previously discussed in section A-16 p. 11, some of these physicians will be academically-based (i.e., teaching medical students or providing patient care in a setting that is affiliated with academia).
Sampling Procedures
Once the frame is formed, a systematic random sample of physicians will be selected within each of four strata: FPs, IMs, OB/GYNs (including gynecologists), and medical oncologists (including those whose specialty is characterized as hematology/oncology). Within each sample stratum, we will sort by variables that will help achieve an implicit stratification prior to sample selection. Candidates for sort variables include region of country, metropolitan statistical area (MSA) status, age group and sex of the physician. We will sample the total number of physicians required for fielding the two questionnaires combined. Then, the primary care physicians (i.e. OB/GYNs, FPs and IMs) will be sent the PCP questionnaire (Attachment 2) while the oncologists will receive the oncologist questionnaire (Attachment 1). The sample sizes selected will take into account expected levels of nonparticipation and ineligibility. The reduction in precision due to design effects arising from the under sampling of OB/GYNS for any analyses where all three PCP specialties are grouped as a single analytic domain is recognized and is considered acceptable because estimates related to OB/GYNs are not a primary focus of this study. A reserve sample will be selected to be used in the event that some of the assumptions used in determining initial sample sizes depart from what was expected.
Power Analysis
There are a number of analytic objectives for this survey effort. A margin of error of roughly plus or minus three percent at the 95 percent confidence interval is targeted for national estimates for medical oncologists as well as for IMs and FPs pooled together. To meet this objective, we are targeting 1,100 completed questionnaires from oncologists and 500 each from FPs and IMs.
Table B.2 - 1 shows the sample allocation planned for this study in terms of the targeted sample yield (number of participating physicians). It also shows the estimated power to detect a difference of 10 percent (comparing estimates of 60 percent to 50 percent) with an alpha level of .05, the estimated effective sample sizes. The power values are all over 90 percent. Note that the power for detecting existing differences when comparing all Academic to all Community physicians (i.e., after pooling oncologists, FPs, and IMs together) is lower than that for oncologists alone. This is due to the large design effect (estimated as 2 for the power calculations here) arising from the fact that oncologists are sampled at a far higher rate than FPs and IMs (the targeted yield for oncologists is 1,100 when sampling from a frame of roughly 10,000 oncologists, while the corresponding targets for FPs and IMs are 500 while sampling from over 70,000 physicians for each of these two specialties).
Table B.2 – 1 Estimated Power with Study Allocations
Allocation Strategy (Targeted Sample Yield) |
|
|
|
|
Expected Power to detect the specified difference between: |
Academic and Community Oncs |
Academic and Community Physicians |
|
|
FP |
IM |
Onc |
|
Onc and PCPs |
FP and IM |
||
Study |
500 |
500 |
1,100 |
|
99.9 |
93.8 |
94.5 |
93.1 |
Precision
Table B.2 – 2 shows the expected precision levels for most of the primary analytic domains of interest: FPs or IMs as a single grouping; PCPs (FPs and IMs pooled together); Medical Oncologists (Oncs); Academic Oncs (the smaller of the two comparison groups, Community vs. Academic); and Academics (Oncs plus PCPs). The level of precision expected is presented as the half-width of a 95% confidence interval for an estimated percentage of 50 percent. The precision requirement for the survey was to obtain roughly plus or minus three percent precision for PCPs as a group and oncologists as a group. Since OB/GYNs are of limited interest for this study, the PCP half-interval is presented for FPs and IMs grouped as a single analytic domain.
Table B.2 – 2 Estimated Precision Achieved for Target Populations
Allocation Strategy (Targeted Sample Yield) |
|
|
|
|
Expected half-width of a 95% confidence interval for full sample estimate of 50 percent |
Academic Oncs |
Academics (Oncs plus FPs plus IMs) |
|
FP |
IM |
Med Onc |
|
FP (or IM) only |
PCPs (FP plus IM) |
Med Onc |
||
500 |
500 |
1,100 |
|
4.38% |
3.1% |
2.95% |
4.79% |
5.07% |
Sample Size
Based on previous experience with collecting data from physicians, we believe that we will obtain a response rate of at least 70%. Since the eligibility criteria for this study should be similar to those used for the CSS, we expect that the eligibility rates will be around 80 percent for FPs and OB/GYNs and about 70 percent for IMs. There is some uncertainty about what the eligibility rate will be for oncologists. An 80 percent eligibility rate will be assumed, and the sample yields will be closely monitored. If it turns out that the oncologist eligibility rate is somewhat lower than 80 percent, we will release a random portion of a reserve sample in order to obtain the targeted sample yield. Table B.2 - 3 shows the targeted sample yields, based on the precision and power considerations, and the corresponding sample sizes required to achieve them. In total, we plan to sample 4,065 physicians for initial release for the survey. As described for oncologists, a reserve sample of all physician specialties will be selected to permit the supplementation of the sample, should assumptions about eligibility or response rates depart substantially from initial expectations.
Table B.2 - 3 Targeted Sample Yields and Corresponding Sample Sizes Need to Achieve Them
|
Family Practitioner |
Internist |
OB/GYN |
Oncologists |
Targeted Sample Yield |
500 |
500 |
100 |
1,100 |
Number to be Sampled |
895 |
1,025 |
180 |
1,965 |
Weights
Since a complex sample design is being employed, sample weights will be developed for use in the analyses. An initial or base weight will be assigned to each sample physician, reflecting his or her probability of selection. Base weights will vary by physician type, due to the under sampling of OB/GYNs and the over sampling of oncologists to meet the various analytic objectives of the study. After the base weight assignment, weights will be adjusted for non-response using information from the AMA file available on both respondents and non-respondents. These will include variables such as age, sex, and geographic location as well as any relevant practice related data. An evaluation of these variables will be undertaken to identify those that appear most effective in characterizing the propensity to respond. Cells will be formed from the variables so identified, and the weights of participating physicians associated with the cell will be adjusted to compensate for those in the same cell who do not participate. Since there are no independent counts of the population of physicians who are survey eligible, no post-stratification of the non-response adjusted weights is planned.
For purposes of variance estimation and analyses, either replicate weights will be formed using a jackknife replication methodology or “stratum” and “PSU” variables will be formed for use with a Taylor Series approach to variance estimation.
Survey Procedures
As discussed earlier, for fielding purposes, Westat will draw two samples – one for primary care physicians and one for oncologist physicians - of 2,200 physicians from the frame, as well as a reserve sample. To help maintain a high level of response, we will make telephone calls to the primary care physician and medical oncologist’s offices prior to the survey mailing. The script that will be used when placing calls to primary care physician and oncologist offices can be found in Attachments 8 and 9, respectively. The call will verify that the potential respondent is within the scope of the survey, and that the address is correct. For those physicians who have moved, a new address will be obtained from the old office or through tracing procedures using a variety of Internet sources.
The call will also provide estimates of the number of physicians eligible for the study across the various physician specialties. Based on the results of this screening effort, the need to supplement the sample to achieve targeted sample yields can be evaluated. If necessary, supplemental samples can be selected from the reserve sample of physicians for each physician type sample that appears to require supplementation.
Each sampled physician will then be sent a package containing the survey instrument, a cover letter from NCI describing the study and the importance of their participation, a document describing the study in more detail, a letter of support from a specialty organization, a $50 honorarium check, and a postage-paid return envelope. The initial package will be sent by Priority Mail. These documents can be found in Attachment 6.
Subsequent survey packages will be sent by Priority Mail 18 days after the initial mailing to those who have not responded. This second package will contain a more urgently worded cover letter from NCI (Attachment 7), a questionnaire, and a prepaid return envelope. Follow-up calls for this group will begin the day after the second package arrives in the physician’s office. Callers will verify that the survey has been received by the physician, and will leave a reminder message. At this time, physicians will also be offered the option of completing the survey over the phone. An additional survey and a replacement honorarium check will be sent if necessary. If no response is obtained after several weeks, additional reminder messages will be left. Some physicians have provided email and FAX information to the vendor of the AMA Master File; since many physicians prefer this contact method, reminders will also be sent using these methods. Physicians who have agreed to complete the survey, but from whom no survey has been received within two weeks from the date the physician agreed to complete the survey will receive follow-up reminder phone calls. Finally, an attempt will be made to gain physician cooperation through a call to an office administrator.
A third survey package will be sent by Priority Mail to those physicians who say they have lost the second package and to physicians who have not responded within approximately 8 weeks after the initial mailing. This package will contain a cover letter (Attachment 7) urging the non-respondents to participate, a survey and a prepaid return envelope. Follow-up reminders will also be left after the third mailing using telephone, FAX, or email. If necessary to achieve the target response rate, nonresponder conversion efforts will be implemented utilizing the contact method thought to be most effective for each nonresponder.
Exclusion criteria
Data received by the contractor will be reviewed for eligibility. Cases in the PCP sample will be excluded from the study at the screener level if office staff indicates that the doctor is not currently practicing in a primary care specialty, or that the doctor sees patients solely in a hospital, nursing home, correctional facility, or federal facility.
Cases in the oncology sample will be excluded from the study at the screener level if office staff indicates that the doctor is not currently practicing as an oncologist. In addition, practicing oncologists whose staff report that they treat neither colon nor breast cancer are not eligible for the study and will not receive a mailed survey. For both PCP and oncologists, physicians who indicate on the survey that they spend less than 20 percent time in clinical care (PCP: item 23, Oncologist: item 24) will be excluded.
Several procedures will be implemented to maximize the response rate. Physician mail survey response rates are more robust when the research topic is salient to their practice, when the questionnaire has been well designed for maximum ease of administration, and when the data collection protocol is tailored through a variety of incentives and accommodations to acknowledge physician cooperation and contribution. The presentation of the survey is also important to differentiate it from the multitude of research studies for which physicians are targeted. For this reason, the survey will be carefully designed with a graphically simple but pleasing layout, and sent via Priority Mail, which has proven to be a more effective mechanism for gaining the attention of physicians than standard U.S. mail.
The initial screening telephone call helps to improve the response rate by identifying ineligible physicians and physicians who are difficult to find before the surveys are mailed. Cases that are ineligible are quickly identified and accounted for during the screener, lessening the respondent burden of ineligible physicians filling out the questionnaire and the proportion of ineligible cases among nonresponders. Physicians that have moved from the AMA Master File address are traced intensively before the mailing of the survey to ensure there is plenty of time to find an updated mailing address. Because respondents that are not locatable are counted as nonresponders, this initial call and comprehensive tracing procedure helps to minimize non-response rates. Reserve sample will be added, as needed, to supplement the initial sample release to ensure the sample size is maintained for each physician group.
The introductory letter accompanying the questionnaire will indicate that the study is sponsored by NCI, recognized to be the national leader in cancer research. The letter will succinctly inform the reader of the importance of the survey, and procedures for maintaining the confidentiality of respondents (i.e., identities of individual physicians will not be released, identifying information will be stored separately from the survey responses, and all information collected will be analyzed in the aggregate).
In addition to the one-page letter, a two page document describing the study in greater detail will be enclosed. Since many physicians are reluctant to read a letter greater than one page in length, the initial letter will not exceed one page. However, some potential respondents with questions about the study will want more detailed information, which will be included in this second document.
The mailing will also include a letter of recommendation from the professional specialty society to which the respondent is likely to belong. A letter from the American Academy of Family Physicians will be sent to FPs, one from the Society of General Internal Medicine will be sent to general internists, and one from the American College of Obstetricians and Gynecologists will be sent to OB/GYNs. Oncologists will receive a letter from the American Society of Clinical Oncology (ASCO). These letters will emphasize the importance of the study and the need for physicians’ cooperation in completing the survey. The materials included in the initial mailing can be found in Attachment 6.
A monetary incentive or honorarium that sufficiently acknowledges the time and cooperation of the physician is increasingly common and expected. An incentive of $50 will be included in the survey mailing. A full discussion of how the incentive amount was determined may be found in Section A.9 (page 4).
The strategy for telephone follow-up has been carefully designed and will be staffed with callers with experience contacting physicians’ offices. The callers will be trained and supervised by an individual with extensive expertise in interfacing with physicians’ “gate-keepers,” who might otherwise dissuade direct contact with the physicians. By placing follow-up calls on the day that the second mailing is received and the days immediately following, the package is highly likely to be near-at-hand when the physician agrees to participate. By following up consistently and persistently, NCI will demonstrate that it is committing time and energy to obtain the most valid data possible by obtaining the opinions of as many physicians as possible. Since many physicians now prefer FAXes and emails to telephone messages, reminder FAXes and emails (Attachment 11) may also be sent to physicians who have provided this contact information to the AMA Master File vendor.
Survey staff will work with the physician to obtain data in whatever manner is convenient to the physician. Additional copies of the survey package will be sent to the physician if so required, and the physician will have the option of calling Westat to respond to the survey by telephone. Emails with a link to a printable survey will be sent to nonresponding physicians who have provided email addresses to telephone interviewers or to the vendor. In addition, the survey will also be FAXed to physicians whose office staff request it.
Consistent with the response rate calculations approved by the American Association for Public Opinion Research (AAPOR), response rates for this study will be calculated as follows:
Number of Completed Surveys
Number of Completed Surveys + Number of Non-respondents
Note that sampled physicians who, based on the responses provided, can be definitively characterized as “ineligible” for the study will be included as a “completed survey”. Sampled physicians who do not provide a completed survey, including those classified as “unlocateable”, will be included among the non-respondents. Both weighted and unweighted response rates will be computed.
Westat’s method for survey data collection from physicians has been consistently successful in obtaining response rates of over 68 and 80 percent. We expect to obtain a high response rate because of the initial contact by telephone to present the survey, the flexibility in the methods of response, and the extensive follow-up calls to interim non-responders, accompanied by re-mailing of the survey. Westat uses refusal avoidance methods during all communications to lessen the need for refusal conversion. For physicians who do refuse, an experienced refusal conversion interviewer will attempt to collect responses on questions deemed most critical.
A test of the oncologist screening interview was conducted with nine physicians to confirm that the screening procedures would yield the requested information for each physician respondent, and to refine these procedures. The PCP screening interview required no further testing, as it had been implemented successfully in previous studies.
Each survey instrument was cognitively tested with nine PCPs and nine Oncologists. In response to their comments, several questions were dropped or combined, response categories added to several items, and several small wording changes were made.
Individuals who are providing statistical consultation on this project include:
Noreen
M. Aziz MD, PHD, MPH
Ralph DiGaetano, MA Senior Statistician Westat (301) 294-2062
Carrie Klabunde, PHD,
Epidemiologist |
Caroline McLeod, PHD Senior
Study Director
Arnold L. Potosky, PHD Health Services and Economics Branch Applied Research Program Division of Cancer Control and Population Sciences National Cancer Institute
Michael Stefanek, PHD Director and Vice President of the Behavioral Research Center American Cancer Society 404-329-7795 |
Reference List
Aziz NM. Late effects of cancer treatments. In Ganz PA (editor), Cancer Survivorship Today and Tomorrow (pp. 54-76). New York: Springer, 2007.
Aziz NM, Oeffinger KC, Brooks S, Turoff AJ: Comprehensive long-term follow-up programs for pediatric cancer survivors. Cancer 107 : 841 -848, 2006
Division of Occupational and Employment Statistics, U.S. Bureau of Labor and Statistics. May 2006 National Occupational Employment and Wage Estimates, United States. 24 October 2007 http://www.bls.gov/oes/current/oes_nat.htm#b39-0000 (Accessed: 31 March 2007).
Ganz PA: Monitoring the physical health of cancer survivors: A survivorship focused medical history. J Clin Oncol 24 : 5105 -5111, 2006
Gunn WJ, Rhodes IN. Physician response rates to a telephone survey: effects of monetary incentive level. Public Opinion Quarterly 1981; 45(1):109-115.
Helft PR, Hlubocky F, Daugherty CK. Journal of Clinical Oncology, Vol 21, Issue 5 (March), 2003: 942-947.
Hewitt, M., S. Greenfield and E. Stovall, Eds. From Cancer Patient to Cancer Survivor: Lost in Transition Committee on Cancer Survivorship: Improving Care and Quality of Life, Institute of Medicine and National Research Council. (7 November 2005).
Hewitt M, Weiner SL, Simone JV: Childhood Cancer Survivorship: Improving Care and Quality of Life. Washington, DC, The National Academies Press, 2003
Hewitt M, Simone JV: Ensuring Quality Cancer Care. Washington, DC, The National Academies Press, 1999
Institute of Medicine. Implementing Cancer Survivorship Care Planning: Workshop Summary. Washington, DC: The National Academies Press, 2007.
Kalton G, Flores-Cervantes I. Weighting Methods. Journal of Official Statistics 2003; 19(2):81-97.
Oeffinger KC, McCabe MS: Models for delivering survivorship care. J Clin Oncol 24 : 5117 -5124, 2006
President's Cancer Panel 2005/2006 Annual Report: Assessing Progress, Advancing Change. Bethesda, MD, National Cancer Institute, 2006
President's Cancer Panel 2003/2004 Annual Report: Living Beyond Cancer: Finding a New Balance. Bethesda, MD, National Cancer Institute, 2004 Rizzo L, Kalton G, Brick JM. A comparison of some weighting adjustment methods for panel nonresponse. Survey Methodology 1996; 22(1):43-53.
Rizzo L, Kalton G, Brick JM. A comparison of some weighting adjustment methods for panel nonresponse. Survey Methodology 1996; 22(1):43-53.
| File Type | application/msword |
| File Title | WESTAT.DOT |
| Subject | Default Westat Styles |
| File Modified | 2008-09-18 |
| File Created | 2008-09-18 |