ss_BPHC Patient Survey - NEW - PART B

Supporting Statement
Bureau of Primary Health Care Patient Survey

B.

Collection of Information Employing Statistical Methods
1.

Respondent Universe and Sampling Methods

The respondents for the survey are patients who receive services from section 330-funded
health center grantees. These respondents are clustered within health center sites within
each grantee. The grantee, site, and patient universes are described in greater detail in the
subsections that follow. Please see Attachments 8 and 10, the Statistical Design Plan
and the Survey Methodology and Selection Specifications for further detail.
a. Grantee Universe
The survey sample members will be selected from eligible grantees. The updated BPHC
Uniform Data System (UDS) grantee-level data file will be used to identify eligible
grantees. All grantees within the UDS and funded by section 330 will be eligible except
the following:
•
•
•
•
•

Grantees located in U.S. territories or possessions (i.e., those in Puerto Rico,
the Virgin Islands, and the Pacific Basin);
Grantees funded by section 330 which have been operating less than 1 year;
Grantees funded through the CHC funding program that only operated schoolbased sites;
Grantees that received Migrant Health Center funding program only and that
only served clients through a voucher program; and
Any grantee that is no longer a section 330-funded grantee.
b. Site Universe

Many grantees operate multiple health center sites. The sites eligible for the survey
within selected grantees are required to participate in at least one of the four specific
funding programs (CHC, MHC, HCH, and PHPC). All sites within selected grantees are
eligible unless:
•
•
•
•

The site has been operating under the grantee for less than 1 year;
The site is a school-based health center;
The site is a specialized clinic, excepting clinics providing OB/GYN service; and
The site provides services only through a migrant and seasonal farmworker
voucher-screening program.
c. Patient Universe

The patients eligible for the survey must satisfy the following eligibility requirements:
•

Patients must have received face-to-face services at the site from a clinical
staff member who exercises independent judgment in the provision of service
at least once in the 12 months prior to the current visit.

•
2.

If the patient is under the age of 18, the patient must be accompanied by an
adult.
Procedures for the Collection of Information
a. Sample

As noted, the section 330-funded grantees operate one or more sites. The sites are
clustered within grantees, and the patients are clustered within the sites within the
grantee. A three-stage sample design will be used in which the grantees are selected as
the primary sampling units (PSUs), sites are selected within participating grantees, and
patients are selected within selected sites. Our target is to recruit 115 grantees and
complete 4,526 interviews, specifically 2,210 for CHC, 826 for MHC, 826 for HCH and
660 for PHPC.
b. Grantee Sample
The sampling frame is constructed from all the eligible grantees (the grantee universe) in
the BPHC’s current UDS data file. 115 grantees will be selected through a stratified
probability proportional to size (PPS) for participation from 12 strata with a pre-specified
sampling rate for each stratum. Independent site and patient samples will be selected for
each funding program if the grantee receives multiple section 330 funds. Three sites will
be selected using the PPS sample using PROC SURVEYSELECT in SAS Version 9.1.3.
The 12 strata will be formed as shown in Attachment 9. They are defined as follows:
1. First Stage Strata
Four mutually exclusive strata by grouping grantees according to the types of
funding they receive (Stratum1, Stratum2, Stratum3 and Stratum4). Those first
stage strata are used to ensure that the selected grantees are representative to the
four funding programs.
2. Second Stage Strata
To ensure the grantees with single funding type of MHC or HCH are represented
in the grantee sample, we split Stratum3 and Stratum4 into two second-stage
strata (Stratum3.1 and Stratum3.2; Stratum4.1 and Stratum4.2).
3. Third Stage Strata
Furthermore, to ensure the selected sample with six first-stage and second-stage
strata are representative of grantees with different patient sizes, we further split
six strata into several third-stage strata according to the patient size of a grantee.
First, we calculate the 33rd and 66th percentile of patient size using entire grantee
sample frame. Then the grantees with patient sizes over the 66th percentile are
defined as “Large” grantees, grantees with patient sizes below the 33rd percentile
are defined as “Small” grantees, and grantees with patient sizes between the 33rd
and 66th percentiles are defined as “Medium” grantees. In order to have the
minimum sample size be larger than 10 in each final stratum, some first-, second-,
and third-stage strata are collapsed due to small sample size.

c. Site Sample
Once the grantees are recruited, our recruiters will work with the grantee’s administration
staff to identify eligible sites using the eligibility criteria discussed in Health Center Site
Universe. Specifically, the following information will be collected from each
participating grantee on their sites (using the Grantee Recruitment Guidelines):
•
•
•

Number of sites serving each patient population (i.e., migrant/seasonal
farmworkers, homeless, public housing, and general community);
Address and contact information for each eligible site; and
Number of patients served in each eligible site overall and by type of patient
population.

If there are three or fewer sites serving a patient population type (i.e., migrant/seasonal
farmworkers, homeless, public housing, and general community), all of the sites will be
included in the sample unless the distance between sites is too great for one interviewer
to cover all of them. For grantees with more than three sites for a specific funding
program, we will use a PPS sampling method to select three sites out of all the sites that
are within a reasonable distance for one interviewer to cover. The number of patients for
the sites of a specific funding program will serve as the size measure in the PPS
sampling. The PPS selection is independent for each funding program in the grantees
with multiple funded programs. We will select three sites using the PPS sample using
PROC SURVEYSELECT in SAS.
d. Patient Sample
Because of the mobile nature of some of the target populations for this study, a random
sample of patients will be chosen for interviews as they enter the site and register with the
receptionist for services. The survey patients will be selected using onsite recruitment
procedures. Project staff will not be allowed to approach any of the health center’s
patients nor obtain any identifying information about a patient unless the selected patient
initiates such contact with field interviewer.
We will evenly allocate the target number of completed interviews for each funding
program to all the grantees serving a special population. For example, if we recruit 31
grantees that serve farmworkers (MHC), then 27 (826/31 is rounded to 27) interviews
will be completed for each grantee. If more than one site is selected for a grantee, then
the number of completed interviews will be distributed evenly to each site.
The procedures for patient selection and interviewing are:
1. Patient Registration: As each patient enters the site during the sample selection
period, the receptionist will register him or her to receive health services and
record a tally mark on the Patient Arrival and Selection Tracking Form (as shown
in Attachment 4). The receptionist will determine whether a patient is eligible for
the survey (i.e., had received services at least once in the past 12 months from one
of the four funding programs; patients under 18 must be accompanied by parent
or guardian).
2. Patient Selection: The receptionist will select patients as they enter and register
according to the detailed sample selection protocols to ensure the selection of a

random sample of patients. Specifically, the receptionist will select the first
eligible patient registered after the field interviewer informs the receptionist that
he or she is ready for the next interview. If the patient is eligible, the receptionist
will read a brief recruitment script to the patient (or to his or her parent or
guardian, for selected children) and give him or her a packet of information on the
Patient Survey. The receptionist will record the number of patients selected in the
Patient Arrival and Selection Tracking Form as shown in Attachment 4. The
Patient Arrival and Selection Tracking Form will be used in the survey for the
non-response adjustment in calculating analysis weights.
3. Patient/Interviewer Contact: If the selected patient is interested in participating
or has questions, he or she will approach the staffer, who will be waiting in a
designated area in the site. The staffer will take the participant to a designated,
private location at the site to begin the screening, informed consent, and interview
process. Plans for handling Spanish-speaking individuals are included in Section
B3f.
4. Disposition of Patient Logs: At the end of each day, the staffer will collect the
Patient Arrival and Selection Tracking Form from the receptionist. Collected
forms will be brought back at the end of data collection. The number of
completed patient interviews for each funding program will be monitored to
ensure that the sample size targets are being met for each site.
If a participating grantee in the study has more than one funding program, independent
patient samples will be selected for each funding program. If a site is chosen for multiple
funding programs, the receptionist at the site will be asked to track and select patients on
the interviewing visiting dates for all funding programs. The interviewer will screen
participating patients to determine patient population types (i.e., homeless,
migrant/seasonal farmworker, public housing, or general community) and will select
respondents based on a predetermined quota. Patients will be instructed to speak with one
of the interviewers waiting in the lobby. After the interviewer explains the study,
interested patients will be taken to a private location at the health center and administered
the survey.
3.

Methods to Maximize Response Rates and Deal with Nonresponse

Response rates for the study will be a function of success in two basic activities: (1)
identifying eligible patients and (2) obtaining those patients’ cooperation and time to
complete the interview. We will train site staff in using a patient recruiting script, train all
field interviewers on critical cooperation-gaining techniques, regularly debrief with data
collection staff, conduct in-person interviews, allow interviewers to schedule interviews
at a later time if necessary, make multiple attempts to reach and reschedule respondents
who miss appointments, and offer a $25-value remuneration to all participants. Given
those plans (which are discussed further below) an 80% cooperation rate is anticipated.
a. Training Site Staff
Site staff will determine whether patients are new or have had services previously,
therefore deeming them potentially eligible for selection. Prior to speaking with potential

respondents and using the patient tracking and selection form, site staff will have
adequate training to assist in obtaining high levels of patient cooperation.
The contractor is responsible for arranging and conducting site staff trainings. These
trainings will last approximately 1.5 hours and will be conducted via telephone with key
health center site staff and administrative staff at each site immediately before data
collection begins. It is anticipated that on average two project staff per site will attend.
Prior to the training self-study materials will be distributed for the site staff to review
which will describe the study and instruct staff how to create and modify an anonymous
roster of patients. In addition, the self-study materials will serve as a reference guide
during data collection with mock scenarios which demonstrate how to handle various
situations.
Training will be provided on the study purposes and procedures. Particular attention will
be paid to the role of the site staff in patient recruitment for the patient survey and as
recipients of referrals for mandatory reporting issues. In addition emphasis will be put on
the appropriate use of project-provided recruitment materials and the importance of
privacy. During data collection contract staff will stay in close contact with site staff to
ensure that project protocols are followed and assist with any data collection concerns
that may arise.
Training Field Interviewing Staff - A comprehensive multi-day training will be
provided to the field interviewing staff. They will be trained on the study purpose and
procedures, interview administration, and the protection of human subjects. Part of the
interviewer training will address in detail specific techniques for gaining cooperation and
averting and converting a respondent refusal. Reasons for refusals and barriers to
participation will be continually evaluated in light of the experience gained during the
data collection process.
b.

c. Regular Debriefings with Data Collection Staff
The project staff will regularly meet with data collection staff to discuss issues related to
data collection operations. Methods to enhance response rates will be a standard agenda
item at these meetings.
d. On-site Data Collection
When surveying a hard-to-reach population, such as health center patients, intervieweradministered modes yield higher response rates than self-administered modes.
Additionally, allowing the option to interview respondents in person at the site right after
selection will assist in maximizing cooperation and in turn response rates. Interviews
will therefore be conducted in-person via CAPI.
If a respondent is interested in participating but unable to complete the interview at the
site at that time the interviewer will have the option of scheduling an appointment at a
later time. Future appointments will take place either back at the site, in a conveniently
located library with a private room, or at the respondent’s home (if applicable). Multiple
attempts to reach respondents who do not appear for scheduled interviews will be a
standard protocol.

e. Offering Remuneration
Providing respondent remuneration during the interviewing phase of data collection
increases the likelihood that sample members will participate, particularly for the lowincome populations. Respondents will be provided with remuneration valued at $25 for
their participation.
In addition, a non-monetary benefit in the form of data analysis and summarization will
be provided to participating grantees with grantee-specific data on patient satisfaction,
behavior, and other characteristics in table format. Comparisons to results from all
participating grantees may also be provided. These data will provide the organization’s
management and board with useful information on patient characteristics, barriers to care,
and grantee-performance.
f. Bilingual Approach
The questionnaire and other respondent materials have been translated into Spanish,
which is likely to be the most commonly encountered second language in the
communities of interest. The bilingual approach, which involves Spanish translation and
interviewing, allows respondents with limited English skills to fully understand the nature
of their participation. Further, it makes it more likely that those respondents will
complete interviews because they will be given the choice of using the language with
which they feel more comfortable. As noted in Section A, for grantees that have a
substantial Hispanic population, an interviewer who is certified as bilingual will be
available. Bilingual interviewers will be available to complete interviews in Spanish or
English depending on respondent preference.
4.

Tests of Procedures and Methods to be Undertaken

The procedures and methods to be undertaken have been tested in various ways.
Cognitive interviewing was utilized to test and finalized the ease of use of the survey
instrument. In addition, all procedures and methods utilized are based on those
successfully implemented during the previously executed user surveys. Additional
details are supplied below.
Cognitive Interviewing
During the survey questionnaire development phase, two rounds of cognitive interviews
were conducted to finalize a questionnaire that was comprehensible, could be
administered within approximately 66 minutes and generated accurate data.
Methods from Past Studies
As mentioned in Section A, the data collection procedures and materials of this survey
were built upon the Health Care for the Homeless User/Visit Survey (HCH) conducted in
2003, the 1995 Community Health Center User/Visit Survey, and the 2002 Community
Health Center (2002 CHC) and National Health Service Corps Site User/Visit Survey.
All of these surveys achieved high response rates and were found to be easily
administered correctly by site and field staff.

5.

Individuals Consulted on Statistical Aspects and Individuals Collecting
and/or Analyzing Data

Names of individuals consulted on statistical aspects of study design along with their
affiliation and telephone numbers are provided below.
Name
Mr. Patrick Chen
Ms. Cynthia Augustine
Dr. Karol Krotki
Ms. Jody Green
Ms. Kristine Fahrney

Title
Sampling Task Leader
Analysis Task Leader
Senior Advisor
Project Director
Project Director

Telephone
919-541-6309
919-541-6154
202-729-2485
919-485-2710
919-485-5531

In addition to these statisticians and survey design experts, HRSA staff have also
reviewed and approved the statistical aspect of the study.
Other Contractors’ Staff Responsible for Conducting the Study
BPHC is being assisted through subcontracted activities to RTI International. Principal
professional staffs from RTI assigned to the study (but not list above) follow:
Name
Mr. Tim Flanigan
Ms. Ann Burke
Mr. Joe Nofziger
Ms. Carrie Borst

C.

Title
Data Collection Task Leader
Data Collection Task Leader
Programmer
Project Manager

Telephone
919-485-7743
312-456-5244
919-541-6650
919-541-6988

Overview of Analysis Topics and Survey Items

The survey data elements cover general topics such as demographics, current health,
access to care and services, substance use and mental health, and income and insurance.
Most items apply to all sample members. However, some sections are only applicable to
a subset of sample members (i.e., questions on pregnancy are only asked of women of
child-bearing age).
Some analysis topics for the survey instrument were outlined in the research and issues
section presented earlier in Section A2. Attachment 3 displays the proposed data
elements for the survey instrument. The data elements are presented as a list of items
arranged by topics.