Incentives

ATTACHMENT 6.
INCENTIVES

6-1

ATTACHMENT 6.
INCENTIVES
Background
Online survey research has grown rapidly, and over the past decade two types of online
panels have gained popularity – probability based panels, where panel participants have a
known, nonzero probability of selection in order to produce population estimates, and
nonprobability panels, such as volunteer panels which use advertisements, e-mails and a variety
of other methods to recruit participants. One example of a nonprobability based sampling method
is river sampling – an approach in which respondents are recruited to complete a survey through
pop-up invitations on a website, for example, but initially have to answer a few qualifying
questions. Sometimes, after completing the survey, such respondents are offered the opportunity
to join an online panel.
One of the biggest concerns for online panels that intend to represent the general
population is coverage error. The Pew Research Center estimates of Internet access indicate that
roughly fifteen percent of the U.S. adult population does not use the Internet on a regular basis
(Zickuhr, 2013). Probability-based panels try to minimize undercoverage by using traditional
methods to recruit participants (good-quality sample frames and face-to-face or telephone
contact) and supplying Internet connection to panel members who are not online.
The inferential paradigm assumes 100% response rate from a probability-based sample,
so findings can be generalized to the target population. To the extent that such response rate is
not achieved, concerns about the presence and magnitude of nonresponse bias arise; thus, panel
participation and maintenance are important concerns. In order to boost enrollment and
participation over time, many panels employ incentives, especially in a combination of
enrollment incentive and participation incentive.
Use of Incentives in Panel Surveys
The recruitment strategy for Web panels typically involves some combination of various
motivators for participation, such as incentives, importance of self-expression, fun, ease of panel
participation, or social comparison (Baker et al., 2010). Incentives have been viewed as the
primary motive for joining a panel—for example, Poynter and Comley (2003) report a mix of
6-2

motivators, with incentives having the most impact (59%), followed by curiosity (42%),
enjoyment in doing surveys (40%), and importance of expressing own views (28%).
The mechanisms that evoke higher participation when incentives are used are unclear.
Two competing theories suggest that incentives may be construed as either a token of
appreciation (social exchange theory – see Dillman, 2000), compensation for one’s time and
effort (economic exchange theory – see Biner and Kidd, 1994), or the subjective weight a
sampled person puts on various factors when the survey request is made (leverage-salience
theory – see Groves, Singer and Corning, 2000). Which mechanism is dominant may not make a
difference in cross-sectional surveys, but would likely affect cooperation in panel surveys, when
the decision to participate in the first data collection (wave) of the survey is, to a certain extent, a
commitment to take part in subsequent data collections and the experience in the first wave is
likely to be the most influential factor on future decisions to participate (Singer et al., 1998).
Longitudinal or panel surveys often use incentives to build initial rapport with the panel
respondents as participation in the baseline wave usually sets the retention rate for the life of the
panel (Singer et al., 1998, Baker et al., 2010). That is why somewhat larger incentives in the first
wave of data collection are often recommended (Singer et al., 1998). For example, in an
incentive experiment on Wave 1 of the 1996 Survey of Income and Program Participation (SIPP,
U.S. Census Bureau), James (1997) found that the $20 prepaid incentive significantly lowered
nonresponse rates in Waves 1-3 compared to both the $10 prepaid and the $0 conditions. Mack
et al. (1998), examining cumulative response through Wave 6, found that an incentive of $20
reduced household, person, and item (gross wages) nonresponse rates in the initial interview and
that cumulative household nonresponse rates remained significantly lower at Wave 6 (24.8
percent in the $20 group vs. 27.6 percent in the $0 incentive group, and 26.7 percent in the $10
group), even though no further incentive payments were made.
Many federally-sponsored longitudinal surveys offer incentives to gain initial cooperation
and minimize attrition. For example, in order to improve response rates, reduce the number of
contacts required to gain cooperation, and address respondent concerns about interview burden,
the National Survey of Child and Adolescent Well-Being (NSCAW, Administration for Children
and Families) in 2002 doubled the incentive offered to respondents from $25 to $50. The Early
Childhood Longitudinal Study-Birth Cohort (ECLS-B, U.S. Department of Education) offered
6-3

parent participants $50 and a children’s book for the first wave and $30 and a children’s book for
subsequent waves of data collection. Over rounds 1 through 10 of the National Longitudinal
Survey of Youth 1997 (NLSY97, Bureau of Labor Statistics) cohort, incentives offered to
respondents ranged from $10 to $50 in an attempt to minimize attrition across waves of data
collection.
The U.S. Census Bureau has also experimented with and begun offering incentives for
several of its longitudinal panel surveys, including SIPP and the Survey of Program Dynamics
(SPD). SIPP has conducted several multi-wave incentive studies, most recently with their 2008
panel, comparing results of $10, $20, and $40 incentive amounts to those of the $0 control group.
The study has examined response rate outcomes in various subgroups of interest (e.g., the
poverty stratum), use of targeted incentives for non-interview cases, and the impact of base wave
incentives on participation in later waves of data collection. Overall, the results suggest that $20
incentives increase response rates and also improve the conversion rate for non-interview cases.
Incentives may also have an additional impact on response rates for households in the poverty
stratum and significantly reduce item nonresponse rates (see Creighton et al. (2007); Clark and
Mack, (2009)). Similarly, SPD has conducted four incentive studies, testing $20, $40, $50, and
$100 amounts in an effort to increase cooperation among poverty households and
nonrespondents and to minimize attrition in subsequent waves of the study. Incentives were
found to have a positive impact on both response and attrition rates; most recently, the fourth
incentive study found that the average interview rate greatly increased with the use of incentives
(Creighton et al., 2007).
Need for Incentives in Self-administered Modes
In addition to the need to engage sampled persons repeatedly and over a longer interval
of time, the need for incentives in Internet panels is exacerbated by the self-administered mode
of data collection. Self-administered modes have historically achieved lower response rates than
classic interviewer-administered modes. This is believed to be largely due to the lack of an
interviewer to gain initial cooperation from a sample member who may be reluctant to
participate. The use of incentives is one of the common remedies used to counteract low
response rates in self-administered surveys (Armstrong 1975; Church 1993; Fox, Crask and Kim
1988; Dillman 2007; Heberlein and Baumgartner 1978; Kanuk and Berenson 1975; Levine and
6-4

Gordon 1958; Linsky 1975; Yu and Cooper 1983). An additional benefit of using incentives is
the potential to decrease nonresponse bias by including sample persons with low topic interest
(e.g., Baumgartner and Rathbun, 1997; Groves, Singer and Corning, 2000).
Prepaid incentives can be used during the recruitment stage for an online panel if sampled
members are contacted in person or via mail. Several large federal surveys have experimented
with a mail screener that includes a small prepaid incentive (typically, $2 or $5) and have
reported on their effectiveness in increasing screener response rates (e.g., National Household
Education Survey, U.S. Department of Education (as part of the transition from a telephone to a
mail mode of administration); The National Survey of Early Care and Education, Administration
for Children and Families). Similarly, to minimize screening cost and time, the proposed study
design will include a mail screener questionnaire sent to all sampled households to determine if
there is an eligible adult tobacco smoker. A $2 prepaid incentive will be enclosed with this initial
mailing to maximize screener response rates.
At the panel enrollment stage and during the life of an Internet panel, Web participants
can be offered various types of compensation - cash, points that can be redeemed for various
goods, sweepstakes, or instant-win games. Incentives in Internet panels are typically contingent
on survey completion (Baker et al., 2010).
Incentive Types and Amounts in Panel Surveys
Panel surveys differ greatly in the design of incentive packages that are offered to
sampled persons. The ANES 2008-2009 Panel Study, an RDD Internet panel recruited via
telephone, offered $10 per month to complete Web surveys for 30 minutes each month. Sampled
persons who reported that they could not use the Internet at home, at work, or at school to
complete surveys were offered free Internet access for the duration of the panel study through
MSN TV2. At the recruitment stage, each sampled telephone number for which an address match
was obtained was mailed an advance letter with a $2 bill enclosed. The letter offered $10 more to
complete a short telephone interview and $10 per month for each monthly survey (DeBell,
Krosnick and Lupia, 2010). Initial refusals were sent a follow-up letter with a $5 bill and several
days later were called by an experienced interviewer.
The Health and Retirement Study, a panel study investigating health, wealth, and
retirement decisions among people 50 and older, offers prepaid incentives of $50–$80 per
6-5

married couple (the amount varies based on interview type), and as much as $100 during the
refusal conversion stage. This incentive design is based on a series of incentive experiments
(e.g., Lengacher et al., 1995; Rogers, 2002) focused on maximization of response rates and
prevention of attrition and comparing the effectiveness of various amounts of monetary
incentives to other incentive types, such as charitable giving (e.g., Lengacher et al., 1995).
The National Longitudinal Survey of Youth (NLSY), which has used financial incentives
since 1979, offered a bonus incentive of $100-$150 to households (amount dependent on
household size) in 1998 in order to address declining response rates. From 2002, the NLSY79
employed an alternative strategy where the base rate for an interview rose from $20 to $40 for
respondents who were called by the survey organization and $80 for those who called in
themselves to do the interview. Gifts in-kind and gift cards or food up to $20 per household are
also offered in some circumstances.
Proposed Incentive Structure for the FDA Tobacco User Panel
Consistent with the above cited studies, we will use a $2 prepaid incentive that will be
mailed to the sampled households along with a prenotification letter and a one-page mail
screener. Consistent with the amounts offered by national panels such as the NLSY ($40) and
HRS ($40), and recent RTI experiments on the Baccalaureate and Beyond Longitudinal Study,
eligible sample members will be offered $35 at enrollment. This amount is commensurate with
the respondent burden at the recruitment stage due to activities such as a 10-minute screening, a
10-minute enrollment survey, respondent training on accessing and navigating the panel website,
and completing the 10-minute baseline survey.
For each experimental or observational study, panel participants will be paid a $15
promised incentive. The amount was largely informed by focus group participants and is
consistent with current national panels – for example, the Panel Study of Income Dynamics
(PSID) incentive practice is to pay $1 per interview minute, sent via check or money order upon
survey completion. We believe this amount is needed to ensure high response rates for each
study.
In order to maintain contact with the panel members between studies (only 3 per year),
we are planning other forms of contact, including email, regular mail, text, or automated

6-6

telephone communications. We also plan to update panel member contact information collected
in the baseline survey during planned experimental or observational studies.

6-7