Early Bird Incentive Clarification

Rationale for Early Bird Incentive Use for EFECT (0910-0788)

In a review of response rate of government surveys (Lu, 2002), the mean response rate by type of study was 84.7% for studies using in-person interviews, 78.2% for mail surveys, 80.2% for telephone interviews, and 86.6% for multi-modal studies. As shown in Table 1, the current response rate for the EFECT surveys to date among those who are eligible has been lower than the multi-modal study average. EFECT’s sample is largely composed of racial and ethnic minority youth, and incentives have been found to be an effective method for recruiting vulnerable and hard to research populations such as racial and ethnic minorities (Yancey, Ortega, & Kumanyika, 2006).

Table 1: Response rates among those eligible, EFECT surveys from baseline to Follow-up 2

EFECT Response Rates	Total Eligible	Total Completed Surveys	% Eligible who Completed
Baseline	2,740	2,194	80.1%
Follow-up 1	3,500	2,403	68.7%
Follow-up 2	2,704	2,051	75.9%

In Table 2, those who participated in both baseline and Follow-up 1 (longitudinal sample) are compared with those who participated in the baseline, but not Follow-up 1 (drop-off sample). While the two groups were similar demographically, the longitudinal sample had larger proportions of nonsusceptible never-smokers and fewer experimenters than the drop-off sample. Significantly more of the drop-off sample also reported having ever tried cigarettes, cigar products, or ENDS products and past 30-day use of cigar products, hookah, and ENDS. This difference in retention by tobacco susceptibility and use is a common limitation for longitudinal surveys on tobacco use. By including an incentive for responding within a specific amount of time, we expect to increase overall response rates and therefore reduce this potential for bias. (Follow-up 2 data are still being analyzed and therefore are not available for this comparison.)

Table 2. Demographic and Psychographic Characteristics by Longitudinal and Drop-Off Samples, Weighted

Characteristics	Longitudinal	Drop-Off
Age
12–13	20.9%	19.3%
14–15	38.5%	39.0%
16–17	40.6%	41.7%
Gender
Female	62.0%	59.3%
Male	37.9%	40.7%
Race/Ethnicity
White, non-Hispanic	7.8%	9.2%
Black, non-Hispanic	55.4%	52.5%
Hispanic	24.8%	26.2%
Other, non-Hispanic	11.9%	12.0%
Smoking Susceptibility and Use***
Never smoker, not susceptible	62.8%	53.3%
Never smoker, susceptible	17.7%	17.6%
Experimenter	17.9%	25.2%
Current or former smoker	1.6%	3.9%

***p < 0.001

Monetary incentives can significantly increase response rates in cross-sectional surveys and reduce attrition in longitudinal surveys with adults and youth (e.g., Abreu & Winters, 1999; Castiglioni, Pforr, & Krieger, 2008; Gajic, Cameron, & Hurley, 2012; Jäckle & Lynn, 2008; Shettle & Mooney, 1999; Singer, 2002). By increasing retention among the longitudinal sample, we will not need to recruit as many new participants resulting in future reduced burden to the public related to screening as many potential participants.

As noted in Supporting Statement Part A, several studies have shown that the use of early bird incentives, an additional monetary amount for completing a survey in a certain time period, can improve response rates. In one study, individuals who received an early bird incentive were 1.8 times more likely to complete the survey within the first 7 days of data collection and were 1.69 times more likely to ever complete the survey (LeClere, Plummer, Vanicek, Amaya & Carris, 2012). 

ExPECTT (OMB #0910-0753), the evaluation of the FDA’s public education campaign for tobacco use among youth (The Real Cost), promised an additional $5 incentive to youth participants who completed the survey online before the specified early bird date. In this study, 85% of Follow-up 1 participants, 81% of the Follow-up 2 participants, 79% of the Follow-up 3 participants, and 80% of the Follow-up 4 participants completed the survey during the early bird period. This method has also been used for the RESPECT (OMB #0910-0808) and RuSTEC (OMB #0910-0753) evaluation studies. RESPECT has been using early bird incentives throughout among its sexual and gender minority sample and RuSTEC among rural male youth.

References

Abreu DA, & Winters F. Using monetary incentives to reduce attrition in the survey of income and program participation. In: U.S. Census Bureau. Proceedings of the Survey Research Methods, 1999: 533-538.

Castiglioni L, Pforr K, Krieger U. The effect of incentives on response rates and panel attrition: Results of a controlled experiment. Surv Res Methods. 2008; 2(3):151-158.

Gajic A, Cameron D, Hurley J. The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. The European Journal of Health Economics : HEPAC. 2012;13(6):789-99. 

Jäckle A, & Lynn P. Offre de primes d’encouragement aux répondants dans une enquête par panel multimodes: effets cumulatifs sur la non-réponse et le biais. Techniques d’enquête. 2008; 34(1): 115-130.

LeClere, F., Plumme, S., Vanicek, J., Amaya, A., & Carris, K. (2012). Household early bird incentives: leveraging family influence to improve household response rates. American Statistical Association Joint Statistical Meetings, Section on Survey Research, 4156 - 4165

Lu, R. (2002). Response Rates Achieved in Government Surveys: Results from an OMB Study. Federal Committee on Statistical Methodology Working Paper #35. http://www.amstat.org/sections/srms/proceedings/y2003/Files/JSM2003-000865.pdf Published 2002. Accessed January 29, 2016.

Shettle C, & Mooney G. Monetary incentives in US government surveys. J Off Stat. 1999;15(2): 231.

Singer E. The use of incentives to reduce nonresponse in household surveys. In: Wiley, ed. Survey nonresponse. 2002; 51:163-177.

Yancey AK., Ortega AN, & Kumanyika, SK. Effective recruitment and retention of minority research participants. Annu Rev Publ Health; 2006;27,1-28.

2