Justification for incentives

Evaluation of the Administration for Children and Families Responsible Fatherhood, Marriage and Family Strengthening Grants for Incarcerated and Reentering Fathers and Their Partners

Justification of Increase in Incentive Payments for 18 Month and 34 Month Data Collection

Our baseline data collection, which began in 2007, included 2010 incarcerated men and 1840 of their partners. Our overall response rate for the 9 month follow up is over 75%. However, as we moved into the 18 month interviews we began to see a slight decay in the response rate and we believe, based on previous research and experience, that the increased incentive from $40 per interview to $75 per interview will help us maintain the 75 percent response rate through the end of the data collection (34 months).

There are several justifications for proposing to increase the incentives in the 18 and 34 month follow up waves.

1. Response rates and attrition rates between waves are a big concern. As is well documented in other surveys, it becomes increasing difficult to locate individuals over time and refusals increase. Increased incentives should help us to maintain our response rates and reduce attrition, which will improve the precision of impact estimates.

2. In addition to the overall response rate, we are concerned about non-response bias that is possible if specific groups are less likely to respond. We are particularly concerned with response rates for men who are no longer incarcerated, and from partners of incarcerated men who are no longer in relationships with those men. We believe the increased incentive will help to reduce non-response bias in these groups, and therefore improve the accuracy and quality of impact estimates.

3. For this survey the male target population is more difficult to locate at later waves because more of the men have been released from prison into the community. Housing post-release is often unstable and increases the challenge and cost of locating respondents. Increased incentives will help reduce refusal rates for the respondents that we are able to locate.

4. We are re-contacting people who have not responded to the wave immediately prior, and there is evidence that higher incentives are helpful for these contacts. However, because there is likely communication between respondents, especially the men who remain incarcerated, we do not want to introduce inequity in amounts offered to respondents.

5. Lastly, as we move further from the baseline data collection, more of the couples will have terminated their relationship. This appears to increase refusal rates. The increased incentive will provide some counterbalance to the respondents’ lack of interest in discussing their failed relationship.

As a reminder, the reasons to offer incentives for this data collection were supported by the final report of the Symposium on Providing Incentives to Survey Respondents (1992) that was sponsored jointly by OMB and the Council of Professional Associations on Federal Statistics. The report recommended that OMB “seriously consider the use of incentives” for surveys like this one that have the following characteristics:

• Our survey targets difficult respondent populations (incarcerated men and their partners).

• Gatekeeper relatives affect respondent access (the incarcerated male provides access to his partner and post-release the partner is likely to have the best information on the formerly-incarcerated parents location).

• For the partner and for the released parent participation may require out-of-pocket costs. For example, respondents may not want to discuss their relationship with their partner with their minor children or other household members present.

In none of our sites will incentives be paid to the respondent who is incarcerated. This is the policy of the prison facilities.

This increase in incentive does not increase the overall cost of the evaluation. We anticipate that this investment in incentives will be offset by other data collection costs relate to locating, mileage, and refusal conversion.

We reviewed a number of studies before proposing an increase. The experience of the Survey of Program Dynamics Studies 3 and 4 (Creighton, King and Martin , 2007) ¹ was particularly influential. In those studies the greater incentive amount was influential in increasing the response rates, both overall and for specific groups. We believe the increased incentive for the later waves will improve data quality in two waves. By increasing response rates and sample size it will improve the likelihood of precise impact estimates.

We do not believe that the payment of incentives will introduce nonresponse bias in this survey. Rather it is likely that increased incentives may reduce nonresponse bias. Respondents to this survey differ on a lot of characteristics, but income is not one of them. So it is unlikely that the offer of a $75 incentive is likely to be more persuasive to some respondents than to others, as may happen when the economic conditions of respondents vary widely. What is more likely is that the additional incentive will be more persuasive to all respondents (even those men who themselves do not receive an incentive because they remain incarcerated). Additionally, unlike some impact studies, the characteristics of respondents were extensively measured at baseline. This will allow us to examine and adjust for any possible non-response bias (from any source). This type of analysis is already included in the evaluation designs. While there is not a substantial set of literature which examines the effect of incentives on nonresponse bias, one study is particularly germane. In the pilot study for the German panel study on families and relationships, the effect of incentives on the partners response rates was examined (Castiglioni and Pforr, 2007) and the authors concluded that the use of incentives can help reduce both non-response and non-response bias in couples surveys.² As with increasing response rates, reducing nonresponse bias will contribute to increased accuracy of impact estimates.

In summary, the project’s payment schedule represents appropriate compensation for respondent burden associated with completing the interview during the data collection period. This payment schedule will also ensure the collection of useful, cost-effective, and policy-relevant data for the MFS-IP evaluation.

1 Creighton, Kathleen, Karen King, and Elizabeth Martin, The Use of Monetary Incentives in Census Bureau Longitudinal Surveys, U.S.. Cenus Bureau, Washington D.C., 2007. http://www.census.gov/srd/papers/pdf/rsm2007-02.pdf

2 Castiglioni, Laura and Klass Pforr, The Effect of Incentives in Rreducing Non-response Bias in Multi-actor Survey, Mannheimer Center for European Social Research (MZES), University of Mannheim, Paper presented at the 2^nd ESRA Conference, Prague, 26^th - 29^th June 2007.