POD Semi-Structured Interviews with Treatment Group Subjects

BENEFICIARY INTERVIEW MATERIALS

1

The process study team will interview a total of 144 beneficiaries (9 in each state across all
eight demonstration sites, per round) in the second and third rounds of data collection to gather
information about their experiences participating in POD. We will include beneficiaries from
both treatment groups and from key subgroups of interest (for example, beneficiaries who
requested to withdraw; low earning offset users; high earning offset users; or control group
members). We organized the topic areas for the beneficiary interviews by evaluation focus area,
and provide a list of topics we will cover during the semi-structured telephone interviews. We
summarize the topics in Table D.1. Before each round of data collection, the task leader will
create semi-structured interview guides from the interview topics relevant to the POD study
subjects’ group assignment and the site visit round.
At the start of the interview, the Mathematica interviewer will explain Mathematica’s role
evaluating the POD demonstration; the purpose of the interview; and that the individual will
receive a $25 non-cash incentive for participating in the call. The interviewer will also convey
that participation in the interview is voluntary and that the individual’s decision to participate or
not participate will have no effect on SSDI benefits or participation in the POD demonstration.
The interviewer will also emphasize that the beneficiary can stop the interview at any time
without being penalized or disadvantaged in any way. We provide the script interviewers will
use at the start of the interview below in Appendix D.1.

2

APPENDIX D.1
INTRODUCTION FOR BENEFICIARY INTERVIEWS: ROUNDS 2-3

Hi, my name is _______, and I work for a company called Mathematica Policy Research.
I am contacting you today because we are doing a study of the Promoting Opportunity
Demonstration, known as POD, for the Social Security Administration (SSA). My understanding
is that you signed up to participate in POD in YYYY. We are calling beneficiaries like you to learn
more about your experiences with POD.
What we learn from you today will help SSA do a better job of serving individuals with disabilities
through its Social Security Disability Insurance (SSDI) benefit program in the future.
I expect that our discussion will last about 45 minutes.
Your participation in this call is voluntary. Whether or not you participate will have no effect on
your SSDI benefits or your participation in POD. You can stop the interview at any time and you
will not be penalized in any way. You also do not have to answer any question you don’t feel
comfortable answering. If you like, we can also break our discussion into two parts, and schedule
a second appointment to finish the discussion at a later time. You will be sent a $25 gift card for
completing the discussion.
Our discussion today will be audio recorded to ensure that our meeting notes are accurate. Only
members of our team will listen to the recording and we will destroy it at the end of the study. This
is also a good time to emphasize that all information you share with us will remain confidential to
the fullest extent possible by law, meaning that your information will be kept as private as possible.
We may need to notify someone if keeping that information confidential could harm you or
someone else. None of our reports will include any information that identifies you. Your responses
today will be combined with the responses collected from about 140 other beneficiaries who
participate in POD. Your responses will only be used for the purpose of this study and will not
impact your DI benefits or your participation in the POD study in any way.
I also want to let you know that I am not an expert on Social Security disability, Medicare, or
Medicaid. So I cannot answer questions or give advice about SSDI benefits or your personal
circumstances.
Are you available and willing to participate in this discussion?
If “Yes,” do you have any questions before we begin?
If “No,” Okay, let’s get started.

3

APPENDIX D.2
INTERVIEW TOPICS FOR BENEFICIARY INTERVIEWS: ROUNDS 2-3

Table D.1. Beneficiary interview topics addressing each research question, by evaluation focus
.

Interview Topics

I. Respondent background and experience
General background information

Duration of SSDI benefit receipt, any SSI benefit receipt, any dependent auxiliary beneficiaries
Summary of employment history, previous SSDI benefit suspension for work
Participation in previous demonstrations (SSA or otherwise)
Approximate date of enrollment in POD and whether or not the beneficiary disenrolled

II. Program Environmenta–community context and the counterfactual
B. Service environment
How do the general employment
environment and other state
features affect benefit offset use?

The role of the general employment environment in beneficiary employment decisions

How prevalent is engagement in the
Ticket to Work (TTW) Program and
VR?

Engagement with VR or TTW before POD
Referral source for VR or TTW services

How did state VR agencies and
other TTW providers previously
serve beneficiaries before POD, and
did this change during the
demonstration period?

Engagement with VR or Ticket to Work after enrollment in POD and referral source
The role of POD in the decision to seek services
Beneficiaries’ perspectives on the accessibility and timeliness of services and success in achieving employment goals
Beneficiaries’ perspectives on unmet service needs

III. POD Recruitment, enrollment, and RA
A. Outreach and recruitment
What types of information did DI
beneficiaries receive about POD?

Initial exposure to POD outreach efforts and initial impressions
Information received about POD through VR, SSA field offices, local WIPA providers, and other entities

How successful were outreach
efforts in helping beneficiaries
understand POD?

Ability to have questions answered during recruitment
Initial points of confusion about POD and if/how these points were clarified
Comparison of initial impressions to later experience in POD
POD participants’ understanding of POD offset rules

How did outreach and enrollment
differ for concurrent beneficiaries
(those who receive SSI and DI)?

The role of SSI receipt in the decision to participate in POD
The role of SSI receipt in POD enrollment

5

.

Interview Topics

B. Enrollment
What were beneficiaries’
experiences with enrollment?

Ability to have questions answered during enrollment
Experience with POD enrollment process
Understanding of random assignment group

VI. Operations and fidelity to the program design
Is the integrity of random
assignment being maintained?

Beneficiary exposure to information about POD treatment or control groups other than the group to which the beneficiary
was assigned
The effect of information on other POD treatment or control groups on beneficiaries’ understanding of their POD group or
attitudes towards POD

VII. POD service delivery
Has the implementation contractor
clearly defined and correctly
provided the intervention?

Beneficiaries’ description of POD in their own words
Beneficiary understanding of when and for how long benefits are effected by employment for beneficiaries in their assigned
treatment or control group
Beneficiaries’’ comfort level with their current understanding of POD
Beneficiary sources of information on POD
Beneficiaries’ ability to have questions about POD answered
How and which service providers have helped participants to understand relevant SSA notices and file
reconsiderations/appeals/waivers of overpayments

What are beneficiaries’ experiences
with POD counselors?

The frequency and nature of beneficiary contact with POD counselors
Role of counselors in explaining POD and setting and helping to achieve employment goals
Changes in beneficiaries’ work goals changed since enrolling in POD

How do state VR agencies collect
earnings and IRWE information from
individuals assigned to a treatment
group?

Beneficiaries’ experiences reporting earnings and IRWE information

How timely are benefit adjustments?

Beneficiary experiences with the timeliness of benefit adjustment
The impact of delays in benefit adjustment

How prevalent are improper
payments?

Beneficiary experiences with overpayments or underpayments while in POD
Beneficiary understanding of the cause of past improper payments
The impact of improper payments on beneficiaries’ employment and attitudes towards POD

6

.
Are monthly earnings and IRWEs
reported on time?

Interview Topics
The frequency and methods beneficiaries use to report earnings and IRWEs
Beneficiary receipt of assistance in reporting earnings and IRWEs
Effect of outreach efforts or reminders on monthly earnings reporting

VIII. Administration, coordination, and inter-agency relationships
How do the organizations
implementing POD communicate
and coordinate with one another?

Beneficiary experience interacting with multiple organizations implementing POD or delivering disability-related services
and cross-agency communication

POD participation patterns and experience
Do beneficiaries favorably perceive
the interventions?

Beneficiary highlights of POD and perceived shortcomings of POD
The effect of POD on beneficiaries’ quality of life
Attitudes about employment and the evolution of employment attitudes over the course of the demonstration

What are most prevalent reasons for
volunteering or not volunteering?

Beneficiaries’ motivations to participate in POD
Beneficiaries’ hesitations in participating in POD and if/how they were overcome

What are the events causing POD
withdrawals, if applicable?

Beneficiary awareness of the option to revert to current law
The timing of disenrollment from POD

What are the participants’ reasons
for withdrawing?

Factors affecting beneficiary decisions to remain in POD and revert to current law

IX. Successes and challenges
What aspects of recruitment and
enrollment were successful? Less
successful?

Factors that contributed to enrollment in POD
Obstacles that deterred from POD enrollment

What facilitated timely earnings and
IRWE reporting?

Facilitators of timely earnings and IRWE reporting
Ability to replicate facilitators in other sites

What are the obstacles preventing
timely earnings and IRWE
reporting?

Obstacles preventing timely earnings and IRWE reporting
How obstacles were addressed
Success in addressing obstacles

What facilitated benefit offset use?

Facilitators of employment and benefit offset use

What are the obstacles preventing
benefit offset use, if any?

Obstacles preventing substantial employment and benefit offset use
How obstacles were addressed
Success in addressing obstacles

7

.

Interview Topics

X. Outcomes and lessons learned
How does POD change the
participant’s use of other
government programs or income
supports?

Increase or decrease in POD participant use of other government programs or income supports
Which programs/supports, and how participant use changed
Unexpected consequences of participating in POD
Factors that contributed to these consequences

What lessons learned can be shared
with other sites gearing up to
implement a similar program?

Beneficiary suggestions for improvement

aSome

of this information will be gathered prior to the site visit.

8