Download:
pdf |
pdfSITE VISIT QUALITATIVE INTERVIEW MATERIALS
ASSESSMENT OF IMPLEMENTATION FIDELITY
A. Purpose of assessment
The POD process study will include a thorough assessment of implementation quality and fidelity to the program design. This
assessment will be used to: (1) describe the fidelity with which staff implements the intervention in all eight sites; (2) assess the extent
to which implementation at each site is consistent and the services available are uniform within and across all local sites; and (3) identify
areas where additional resources could support the implementation. We will inform our assessment by: the statement of work for the
implementation contractor which specifies the basic structure SSA intends to implement for POD; the implementation contractor’s
design report which will serve as the blueprint for the implementation of the demonstration; and the POD implementation manual
developed by the implementation contractor for POD benefits counselors and VR agency staff to use as a resource. We will use these
three sources as our benchmark against which to assess implementation quality. The assessment will consider overall implementation
as well as variation in implementation across each state.
We will assess two main aspects of implementation: (1) delivery of work incentives counseling and other participant supports; and
(2) reporting of monthly earnings and IRWE information to SSA and adjustment of DI benefits. Delivery of work incentives counseling
will include onboarding of new treatment subjects; development of the benefits summary and work incentives plan; provision of ongoing
work incentives counseling; and coordination of participant transitions back to current program rules at the end of their participation in
POD. Reporting of monthly earnings and IRWE information to SSA will include: the collection and maintenance of participant earnings
and impairment-related work expenses (IRWEs) information; processing of earnings records; adjustment of benefits under POD rules;
and the annual reconciliation of benefits. We will assess specific measures under each category to determine the extent to which site
staff are implementing them as planned, and if staff needs additional technical assistance to support implementation. For example, under
provision of POD work incentives counseling, we will assess the frequency and intensity of counselors’ contacts with treatment subjects
in each site and the extent to which counselors connect participants to SSA work incentive programs and other resources.
B. Analysis
We will assess implementation quality and fidelity drawing on programmatic data the Implementation Contractor provides, and
enrollment information we capture in Mathematica’s SMS. To conduct the fidelity assessment, we will populate a set of tables organized
by subject at three points in time to assess operations at each site and across all sites combined. Tables C.1a and C.1b are illustrative
examples of the tables showing a partial list of measures. We will assess implementation using the full range of programmatic and
enrollment data (e.g., using percentages or averages to assess the level of fidelity achieved) to identify which sites have higher and
comparatively lower levels of fidelity on each set of measures. Demonstration sites with lower than average levels of fidelity on each
set of measures may suggest a need for additional TA, training, or other resources.
3
Table C.1.a Illustrative example of fidelity assessment table: Indicators of implementation context and
fidelity of work incentives counseling in YYYY
Fidelity
Measure
Site 1
Onboarding of new treatment subjects
Average amount of time to first WIC contact attempt
x
Percent of subjects reached by a WIC
Percent of subjects reached by a WIC who opt out of
counseling services
x
x
Develop benefits summary and analyses and work incentives plan
x
Percent of clients with benefits planning query before BS&A
Percent of all clients with a BS&A
x
Percent of employed clients with a BS&A
Percent of clients with an employment goal with a BS&A
Percent of non-working clients without employment
goals with a BS&A
Percent of all clients with a WIP
x
Percent of employed clients with a WIP
Percent of clients with an employment goal with a WIP
Percent of non-working clients without employment
goals with a WIP
Average duration between WIP delivery and next contact
Deliver ongoing work incentives counseling
Average number of contacts per WIC client last quarter
Average number of e-mail contacts per client
Average number of phone or in-person contacts per
client
Average number of employment-related referrals last
quarter
x
x
Average number of referrals to EN
Average number of referrals to VR
4
Site 2
Site 3
Site 4
Site 5
Site 6
Site 7
Site 8
All Sites
Combined
Average number of referrals to AJC
Percent with benefit suspense who received counseling
x
within one month of initial suspense
AJC = American Job Center; BS&A = Benfits Summary and Analysis; EN = Employment Network; WIC = work incentives counselor; WIP = work incentives plan; VR
= vocational rehabilitation
Table C.1.b Illustrative example of fidelity assessment table: Indicators of implementation context and
fidelity of reporting monthly earnings to SSA in July YYYY
Fidelity
Measure
Earnings Reporting
Percent of all treatment subjects known to have earnings
over the POD threshold in July YYYY
Percent of treatment subjects known to have earnings over
POD threshold who reported earnings timely for July YYYY
Percent of treatment subjects who reported IRWEs in July
YYYY that were not approved
Percent of treatment subjects who reported IRWEs in July
YYYY that were approved
Percent of treatment subjects known to have a higher IRWE
threshold who reported timely for July YYYY
Site 1
x
x
x
x
Reporting Mode for Reports Made in July YYYY
Percent reported electronically
Percent reported by mail or fax
Percent reported in person
Earnings Record Processing
Average processing time for earnings records statused as
“Complete” in July YYYY
Percent of submitted earnings records submitted in July
YYYY requiring follow-up
Percent of records reviewed in July YYYY that fail QC
review
Average time to complete records that failed QC review in
July YYYY
x
x
x
x
5
Site 2
Site 3
Site 4
Site 5
Site 6
Site 7
Site 8
All Sites
Combined
Percent of treatment subjects known to be over the POD
x
threshold with complete records submitted to SSA timely
for July YYYY
IRWE = impairment related work expese; QC = quality control; POD = Promoting Opportunity Demonstration; SSA = Social Security Administration;
6
RESEARCH TOPICS BY ROUND OF PROCESS DATA COLLECTION
Table C.2. Research topics by round of data collection
Round 1
(Winter 2017/2018):
Early implementation
Round 2
(Fall 2018):
Recruitment and RA
Round 3
(Fall 2019):
One year after RA
Round 4
(Winter 2020/2021):
Two years after RA
Program environment
I
.
U
U
Planning and early implementation
I
.
.
.
POD outreach efforts
I
U
.
.
POD recruitment, enrollment, and RA
I
U
.
.
Development of the POD infrastructure
I
U
.
.
Data systems and use of the MIS
I
U
U
U
Operations and fidelity to the program design
I
U
U
U
POD service delivery
I
U
U
U
Administration, coordination, and inter-agency relationships
I
U
U
U
POD participation patterns and experiencea
I
U
U
U
Successes and challenges
I
U
U
U
Outcomes and lessons learned
.
.
I
U
.
I = initial data collection. U = updated information
POD = Promoting Opportunity Demonstration; RA = random assignment.
a Information about participant experiences will draw from interviews conducted with POD treatment subjects in Rounds 2 and 3 and MIS data.
C.8
RESEARCH QUESTIONS BY EVALUATION FOCUS AND DATA SOURCE
Table C.3. Research questions addressing each evaluation focus, by
respondent
.
VR
agency
directo
r
POD work
incentive
s
counselor
Program
environment
.
.
VR/WIP
A
agency
staff
.
SSA
paymen
t center
staff
Implementatio
n contractor
staff
TA
provider
s
POD
treatmen
t
subjects
.
.
.
.
How do the general
employment
environment and
other state features
affect benefit offset
use?
X
X
X
.
X
X
X
How prevalent is
engagement in the
Ticket to Work (TTW)
Program, and how
are ENs structured?
X
X
X
X
.
.
X
How did state VR
agencies previously
serve beneficiaries
before POD, and did
this change during
the demonstration
period?
X
X
X
.
.
.
X
Has POD
implementation
affected the
operation of other
state or local
entities?
X
.
X
.
.
.
.
Planning and early
implementation
.
.
.
.
.
.
.
What types of
training did staff
receive to implement
the demonstration?
X
X
X
X
X
X
.
What types of other
resources and
materials did staff
receive to implement
POD?
X
X
X
X
X
X
.
Have implementation
activities been
carried out as
intended?
X
X
X
X
X
X
.
What aspects of
implementation could
benefit from
additional training
and technical
assistance?
X
X
X
X
X
X
.
VR
agency
directo
r
POD work
incentive
s
counselor
VR/WIP
A
agency
staff
SSA
paymen
t center
staff
Implementatio
n contractor
staff
TA
provider
s
POD
treatmen
t
subjects
What aspects of
early program
implementation have
been successful?
X
X
X
X
X
X
.
What aspects of
early program
implementation did
not proceed as
expected?
X
X
X
X
X
X
X
.
POD recruitment,
enrollment, and RA
.
.
.
.
.
.
.
Outreach and
recruitment
.
.
.
.
.
.
.
What types of
information did DI
beneficiaries receive
about POD?
X
X
X
.
.
.
X
How successful were
outreach efforts in
helping beneficiaries
understand POD?
.
X
.
.
X
X
X
What strategies were
used to inform DI
beneficiaries about
POD?
.
X
.
.
X
X
.
How did outreach
and enrollment differ
for concurrent
beneficiaries (those
who receive SSI and
DI)?
X
X
.
.
X
X
X
What efforts were
made to inform
stakeholders about
the demonstration?
X
X
.
.
X
X
.
Has the targeted
and/or broad
outreach been
effective in reaching
potential
participants?
X
X
X
.
X
.
.
What challenges
were encountered
during recruitment?
X
X
.
.
.
X
.
What aspects of the
recruitment strategy
were successful?
X
X
.
.
.
X
.
Enrollment
.
.
.
.
.
.
.
VR
agency
directo
r
POD work
incentive
s
counselor
VR/WIP
A
agency
staff
SSA
paymen
t center
staff
Implementatio
n contractor
staff
TA
provider
s
POD
treatmen
t
subjects
What factors affected
enrollment/take-up
rates?
X
X
.
.
.
.
.
What were
beneficiaries’
experiences with
enrollment?
.
X
.
.
.
.
X
.
Random
assignment
.
How have
withdrawals affected
the balance achieved
through random
assignment?
Development of the
POD infrastructure
.
X
.
.
.
.
.
.
.
.
.
.
.
X
.
.
.
.
.
.
What aspects of
implementation
activities need
correction?
X
X
X
X
X
X
.
To what extent are
discovered issues
systematic in nature?
X
X
X
X
X
X
.
How have
discovered issues
affected operations?
Service delivery?
Offset use?
X
X
X
X
X
X
.
What aspects of the
state-level activities
need TA?
X
X
X
.
X
X
.
Data systems and
use of the MIS
.
.
.
.
.
.
.
What data are
captured in the
implementer’s
management
information system
(MIS)?
X
X
.
.
X
.
.
What supporting
functionality does the
implementer’s MIS
have for program
operations and
monitoring?
X
.
.
.
X
X
.
What data are
transferred to SSA to
facilitate
administration of the
interventions?
X
.
.
X
X
.
.
VR
agency
directo
r
POD work
incentive
s
counselor
VR/WIP
A
agency
staff
SSA
paymen
t center
staff
Implementatio
n contractor
staff
TA
provider
s
POD
treatmen
t
subjects
Are the staff using
the project’s MIS as
designed?
X
X
X
X
X
X
.
What specific
modifications to the
system and/or
training are needed
for accurate data
collection?
X
X
X
X
X
X
.
.
Operations and
fidelity to the
program design
.
.
.
Has the
implementation
contractor clearly
defined and correctly
provided the
intervention?
X
X
What aspects of SSA
operations need
support?
X
X
What aspects of
coordination with
state VR agencies
need support?
X
X
What aspects of
Implementation
activities need
correction?
X
What strategies are
used to maintain the
integrity of random
assignment?
.
.
.
.
X
X
.
X
X
X
.
X
.
X
X
.
X
X
X
X
X
.
X
X
X
.
X
X
.
Is the integrity of
random assignment
being maintained?
.
X
.
.
X
X
X
Was POD
implemented with
fidelity to the
program design?
X
X
X
X
X
X
.
POD service
delivery
What are
beneficiaries’
experiences with
POD counselors?
.
.
.
.
.
.
X
.
.
.
X
.
.
.
X
X
VR
agency
directo
r
POD work
incentive
s
counselor
VR/WIP
A
agency
staff
SSA
paymen
t center
staff
Implementatio
n contractor
staff
TA
provider
s
POD
treatmen
t
subjects
How do state VR
agencies collect
earnings and IRWE
information from
individuals assigned
to a treatment
group?
.
X
X
.
X
.
X
What role do state
VR agencies play in
making adjustments
to benefit payments?
X
X
X
.
X
.
.
How timely are
benefit adjustments?
.
.
.
X
X
.
X
What are the major
reasons for benefit
adjustment delays, if
any?
X
X
.
X
X
.
.
How prevalent are
improper payments?
.
.
.
X
X
.
X
How does monthly
earnings reporting
differ for concurrent
beneficiaries?
X
X
X
X
X
.
.
Are monthly earnings
and IRWEs reported
on time?
X
X
X
X
X
.
X
.
Administration,
coordination, and
inter-agency
relationships
.
.
.
.
.
.
.
What is the
organizational and
management
structure of the state
entities implementing
POD?
X
X
X
.
.
X
.
How do the
organizations
implementing POD
communicate and
coordinate with one
another?
X
X
X
X
X
X
X
What specific gaps
or weaknesses exist
in the partnerships?
X
X
X
X
X
X
.
POD participation
patterns and
experience
Do beneficiaries
favorably perceive
the interventions?
.
.
.
.
X
.
.
.
.
.
X
.
.
X
VR
agency
directo
r
POD work
incentive
s
counselor
VR/WIP
A
agency
staff
SSA
paymen
t center
staff
Implementatio
n contractor
staff
TA
provider
s
POD
treatmen
t
subjects
What are most
prevalent reasons for
volunteering or not
volunteering?
X
X
.
.
X
.
X
What are the events
causing POD
withdrawals, if
applicable?
X
X
.
.
X
.
X
What are the
participants’ reasons
for withdrawing?
X
X
.
.
X
.
X
.
Successes and
challenges
.
.
.
.
.
.
.
What aspects of
planning and early
implementation were
successful? Less
successful?
X
X
X
X
X
X
X
What are the
administrative (i.e.,
implementation,
operations, or
systems)
successes/challenge
s in providing the
intervention?
X
X
.
X
X
X
.
What facilitated
timely earnings and
IRWE reporting?
X
X
.
X
X
X
X
What are the
obstacles preventing
timely earnings and
IRWE reporting?
X
X
.
X
X
X
X
What facilitated
benefit offset use?
.
X
.
X
X
.
X
What are the
obstacles preventing
benefit offset use, if
any?
.
X
.
X
X
.
X
.
VR
agency
directo
r
POD work
incentive
s
counselor
Outcomes and
lessons learned
.
.
VR/WIP
A
agency
staff
.
SSA
paymen
t center
staff
Implementatio
n contractor
staff
TA
provider
s
POD
treatmen
t
subjects
.
.
.
.
How does POD
change the
participant’s use of
other government
programs or income
supports?
X
X
X
.
.
X
X
What lessons
learned can be
shared with other
sites gearing up to
implement a similar
program?
X
X
X
X
X
X
X
INTERVIEW TOPICS FOR STAFF INTERVIEWS: ROUNDS 1-4
17
The process study team will interview a variety of respondents to gather information about
implementation of POD in each of the eight demonstration sites. The team will conduct two
rounds of in-person site visits to each of the POD sites in the first and third rounds of data
collection to observe implementation activities and gather information at the start of recruitment
and one year after the last POD participant is enrolled. We will conduct in-person visits when
key respondents are working centrally on site at the VR agency. However, in states that rely
more on telecommuting staff, such as WIPAs who employ staff who typically work at home, we
will conduct interviews by telephone to collect information from the geographically dispersed
key informants to make data collection less costly. The team will conduct two rounds of
‘virtual’ telephonic site visits in the second and fourth rounds, corresponding with random
assignment and two years after enrollment of the last participant.
We organized the topic areas for the staff interviews described above by evaluation focus
area, and provide a list of topics we will cover during the in-person and ‘virtual’ telephonic site
visits with eight POD sites for the POD process study. We summarized the topics in Table C.4.
Not all topics are applicable to all POD sites or respondents. For example, we will focus on
program administration and management practices during the discussions with program
managers, and we will focus on participant engagement and service delivery during discussions
with POD work incentives counseling staff. Before each round of data collection, the task leader
will create semi-structured interview guides from the interview topics that are relevant to that site
visit round and key respondent. For each round of data collection, the team will develop an
interview guide tailored to each respondent (between 5-6 interview guides total per round) which
builds on information obtained during early program implementation.
18
Table C.4. Staff interview topics addressing each research question, by evaluation focus
.
Interview topics
I. Respondent background and experience
General background information
History and mission of the organization/agency
Respondent’s tenure and role in organization/agency
Respondent’s role in and time spent on POD
Other staff within organization/agency working on POD (number, roles, FTE, and qualifications)
How POD fits within organization/agency’s overall structure
II. Program environmenta–community context and the counterfactual
A. Local environment
How do the general employment
environment, state policies, and
other state features affect benefit
offset use?
Implications of the economy on the labor market over past year
Employment opportunities available to people with disabilities
Implications of political climate on service environment over past year
State/local policies and features (i.e., public transit) that assist or hinder people with disabilities in securing employment
Description of geographic region/service area (size and distribution of population)
Demographic characteristics of those living in service area
Characteristics of people with disabilities living in the service area
B. Service environment
How prevalent is engagement in the
Ticket to Work (TTW) Program, and
how are ENs structured?
Pre-existing services available through the local Work Incentives Planning and Assistance (WIPA) providers for DI
beneficiaries (before POD)
How did state VR agencies
previously serve beneficiaries before
POD, and did this change during the
demonstration period?
Pre-existing services available through the local VR agency for adults with disabilities in general and for DI beneficiaries
(before POD)
Changes in VR services during the demonstration period
Successes implementing POD through VR agency
Challenges encountered implementing POD during the demonstration period
19
.
Interview topics
C. Planning and early implementation
What types of training did staff
receive to implement the
demonstration?
Training received and still needed
Technical assistance received and areas where TA is still needed
Assessment of training and TA materials
What types of other resources and
materials did staff receive to
implement POD?
Types of TA resources made available to organization/agency
Assessment of training and TA resources; which have been especially effective and helpful
What pre-implementation activities
were carried out to design and plan
for implementation of POD?
Organizations/agencies involved in initial planning process and nature of involvement
Inter-agency and other agreements and contracts that were developed
Major steps in the planning process
Development of operational policies and procedures for SSA offices, VR agencies, and POD benefits counselors
Manuals and other materials developed in preparation for the demonstration
Have implementation activities been
carried out as intended?
Assessment of planning and early implementation period, and whether start-up was implemented as planned
What aspects of implementation
could benefit from additional training
and technical assistance?
Aspects of implementation that could benefit from additional TA
Types of training and TA still needed
What aspects of early program
implementation have been
successful?
Aspects of implementation that functioned well during demonstration start-up
Factors that contributed to these outcomes
What aspects of early program
implementation did not proceed as
expected?
Challenges encountered during demonstration start-up
Aspects of implementation that did not proceed as planned
Efforts to address issues and solutions that were least effective/most effective
III. POD Recruitment, enrollment, and RA
A. Outreach and recruitment
What types of information did DI
beneficiaries receive about POD?
Process for enrolling POD participants
Strategies used to inform SSDI beneficiaries about POD
Efforts to respond to questions from individuals assigned to treatment groups
What strategies were used to inform
SSDI beneficiaries about POD?
Strategies and methods used to recruit POD participants, and level of effort required
Information made available to SSDI beneficiaries through SSA field offices, local WIPA providers, and other entities
20
.
Interview topics
How did outreach and enrollment
differ for concurrent beneficiaries
(those who receive SSI and DI)?
Aspects of outreach and enrollment that are different for concurrent beneficiaries
What efforts were made to inform
stakeholders and service providers
about the demonstration?
Strategies to inform stakeholders about POD
Stakeholders’ reactions and response to demonstration
Which outreach methods been
effective in reaching potential
participants?
SSDI beneficiaries’ motivations to participate in POD
POD participants’ understanding of POD offset rules
Incentives used to attract POD participants
What challenges were encountered
during recruitment?
Recruitment challenges encountered
Reasons why SSDI beneficiaries refused to participate in POD
What aspects of the recruitment
strategy were successful?
Recruitment successes
Aspects of recruitment approach that enticed SSDI beneficiaries to participate in POD
B. Enrollment
What factors affected
enrollment/take-up rates?
Information conveyed to volunteers before and after random assignment
How information varies by treatment group, state, and disability type
Did prospective enrollees complete
baseline surveys properly?
Assessment of quality and completeness of baseline surveys completed by volunteers.
Did prospective enrollees complete
informed consent form properly?
Assessment of quality and completeness of informed consent forms completed by volunteers.
IV. Development of the POD infrastructure
What aspects of implementation
activities need correction?
Extent to which the demonstration was implemented as planned, overall and for each participant group
Consistency of project activities and services with project design
Implementation activities that are disrupting other functions
To what extent are discovered
issues systematic in nature?
Frequency of discovered issues across sites
Consistency of discovered issues over time
How have discovered issues
affected operations? Service
delivery? Offset use?
Changes to operations, service delivery, and offset use that resulted from discovered issues
Extent to which issues disrupted operations, service delivery, and offset use
Ability of site to address issues without disrupting operations, service delivery, and offset use
21
.
What aspects of the state-level
activities need TA?
Interview topics
TA needs related to communication with participants, collection and coordination of earnings and IRWEs, work
incentives counseling, MIS, sample crossover and contamination.
Consistency of TA needs across sites
V. Data systems and use of the MIS
What data are captured in the
implementer’s management
information system (MIS)?
Data elements in RAPTER and Implementation Contractor’s MIS
How data are used to support benefit adjustments under POD offset
How data are used to support end of year reconciliation process
What functionality is the
implementer’s MIS equipped with to
support program operations and
monitoring?
Functions and utility of RAPTER, Implementation Contractor's MIS, and data entry processes
Processes used to track and monitor site operations
What data are transferred to SSA to
facilitate administration of the POD
offset?
Data transfers between sites and SSA to support administration of the POD offset
Process for adjusting the benefits based on earnings and IRWE information
Are the staff using the project’s MIS
as designed?
Collection of participants’ earnings and IRWE information
Degree to which site staff utilize MIS consistently across sites
Training and technical assistance received
What specific modifications to the
system and/or training are needed
for accurate data collection?
Program adjustments made in response to formative evaluation findings
Training and technical assistance still needed
VI. Operations and fidelity to the program design
Has the implementation contractor
clearly defined and correctly
provided the intervention?
Extent to which sites adhere to the program design
Consistency of program implementation across sites
Extent to which benefits counselors and VR agencies use the POD manual
Use of technical assistance to correct implementation issues
What aspects of SSA operations
need support?
Aspects of SSA operations that need support, as they relate to adjustment of benefits based on earnings
Other aspects of SSA operations that need support
Staffing capacity and changes in staffing during the demonstration
What aspects of coordination with
state VR agencies need support?
Aspects of coordination related to SSA operations
Aspects of coordination related to benefits counselors
Aspects of coordination related to implementation and evaluation contractors
22
.
Interview topics
What strategies are used to
maintain the integrity of random
assignment?
Efforts made to keep the treatment and control groups from sharing knowledge about the demonstration
Procedures in place to maintain the integrity of the random assignment process
Was POD implemented with fidelity
to the program design?
Extent to which POD was implemented as planned, overall and for each group
Extent to which POD operations or services changed, and reasons for change(s)
VII. POD service delivery
Has the implementation contractor
clearly defined and correctly
provided the intervention?
Role of Implementation Contractor in making benefit adjustments
How sites support participants to understand relevant SSA notices and file reconsiderations/appeals/waivers of
overpayments
How do state VR agencies collect
earnings and IRWE information from
individuals assigned to a treatment
group?
Process for collecting earnings and IRWE information from individuals in treatment groups
What role do state VR agencies play
in making adjustments to benefit
payments?
Role of state VR agencies in making benefit adjustments
How quickly are benefit payments
adjusted for offset use?
Processing time for implementation contractor to process submitted earnings and IRWE information and transmit to
SSA
Processing time for SSA to apply benefit offset after receipt of earnings records over POD threshold
What are the major reasons for
benefit adjustment delays, if any?
Major reasons for benefit adjustment delays, if any
How does monthly earnings
reporting differ for concurrent
beneficiaries?
Process for collecting earnings and IRWE information from concurrent beneficiaries in treatment groups
Process for adjusting benefits based on earnings for concurrent beneficiaries
Are monthly earnings and IRWEs
reported on time?
Incidence and frequency of monthly earnings and IRWE reporting
Effect of outreach efforts or reminders on monthly earnings reporting
VIII. Administration, coordination, and inter-agency relationships
What is the organizational and
management structure of the state
entities implementing POD?
Roles of partner organizations/agencies in implementing POD
Overall organization/agency and management structure of state entities implementing POD
Formal and informal agreements between organizations/agencies involved in implementing POD
How do the organizations
implementing POD communicate
and coordinate with one another?
Nature of communication and collaboration between organizations/agencies involved in implementing POD
How inter-agency relationships have changed over time
23
.
What specific gaps or weaknesses
exist in the partnerships?
Interview topics
Gaps and weaknesses in demonstration partnerships
Successful aspects of demonstration partnerships
24
.
Interview topics
IX. Successes and challenges
What aspects of planning and early
implementation were successful?
Less successful?
Specific successes during the planning and start-up phases of implementing POD
Factors that contributed to success during planning and early implementation
Specific areas in which technical assistance or support was needed
Solutions that were most/least effective
What are the administrative (i.e.,
implementation, operations, or
systems) successes/challenges in
providing the intervention?
Specific successes/challenges encountered in implementing POD
Features of the site or individuals that promoted these successes
How challenges were addressed
Success in addressing challenges
What facilitated timely earnings and
IRWE reporting?
Facilitators of timely earnings and IRWE reporting
Ability to replicate facilitators in other sites
What are the obstacles preventing
timely earnings and IRWE
reporting?
Obstacles preventing timely earnings and IRWE reporting
How obstacles were addressed
Success in addressing obstacles
What facilitated benefit offset use?
Facilitators of benefit offset use
Ability to replicate facilitators in other sites
What are the obstacles preventing
benefit offset use, if any?
Obstacles preventing benefit offset use
How obstacles were addressed
Success in addressing obstacles
X. Outcomes and lessons learned
How does POD change the
participant’s use of other
government programs or income
supports?
Increase or decrease in POD participant use of other government programs or income supports
Which programs/supports, and how participant use changed
Unexpected consequences of participating in POD
Factors that contributed to these consequences
What lessons learned can be shared
with other sites gearing up to
implement a similar program?
Promising practices in the implementation of POD, both overall and at each site
Extent to which practices are replicable in other sites
Lessons learned through the implementation of POD
Extent to which these lessons apply to other sites
aSome
of this information will be gathered prior to the site visit.
25
| File Type | application/pdf |
| File Modified | 2017-08-03 |
| File Created | 2017-08-03 |