Staff reports of program service receipt

appendix G

staff reports of program receipt

This page has been left blank for double-sided copying.

This appendix provides details on data collected through the web-based information system developed for the evaluation, the Random Assignment, Participant Tracking Enrollment, and Reporting system (RAPTER). RAPTER was developed for program staff to conduct random assignment of applicants to the program and collect data on program participation. Justification for the data elements collected by RAPTER is provided in Table 1. The wireframes included in this appendix depict the planned arrangement of the RAPTER’s content, including data elements and navigation instructions. Also embedded in the RAPTER system will be Attachment B Baseline Questionnaire, which will be launched from RAPTER for each client as part of the intake process. Each wireframe has a descriptive title and represents a planned screen in RAPTER. The wireframes are grouped into screens pertaining to client services, group activities, and system administrative functions.

Client services screens. Screens starting with a “C” (C1-C18) are for randomly assigning clients and documenting individual services provided to a client.

• Screen C1. This is the home page. This is the screen users will see when they first log into the system. On the home page, users can click on the “Clients” button at the top of the screen which will route them to screen C2, the All Clients Search page.

• Screen C2. On this page, a user can search for an existing client or add a new client. The user can also click on the “My Clients” hyperlink, which will route them to screen C3 where they will see a listing of their assigned clients.

• Screen C3. This shows a list of the clients assigned to the staff member.

• Screen C4. Shows the first step of the intake process. The client’s name is entered and whether the client meets any study eligibility criteria; eligibility criteria will be specific to each site in the evaluation. Intake staff will be prompted to enter the program-specific ID number. We will link the study ID number to the program-specific ID number.

• Screen C5. The consent statement is read to the client and the client’s consent is recorded.

• Screen C6. Staff launch the baseline questionnaire.

• Screen C7. After staff complete the baseline questionnaire, they answer a question about the referral source and two questions about their perception of the likelihood of the client participating in services before clicking the random assignment button. Staff will also answer a question about the location of the intake.

• Screen C8. This screen shows what the staff member will see when a client submitted for random assignment is flagged as someone who has already been randomly assigned.

• Screen C9. This will be the screen that the staff member will see after a client is successfully randomly assigned. Once random assignment is complete, this screen summarizes the client’s service receipt going forward.

• Screen C10. This screen will be used to assign a coach or case manager to a client by selecting a name from a drop-down list.

• Screens C11a and C11b. These screens are how appointments are scheduled for coaching and case management sessions by selecting a date, time, and location for the appointment.

• Screen C12. This is the service history tab, which is one of the tabs displayed on the Client Profile (Screen C9). On Screen C12 users can view all upcoming case management appointments; start a new case management session; cancel, reschedule, or edit an existing case management appointment; or add and view information about programmatic incentives.

• Screen C13. This is the coaching tab and is similar to the service history tab; however, this tab will only be visible to coaches. It is where coaches can view upcoming coaching sessions and cancel, reschedule, or edit an existing coaching appointment.

• Screen C14. This screen is where coaching and case management appointments can be cancelled or rescheduled. On this screen the user would provide some information about the cancellation/rescheduling and choose a new appointment time if the appointment is being rescheduled.

• Screen C15. This is the screen coaches will use each time they complete a coaching session to record the length of the session, content and methods, and goals that were discussed.

• Screen C16. This is used by case managers after each case management session to record information about the session including the length of the session and session content.

• Screen C17. This is a screen that staff can use to record any programmatic incentives that are distributed to the client; staff will record the date it was provided, the type of incentive, and the incentive value.

• Screen C18. This is the screen staff will use to record information about a client’s work experience assignment. This screen will also display a list of all work experiences that have occurred during the client’s enrollment in the program.

Group activity screens. Screens starting with a G (G1-G2) are for documenting group activities such as workshops and trainings.

• Screen G1 displays the list of activities that have already been created and are available to all members of the treatment group. Staff can search for activities on this page or click the button to create a new activity.

• Screen G2 allows staff to create a new activity and to record attendance for the new activity or an activity selected from the search list.

System administrative function screens. Screens starting with a U (U1-U4) are general screens to assist system users with creating new user accounts and generating reports.

• Screen U1 is a screen where the supervisor or system administrator for a site can view and edit the user accounts for their staff.

• Screen U2 is the screen where a new user can be added with their user type and permission level. Permission levels and user types will control what information a user has access to in the system. The evaluation team will designate a staff member at each program with a permission level that enables creating new accounts and assigning permission levels.

• Screen U3 is where reports will be displayed for users. Users can click on one of the report hyperlinks to generate the report.

• Screen U4 is where reports are generated after a user selects parameters for the report. Parameters will vary depending upon the topic of the report, but will include things like start and end dates for the report, filtering on subsites, and specific characteristics of the sample such as random assignment status and length of enrollment.

Use of RAPTER screens by site. The use of certain RAPTER screens will vary for the sites selected for the evaluation. The two MyGoals sites will not use RAPTER because evaluation intake and service receipt documentation will take place in a previously developed system already in use by the site. FaDSS will use RAPTER for both evaluation intake and service receipt documentation. However, FaDSS will not use screens that pertain to services that are not offered by the program, such as case management, work experience, and group sessions. Table 2 lists the RAPTER screens in use for FaDSS.

Table 1 RAPTER System: Item-by-Item Justification

Entry	Justification
Intake for Random Assignment
C4. Intake-Step 1 Eligibility Criteria 1 Eligibility Criteria 2 Eligibility Criteria 3	Eligibility criteria will be determined when sites are selected. If a site has program-specific eligibility criteria then staff will ask clients if they meet each criterion and mark yes or no in the system. This is needed to prevent clients who aren’t eligible for the program from taking the baseline questionnaire and being randomly assigned.
Program-specific ID number	We will link the study ID number to the program-specific ID number to facilitate communication with program staff.
C7. Random Assignment-Additional Information Name Date of Birth Social Security Number Gender	Required for checking that the applicant is not already a sample member before conducting random assignment. Information on date of birth, Social Security number and gender will be prefilled from data from the baseline questionnaire.
Intake location Program office Applicant home Other location	Staff will identify where the intake took place for descriptive purposes and to inform program recruitment and retention strategies
Referral source Program’s intake or outreach staff Program’s partner agency Other community agency Court order Word of mouth Advertisement, flyer, or other public announcement Client does not remember Other (please specify)	Staff will identify how the applicant was referred to the program for descriptive purposes and to inform program recruitment strategies.
Likelihood of participation Very likely Somewhat likely Somewhat unlikely Very unlikely	Staff will answer questions about their perception of the likelihood of the client participating in coaching services. These questions will be used with other baseline information about participants to infer likelihood of participation and estimate the impact of treatment on the treated.
Service Receipt
C9. Client Profile Name Contact information (address, phone numbers, email) Additional Contacts Enrollment date (default is random assignment date) Client Status Status Change Date	The client profile will allow staff to track and manage clients that are participating in the program. Enrollment date, client status, and status change date will be used to describe enrollment patterns.
C9. Client Profile Assigned Case Manager Assigned Coach	Assigning a case manager and/or a coach will allow program staff members to schedule appointments with their assigned clients as well as enter and manage client data.
C9. Client Profile Schedule Case Management Meeting Schedule Coaching Session	The scheduling function allows program staff to record scheduled meetings in advance with clients. Appointments will be tracked and will provide data on how frequently clients attended and an overall total of sessions they attended.
C14. Cancel/Reschedule Coaching or Case Management Sessions When did the client cancel/reschedule?	The cancel and rescheduling screen will allow staff to track their appointments with clients and document if clients cancel or reschedule, how much notice was provided, and the reason the client cancelled. This information will be used to describe program participation and assess clients’ punctuality and engagement.
Cancellation/reschedule notice More than a day Same day No show/no notice
Reason for cancelling/rescheduling Transportation Child care Sick Employment Weather No excuse provided Other
New session date
C15. Record Coaching Session Session Date	For each individual coaching session, coaches will record information about whether the client was late, session length, location, mode, methods used, and the content of the session. Recorded information will also include the goals discussed during the session and whether practice assignments were given and/or completed. Coaches will also have the option to add a referral. This information will be used to describe how coaching services are provided and track the goal setting and pursuit process.
Was the client late to the session? Yes, No If yes, how late? 5-10 minutes 11-15 minutes 16-30 minutes More than 30 minutes
How long was the session? Less than 5 minutes 5 minutes-15 minutes 16 minutes-30 minutes 31 minutes-45 minutes 46 minutes-60 minutes 61 minutes-90 minutes 91 minutes-120 minutes More than 120 minutes
Session Location Fill site/office name Client’s home Client’s place of employment Telephone Other community space
Mode of Session Phone In-Person Group Other (please specify)
What methods were used in this session? Assessments Development of action steps Discussion of incentives Goal-setting Assignments Other (please specify)
Choose the topics you went over in the session Work Readiness Child care Education Interviews Job openings Problems on the job Resume Training Transportation Work requirements Other (please specify) Personal Supports Basic needs Budgets Child behavior Domestic violence Family relationships Financial literacy Housing Legal issues Physical health Mental health Stress reduction Substance abuse Other (please specify)
Goals discussed during session
Was a new assignment given to the client? Yes No
Did the client complete their assignment(s) for this session? Completely Partly Not at all N/A
Add referral Did the case manager contact the agency to which the client was referred? Yes, No Choose the type of referral that was made to the client Basic needs Child behavioral issues Child care Clothes, uniforms, tools Domestic violence Family relationships Financial counseling Legal advice, record expungement Physical health Mental health Stable housing Stress reduction Substance abuse treatment Transportation Tuition assistance Other (please specify)
C16. Record Case Management Session Was the client late to the session? Yes, No If yes, how late? 5-10 minutes 11-15 minutes 16-30 minutes More than 30 minutes	For each individual case management session, staff will record information about whether the client was late, session length, location, mode, and the content of the session. Staff will also have the option to add a referral. This information will be used to describe how case management services are provided.
How long was the session? Less than 5 minutes 5 minutes-15 minutes 16 minutes-30 minutes 31 minutes-45 minutes 46 minutes-60 minutes 61 minutes-90 minutes 91 minutes-120 minutes More than 120 minutes
Session Location Fill site/office name Client’s home Client’s place of employment Telephone Other community space
Mode of Session Phone In-person Group Other (please specify)
Choose the topics you went over in the session Work Readiness Child care Education Interviews Job openings Problems on the job Resume Training Transportation Work Requirements Other (please specify) Personal Supports Basic needs Budgets Child behavior Domestic violence Family relationships Financial literacy Housing Legal issues Physical health Mental health Stress reduction Substance Abuse Other (please specify)
Add referral Did the case manager contact the agency to which the client was referred? Yes, No Choose the type of referral that was made to the client Basic needs Child behavioral issues Child care Clothes, uniforms, tools Domestic violence Family relationships Financial counseling Legal advice, record expungement Physical health Mental health Stable housing Stress reduction Substance abuse treatment Transportation Tuition assistance Other (please specify)
C17. Record Incentive Date provided	Staff will record information about incentives that are distributed to clients by the program for meeting program-defined participation and goal-pursuit targets. This information includes the date the incentive was provided, the type of incentive, and the reason why the client received the incentive. Information about incentives will be tracked in the system and will provide information on the frequency and amounts of incentives.
Type of incentive Bus pass or gas card Gift card Products Other (please specify)
Reason for incentive Completed assignment Completed a goal Met with coach more than [X] number of times Other (please specify)
C18. Record Work Experience Employer Name and Address Information Position Supervisor’s Name Supervisor’s Phone Employment Status	This information will be used to describe the kinds of work experiences clients receive while in the program.
Employment subsidy Yes, No Employment subsidy amount Unit Per hour Per day Per week
Hours per week Pay from employer
Start Date/End Date Reason Employment Ended Employment terminated by the employer Employment terminated by the client Employment opportunity ended as planned
G2. Record Group Activity Date of Service Name of Activity Location Record Attendance	This screen allows staff to create a group activity such as a workshop or a training session that multiple clients attend at once. This information will be used to describe the kinds of group activities clients participate in.

Table 2. RAPTER screens to be used by FaDSS

RAPTER page	FaDSS
C1. Home Page	
C2. All Clients - Search	
C3. My Clients List	
C4. Intake – Step 1	
C5. Intake – Step 2	
C6. Intake – Step 3	
C7. Intake – Step 4	
C8. Duplicate	
C9 Client Profile	
C10. Assign Program Staff	
C11a. Schedule Coaching Session	
C11b. Schedule Case Management Meeting
C12. Service History	
C13. Coaching	
C14. Cancel/Reschedule Coaching Session	
C15. Record Coaching Session	
C16. Record Case Management Session
C17. Record Incentive
C18. Record Employment/Work Experience
G1. Group Activity Listing
G2. Record Group Activity
U1. Users	
U2. Add User/Edit User	
U3. On-Demand Reports Listing	
U4. On-Demand Report Generation	

Note: MyGoals will not use RAPTER because evaluation intake and service receipt documentation take place in a previously developed system.