Aging Worker Initiative Evaluation

Site Visit Topics for Project Policy and Administrative Staff (Round 1)

Aging Worker Initiative Site Visit and phone reconnaissance Protocols

Site Visit Introduction Script (to be read to each respondent prior to each discussion/interview):

Hello. My name is ________________ and I am with __________________________, which is conducting an evaluation of the Aging Worker Initiative. Thank you for agreeing to talk with me today about your experiences in administering (or participating in) one of the grant under this Initiative. Like the grant, the evaluation is funded by the U.S. Department of Labor. The study is designed to help identify what worked well or didn’t work well in the Initiative, and ultimately to improve the ability of similar programs to meet the needs of older workers and the employers who may hire them.

The information you provide for our discussion will be used to identify patterns in the management of the projects and the provision of services, including challenges and successes, as well as issues and concerns. We will be talking not only with you but others who were involved in administering the AWI grant.

Please be assured that any information that you and other respondents provide during the site visit (or through this phone reconnaissance) will be treated as private. Information you provide may be shared orally with the U.S. Department of Labor staff and in written, published reports from the evaluation. However, such orally-shared information and written reports will discuss challenges or problems in such as way as to avoid identification of this program or you specifically, so as to encourage candid responses. Published reports may, however, document specific promising practices or program activities, specifically associated with this site. Further, on some topics, our reports and oral comments will describe the range of views expressed by you and other respondents, but no specific comments will not be attributed to you.

Please let me know if you have any concerns or questions about sharing any information orally with the U.S. Department of Labor or in written, published reports, and if you have any questions about how this information will be shared.

List of Protocols

Site Visit Topics for Project Policy and Administrative Staff
(Round 1) 3

Site Visit Topics for Project Policy and Administrative Staff
(Round 2) 3

Site Visit Topics for Direct Service Delivery Staff and Supervisors (Round 1) 3

Site Visit Topics for Direct Service Delivery Staff and Supervisors (Round 2) 3

Site Visit Topics for Representatives of Partner Agencies
(Round 1) 3

Site Visit Topics for Representatives of Partner Agencies
(Round 2) 3

Topics for Data Reconnaissance Phone Calls to Projects
(Fall 2011) 3

Topics for Final Telephone Call to Projects (June 2012) 3

Site Visit Topics for Project Policy and Administrative Staff (Round 1)

1. OVERVIEW OF PROGRAM INITIATIVE

1.1 Impetus for/ Purpose of the Project

• How did this project come about? What individual(s) or organization(s) were the chief instigators or initiators of the project proposal?

  • ^{Who were the key individuals and entities involved in writing the initial grant application to USDOL?}

  • ^{How did you select the grant recipient (grantee organization)?}

• What is the purpose of your project?

  • ^{What are the key challenges facing aging workers who would like to work for pay in your region? Which of these challenges is the project particularly focused on addressing?}

  • ^{What are the key challenges facing employers who need workers with the skill set to match current and anticipated jobs in high growth sectors?}

  • ^{Which of these challenges is the project particularly focused on addressing?}

• Why was the AWI grant announcement attractive to you?

  • ^{What were the perceived opportunities of the grant?}

  • ^{Were there any aspects of the grant requirements that were not such a good fit with your needs and interests?}

1.2 Goals of the Project

• How would you describe your project philosophy or approach?

• What program goals do you hope to achieve? To what extent are these goals quantifiable?

  • ^{How could these goals be assessed? (How will you know if you are successful?)}

  • ^{Have the goals of the project evolved or changed since the grant began?}

• What outcomes do you hope to achieve for individual participants?

• How are these outcomes measured (common measures, other ways)?

1.3 Planning Process

• What were the key steps in planning for this initiative? How much of the design was developed in the grant application? What was the planning process like after the grant was awarded?

• Who were the key players involved in the design of the project and what organizations/ entities did they represent?

  • ^{How engaged were different project partners (including LWIB board and staff, employers and employer associations, organizations specializing in aging worker services, and education and training providers in the planning process? How often did you meet?}

  • ^{Are there organizations that you wish had been at the table, in hindsight, and why?}

• How would you characterize the overall planning process? Did it go smoothly?

  • ^{What were the main issues of focus and/or concern during the planning process?}

  • ^{Were there any particular challenges during the planning of the project?}

• What was the effect of the economic recession on the project?

  • ^{How did the onset of the economic recession influence project planning and design?}

  • ^{How did the onset of the economic recession influence contribution of leveraged resources by project partners?}

  • ^{If partners have received additional funds from the Recovery Act, what effect have they had on contribution of leveraged resources?}

1.4 Target Population

• What specific groups of aging workers does your project target (age, employment status, previous work history, other characteristics)?

  • ^{Why were these particular groups targeted for this program in your area?}

  • ^{Among the target participants, what are the most common challenges to getting and keeping a job?}

• What are the characteristics of participants enrolled to date? What are their education and skill levels, and level of work experience?

  • ^{Are these participants consistent with the targeted groups?}

  • ^{Have you been surprised by any of the characteristics of the enrolled participants?}

5. Target Sectors/ Industries

• What sectors/industries are targeted by the initiative?

  • ^{How were these sectors/industries chosen? How do they relate to the 14 sectors targeted by the High Growth Job Training Initiative (the sectors eligible for H-1B visas)?}

  • ^{How do these sectors/industries meet the criteria for high-growth, high-demand industries and sectors (add substantial numbers of new jobs, significant impact on economy overall, impacts the growth of other industries, transformed by technology/requiring new worker skill sets, new and emerging business that is expected to grow)?}

• Why were specific occupations chosen? Were they selected because they were particularly suitable to the needs of older workers)?

• How do the targeted industries and occupations reflect regional economic development strategies?

5. . Program Eligibility Requirements

• What are the eligibility requirements for participation in the program?

  • ^{Are there any additional restrictions or goals beyond the grant requirement (age 55+)? Do participants need to be currently unemployed or employed in a particular industry?}

  • ^{Have eligibility requirements posed any challenges to program success?}

2. DESCRIPTION OF PROJECT SERVICE AREA

2.1. Description of Local Service Area(s) Targeted for the Project

• Please describe the project service area.

  • ^{What is the size of the service area targeted for the initiative? What is the population of the service area? How is the area defined (e.g., county, zip codes, jurisdiction of LWIB(s))?}

  • ^{What are the essential characteristics of the area (e.g., urban/suburban/rural/mixed)?}

  • ^{How and why was this service area selected?}

• Are there multiple service sites within the project’s service area? Are different project partners active in different parts of the service area? Are different entities managing the project in different parts of the service area?

• Will the initiative be implemented uniformly throughout the service area?

2.2 Description of Local Labor Market in the Designated Service Area

• Please describe the local labor market.

  • ^{What is the local unemployment rate? What are the major industries/employers?}

  • ^{What types of jobs are older workers most likely to have?}

  • ^{What are the sectors of growth and/or decline in the regional economy? For example, are health care jobs, green jobs, etc. considered high growth sectors?}

• How has the recession affected grant planning and implementation?

  • ^{Has the recession had any effect on employer interest in hiring aging workers; in terms of participant interest in enrolling?}

  • ^{Has the recession had any effect on the types of occupations for which participants are trained, or the number of job openings?}

  • ^{What other challenges were created by the economic recession? How were these challenges addressed?}

• How is the project design and implementation experience influenced by the regional labor market and economic trends?

2.3 Description of Other Services Available to Older Workers in the Service Area

• Before this grant, what employment and training services existed for older workers?

  • ^{What services for older workers were available from public workforce development funds, e.g., from WIA adult or dislocated worker funding stream; SCSEP; WIA or other incumbent worker training?}

  • ^{Did other agencies or funding streams also support employment and training services for older workers? If so, provide details.}

  • ^{Were there any public training funds targeted to currently employed or retired workers?}

• Did existing training services make any special arrangements to make their services appropriate for older workers? (e.g., accommodations for people with disabilities; changes in working hours or conditions?)

• What factors, if any, limited the services available to older workers from the general workforce development system?

2.4 Competing Initiatives or Programs for Older Workers

• Are there other programs that offer similar services to older workers in the project service area?

  • ^{Are there programs that target some of the same populations as the AWI project or compete with it for enrollees?}

  • ^{Are services offered by other programs similar to or complementary to the services provided by the AWI project?}

  • ^{Could an individual participate in both programs simultaneously?}

• How does the demand for older worker services compare to the capacity of all local programs offering relevant services. (Are all available programs operating at full capacity or are they competing for customers? Are there waiting lists at the grantee and other similar programs?)

• If they can only participate in one program, why would/do individuals select the AWI program over others? Or why would/do they choose another program instead?

3. DESCRIPTION OF GRANTEE

3.1 Grantee Agency Background

• Please describe the grantee organization.

  • ^{What type of organization is the grantee (LWIB; 501(c)3 organization; other)? How long has the grantee organization been in existence?}

  • ^{Who is the grantee’s fiscal agent (if different from the grantee)?}

  • ^{What is the mission of the grantee organization? What other services does it provide/ activities does it carry out?}

• Please describe how the project fits within the grantee organization.

  • ^{Where does project administration reside within the grantee’s organizational structure?}

  • ^{How does the scope and funding of the AWI grant compare in size to the rest of the grantee budget and activities?}

3.2 Grantee administrative and staffing structure

• What is the overall staffing structure for administering the AWI grant?

  • ^{Describe the administrative positions for the grant? What percentage time do key administrative staff working on the grant allocate to the AWI project?}

  • ^{Describe the staff positions for direct customer services (to workers and employers)? How many different organizations employ individuals who provide direct services to project participants and employers?}

  • ^{Were new staff members hired for the grant or did they come from One-Stop Career Center or partner staff?}

  • ^{If direct service staff work on other programs or projects as well, what proportion of their time or caseload is devoted to AWI participants?}

  • ^{Describe the qualifications and relevant experience of key staff.}

• Is the staffing plan consistent across all entities (e.g. multiple LWIAs) participating in the grant? If not, describe the variations.

3.3 Grantee previous experience Serving Older Workers

• Please describe the grantee’s previous experience serving older workers.

  • ^{What experience does the grantee have serving older workers through WIA or other programs?}

  • ^{How long has the grantee been serving the older worker population?}

  • ^{What are the characteristics of the older workers the grantee has served in the past? How do their characteristics compare to the AWI participants?}

  • ^{If relevant, how successful does the grantee believe that it has been in its past work with this population?}

4. DEVELOPING PROJECT PARTNERSHIPS

4.1 Outreach/ Identification of Potential Partners

• What is the range of organizations and individuals who were invited to participate in the project (e.g. employers, industry associations, educational institutions, training providers, aging organizations, SCSEP grantees, economic development entities, apprenticeship programs, tribal organizations, philanthropic community, community or faith-based organizations)?

• How were potential partners recruited for this initiative? How difficult was it to secure the participation of the targeted partners?

• At what stage were partners recruited? (e.g. before grant application or after) How much say did each partner have in developing overall project scope and design? Who determined what partners to invite?

• What strengths did you look for in each invited partner?

4.2 Formal Relationships Among Project Partners

• What types of contractual relationships, if any, were put in place between grantee and partner agencies?

  • ^{What did those arrangements encompass?}

  • ^{Were any MOUs (Memoranda of Understanding) needed to use client assessment tools or other resources?}

  • ^{Were any roadblocks encountered in getting the necessary contracts or MOUs in place? What affect did that have on the project?}

• What other agreements or procedures govern the relationships between grantee and partners?

4.3 Project Leadership and Oversight

• Describe the project leadership.

  • ^{Does the project have its own advisory board or oversight board? How is this board composed?}

  • ^{Is there a formal leadership team for the project with representation from project partners?}

  • ^{To whom does the project manager/ coordinator report?}

  • ^{Who decides the roles of the different partners and allocates project funds among the participating partners?}

• How does the project communicate with outside entities?

  • ^{Who is the formal liaison for the project with the USDOL?}

  • ^{Who is the formal liaison for the project with the technical assistance providers?}

  • ^{Who is the formal liaison for the project with the evaluation?}

• Who has the authority to identify and resolve problems in project implementation or operations?

4.4 Communication Between Grantee and Partners

• How does internal project communication work?

  • ^{How does the grantee’s project manager communicate with project partners? How frequently and about what types of issues? How do partners communicate with each other?}

  • ^{How do partners communicate with each other?}

• Does the grantee hold regular meetings with partners? If so, what is the purpose of these meetings? Are these meetings held individually or as a group? How long do they last?

  • ^{What types of information are discussed during these meetings?}

  • ^{How helpful are the meetings in assessing project status and guiding the project?}

4.5 Assessment of Partner Involvement

• How would you characterize the overall success involving partners to participate in the initiative?

  • ^{How pleased are you with the final composition of partners in the initiative?}

  • ^{Were there some agencies you initially wanted to partner with but were unable to do so? If so, what were the barriers in establishing that arrangement? Was another agency involved to fill that gap, and if not, is the initiative lacking in some way without that partner?}

  • ^{Are there any other partners that in hindsight you wish were involved in the initiative that are not involved?}

• Please describe any challenges you have experienced in developing effective relationships between the project partners.

  • ^{What challenges did you experience in recruiting partners or in arranging for their specific role in the project?}

  • ^{What challenges did you experience in the quality of the contributions made by individual partners?}

• What are the most successful aspects of your partnership?

• What advice do you have for other projects serving older workers in terms of partnership formation and partner roles?

4.6 Informal Relationships with Other Entities Serving Project Participants

• Please describe any informal relationships that the project has developed with other entities to expand the services available to project participants.

  • ^{To what extent do the workers recruited for this program receive services from other community agencies or programs?}

  • ^{What other agencies and/or programs are involved—SCSEP, One-Stop Centers (VT, CA and TX), Area Agencies on Aging?}

  • ^{To what extent are participants referred to these agencies? What proportion of participants access these services?}

  • ^{To what extent are participants referred from other agencies?}

• What are the experiences of participants referred to these providers?

• Would the project have benefited from more formal relationships with these agencies?

5. Project Partners And Their Roles

5.1 Involvement of the Public Workforce Investment System (e.g. One-Stop Centers and Constituent Programs)

(If grantee is a local WIB, describe the involvement of One-Stop staff and programs that are not directly part of the funded AWI project.

• What role did/do public workforce investment partners play in designing, managing, or overseeing the grant, if any?

• What roles do public workforce investment partners play in providing services to employer or worker customers?

• How much involvement does the AWI project have with other services provided at the One-Stop Career Centers?

  • ^{Do AWI project participants utilize WIA core and intensive services? If so, for what types of services? Are workers served under AWI co-enrolled in the One-Stop system (or SCSEP) under WIA?}

  • ^{Are other workforce investment partners involved in providing services to AWI project participants? What is the nature of that involvement?}

• How does the AWI project fit in with other workforce initiatives, such as, SCSEP or Medicaid Infrastructure Grants (MIG)?

  • ^{Are the services offered across these initiatives substitutes or complements?}

  • ^{Do AWI participants need services available through SCSEP? Can AWI participants be co-enrolled in SCSEP or participate in any of its services?}

  • ^{Describe the level of coordination and communication across the agencies administering these initiatives (AWI, SCSEP and Medicaid Infrastructure Grants (MCI)?}

• How have public workforce investment partners contributed to the success of the project?

^{Note: MCI grants from the US Department of Health and Human Services are intended to develop a comprehensive system of employment supports for people with disabilities.}

5.2 Involvement of Organizations with Expertise on Aging

• What role did/do “aging organizations” play in designing, managing, or overseeing the grant, if any?

• What other roles do “aging organizations” play in providing services to employer or worker customers?

• How have “aging organizations” contributed to the success of the project?

5.3 Involvement of Educational Institutions and Training Providers

• What role did/do educational institutions or training providers play in designing, managing, or overseeing the grant, if any?

• What other roles do educational institutions or training providers play in providing services to employer or worker customers?

• How have education and training institutions or training partners contributed to the success of the project?

5.4 Involvement of Economic Development Entities

• What role did/do economic development entities play in designing, managing, or overseeing the grant, if any?

• What other roles do economic development entities play in providing services to employer or worker customers?

• How have economic development entities contributed to the success of the project?

5.5 Involvement of Local Employers, Employer Associations, or Business Intermediaries

• What role did/do employers or business intermediaries play in designing, managing, or overseeing the grant, if any?

• How did you recruit business partners? What were your selection criteria, if any?

• What other roles do employers or business intermediaries play in providing services to employer or worker customers? Are they involved in services to their own incumbent workers; in directly recruiting and hiring project participants, or in some more general way?)

• How have employers or business intermediaries contributed to the success of the project?

• How has employer involvement been affected by the economic recession?

5.6 Involvement of Other Partners
(e.g., faith-based organizations, community organizations, philanthropic institutions, apprenticeship programs, tribal organizations, SCSEPgrantees)

• What role did/do other partners play in designing, managing, or overseeing the grant, if any?

• What other roles do other partners play in providing services to employer or worker customers?

• How have other partners contributed to the success of the project?

6. Overview and Sequencing of Services

6.1 Participant Recruitment and Referral

• How are participants recruited to the program?

  • ^{What proportions of participants are recruited through grant-specific outreach? (How do you advertise the project (e.g. brochures, public service announcements, speakers, requests for referrals)?}

  • ^{What proportion of participants are referred by:}

    • ^{One-Stop Career Centers?}

    • ^{Local education providers?}

    • ^{Other partners in the grant?}

    • ^{Other means}

• What type of special emphasis is there on recruiting disadvantaged populations (veterans, people with disabilities, military spouses, ex-offenders, minorities, new Americans)?

• Did you face any challenges in recruiting participants?

  • ^{Were there any delays in the start of participant enrollment?}

  • ^{If yes, what caused them?}

  • ^{What strategies did you use to overcome those challenges?}

• To what extent and how is recruitment of older workers linked to employer requirements?

6.2 Enrollment in Project

• At what point in the receipt of services is an individual officially “enrolled” in the project (reportable to USDOL as a participant)?

  • ^{Does receipt of a specific service or participation in a specific activity automatically activate “enrollment?”}

  • ^{Is enrollment reserved for individuals who decide to participate in occupational skills training?}

  • ^{If not, what is the ratio of enrollees who participate in training to enrollees who do not participate in training?}

  • ^{Do all training options include occupational skills training? (Or do some participants receive only pre-employment training or only job search training or only basic skills training without occupational skills training?)}

6.3 Service Components (including Orientation and Pre-Employment Services)

• What are the different service components developed by the project to meet the needs of aging workers served by the project? (E.g., orientation, assessment, service planning/career counseling, pre-employment training, skills training, academic counseling, internships or temporary work experience, job search/job placement, post-placement services)

  • ^{What is the duration and content of each service?}

  • ^{Who provides each service?}

  • ^{Is each service provided using grant funding or through leveraged funds by the providing agency?}

• What changes have occurred in the service components over time? (new services added, services redesigned, services discontinued) Why and how?

6.4 Sequencing of Services

• What is the typical sequence of project services (variations depending on customer needs?)

  • ^{What services have participants received from other sources before enrolling in AWI? Describe the depth and quality.}

  • ^{How long, on average, do participants remain active in the project?}

  • ^{To what extent do participants drop out of service delivery? At what point in service delivery do clients typically drop out? For what reasons? What type of follow-up is made with drop outs to encourage continuing participation or obtain outcome information?}

  • ^{For participants who stay with the project, what is the point (points) at which an individual is considered to have “completed” the project?}

  • ^{At what point in the service process are individuals considered to have exited the project?}

  • ^{What follow-up services are provided after project exit? When does follow-up end?}

• What is the frequency of different services?

  • ^{What project services, if any, are received by all participants? Do all participants receive occupational skills training? Pre-employment training? Other services?}

  • ^{What services are received by only a portion of all participants? What determines whether a customer will receive a given service?}

  • ^{What other types of services, including WIA-funded services are provided to AWI participants? How many individuals received these services (by type)?}

• What individuals and entities are responsible for the delivery of different services?

  • ^{In the course of participation, what project staff will participants come into contact with? Who does each of these service providers work for?}

  • ^{Do participants have to travel to different locations for different services?}

  • ^{If participants are referred to an education and training, or other project partner, what are the roles and responsibilities of the grantee? Who maintains the case—the grantee, the partner, or both?}

6.5 Co-enrollment of AWI Project Participants in Other Workforce Development Programs

• How frequently are AWI project participants provided with an orientation to the core One-Stop services (e.g. resource room and online labor market information and assessment tools) as part of their participation in the AWI project?

• How frequently are AWI project participants co-enrolled in other programs operated out of One-Stop centers (e.g. WIA, TAA. SCSEP, Employment Services (Wagner-Peyser), or other programs? At what point in the service sequence does co-enrollment occur?

• How is the delivery of other workforce development services coordinated with the delivery of services funded under the AWI grant? What program pays for what services?

• How is case management of AWI participants handled if they are also enrolled in another program?

6.6 Project Exit

• At what point do individuals officially “exit” the project?

  • ^{Is project exit initiated automatically by the MIS system after 90 days without service contact?}

  • ^{What is usually the last service received before exit?}

  • ^{When does exit occur in relation to the completion of training or placement into a job?}

7. DESIGN OF SPECIFIC SERVICES

Note: Training services are covered in section 8. You might want to cover the training services first and then come back to the topics in this section.

Some services may still be in the planning or pilot stage. Distinguish between active and planned services.

7.1 Career Awareness Information

The US Department of Labor has identified promoting career awareness among aging workers as one potentially important component of a strategy to help aging workers enter jobs in high growth sectors.

• Is providing information on careers in the targeted industries an important aspect of your project?

• Please describe any specific techniques or activities you have created to provide career awareness.

  • ^{How is career awareness integrated into the training and non-training services?}

  • ^{What types of resources (web sites, videos, etc.) are used to develop this awareness?}

  • ^{Are job shadowing or informational session opportunities available to promote career awareness?}

• How does the project rate the quality of career awareness services? How does career information influence participant decisions about training, entry occupations, or career paths?

  • ^{Does the project think that its career information service design is effective? Is it worthy of replication by other projects?}

  • ^{How could career information services for older workers be improved?}

7.2 Assessment Practices

Get copies of assessment tools. If on-line assessment, then ask for screenshots and/or a list of information collected.

• What is the goal of project assessment practices?

• What types of assessments are conducted?

  • ^{Assessment based on case manager or counselor’s interview with participant?}

  • ^{Commercial assessment instruments (what products)?}

  • ^{Products designed by state or LWIA}

  • ^{Was any tool or process developed or modified specifically for this program?}

• How was the decision made regarding which tool to use?

• Please describe the assessment process:

  • ^{Who conducts the assessment?}

  • ^{How long does it take?}

  • ^{When is it conducted?}

  • ^{What information is gathered?}

  • ^{How does the assessment compare to its counterpart under WIA and SCSEP?}

• How is this information used to determine whether participants receive readiness training, education/formal training, job placement services, or other services?

• Are these services funded through the grant, or through leveraged resources?

• How does the project rate the quality of its assessment services?

  • ^{Does the project think that its assessment practices are effective? Are they worthy of replication by other projects?}

  • ^{How could assessment practices be improved?}

7.3 Other “Front-End” Services

• Please describe other “front-end” services, such as pre-employment or pre-training workshops for all participants.

  • ^{What do front-end services consist of?}

  • ^{What is the goal of these services?}

  • ^{How were they developed?}

  • ^{How do they respond to the special needs of older workers?}

• How does the project rate the quality of its front-end services?

  • ^{How could front-end services be improved?}

  • ^{Does the project think that its front-end workshops or services are effective? Are they worthy of replication by other projects?}

7.4 Planning for Employment and Career Pathways

The key issue underlying this section is how the project develops service plans for individual participants and whether it emphasizes planning for longer-term career pathways (including advancement and lateral moves that build on a worker’s transferrable skills) in addition to finding an immediate job.

• Please describe how service plans and employment goals are established for an individual.

  • ^{Are these reflected in a written plan? Do all participants develop a service plan/employment *plan?}

  • ^{Are service plans/employment plans developed for participants who do not participate in training (if any)?}

  • ^{What information is used to develop the employment plan?}

  • ^{Does the plan for services describe both a short term and a long term career goal?}

  • ^{What occupational training and placement goals are available? Are all project participants prepared for the same occupation/industry or is there customer choice involved?}

• What are some examples of service plans/occupational goals for typical customers? (Ask a case manager to talk to you about the planning process and/or show you the file of a recent participant. Is there a plan for career advancement beyond initial employment?)

• How is planning for career pathways and career development built into the planning process?

  • ^{Are participants concerned about career advancement?}

  • ^{Are participants encouraged to move vertically up the career ladder, or laterally across occupations/industries? How does that vary by participant characteristics?}

  • ^{Do participants feel that they are being supported in developing skills for advancement in the targeted occupation?}

  • ^{How has the USDOL emphasis on developing career pathways influenced the project’s approach to service planning and employment goals?}

• To what extent can AWI career planning practices provide a model for other programs and services?

  • ^{For grantees that also serve WIA or SCSEP participants, how does service planning for AWI differ from other practices?}

  • ^{How could service planning for older workers be improved? What could other programs learn from the AWI service planning model?}

  • ^{Are any of the project’s career planning designs and practices more generally applicable to other groups receiving workforce development services?}

7.5 Case Management Practices and Participant Support

• Please describe case management practices for the AWI project.

  • ^{What are the goals of case management for this project?}

  • ^{Are there designated individuals (s) who provide case management services?}

  • ^{When does case management begin and when does it end?}

  • ^{How often and why does the case manager interact with the participant?}

  • ^{What are common topics discussed at case management meetings}

  • ^{What is the most common form of communication between case managers and program participants—in person, telephone?}

  • ^{Are case management and participant support funded through the grant, or through leveraged resources?}

• How does case management differ for participants who enroll in training and participants who do not participate in training? (Are there any active participants who forgo training?)

• What other types of support (e.g. academic counseling, peer support) are provided to project participants

  • ^{What other types of support do older workers need?}

  • ^{How are other types of support provided, by whom, and with what funding?}

  • ^{What types of support seem to work best for engaging participants and keeping them involved in the program?}

• How does the project rate the quality of its case management services?

  • ^{How could case management practices for older workers be improved?}

  • ^{Does the project think that its case management design is effective? Is it worthy of replication by other projects?}

7.6 Job Search Support and Job Placement Services

• Please describe the job search support and job placement services available to AWI participants.

  • ^{What entity (entities) provide job search support and job placement services to AWI grantees (role of grantee, training provider, other partners)?}

  • ^{How do job search/job placement services vary by occupation and/or participant characteristics?}

• How do job search support and placement services for AWI participants draw on resources available through One-Stop service systems?

  • ^{To what extent do AWI participants use placement services available through the local One-Stop system?}

  • ^{To what extent are grant funds used to support job search and job placement services versus leveraged funds from another source?}

7.7 Post-Placement Services

• What are participants’ service needs after they begin working? How does the project respond to these needs?

  • ^{What post-placement services are available to participants? How long are these services available?}

  • ^{Are post-placement services available both to training participants and participants who did not participate in training?}

7.8 Supportive Services and Service Referrals

• Please describe how the project responds to the supportive service needs of AWI participants?

  • ^{What supportive service needs do participants have when they enroll in the project? (e.g. assistance with health or health insurance issues, financial issues, nutrition, housing, disability, or other social service issues)}

  • ^{How does the project respond to these needs? (formal or informal referral linkages; effectiveness of linkages)}

  • ^{What proportions of participants receive supportive services from the project?}

• To what extent are supportive services paid for from project funds versus funds leveraged from other funding streams?

8. Design and Delivery of education and training services

8.1 Training Options Available to AWI Participants

• (Get menu of training options, and information to fill in Training Options Matrix included as Section C).

• What types of education or training opportunities are available to older workers participating in the AWI grant?

  • ^{What is the range of short-term versus longer term training options?}

  • ^{How were these training opportunities developed or selected for se by the project? Did industry representatives or partners participate in selecting the targeted training opportunities?}

  • ^{How do these training choices relate to high-demand occupations, as required by WIA; H1-B industries and occupations, as required for this grant; state or local high growth/high wage criteria?}

  • ^{How are participants notified of the available education and training programs? If there are multiple training options, how do participants select among them?}

  • ^{Do project participants have to pass entry screening tests for specific training programs?}

• Characteristics of training providers most frequently used:

  • ^{What is the nature of the MOU between the training provider(s) and the project administrator?}

  • ^{What is the capacity of the training provider to serve aging workers enrolled in the AWI project?}

  • ^{Do any of the training providers offer separate classes or class sections exclusively for AWI participants? If so, how is the curriculum and- training delivery in these classes modified or designed to meet the needs of aging workers? (e.g. part-time training, technology-based training, independent self-paced study, hands-on learning)}

  • ^{Do training providers participate in recruiting students for these classes?}

  • ^{Do the training programs lead to completion of a certificate or credential? Describe the credential(s) and how it is (they are) perceived by local employers.}

• What payments, if any, can the project make for additional training costs?

  • ^{Does the project pay for additional costs associated with training or first job (e.g. equipment, post-training services such as qualifying exams, externships, etc?)}

  • ^{Does the project offer any living stipend or financial payment during training?}

8.2 Entrepreneurship Training and Services to Support Self-Employment Outcomes

• Does the project provide services to help participants start up a small business?

  • ^{How many participants are interested in this outcome?}

  • ^{Is the grantee partnered or have any type of relationship with Small Business Administration programs? If so, which specific programs and how are they utilized?}

  • ^{What types of entrepreneurial training are available to older workers through the grant?}

  • ^{Is this training available to all participants, or only those thinking of businesses in certain industries/sectors?}

  • ^{Is entrepreneurship training linked to occupational content training or is it a stand-alone training option?}

• How many participants are participating in entrepreneurial training? (Examples?)

• How does the project rate the quality of its entrepreneurial training services?

  • ^{How could the entrepreneurial training be improved?}

  • ^{Does the project think that its entrepreneurial training design is effective? Is it worthy of replication by other projects?}

8.3 Individual Decisions about Training

• How does an individual select a training plan and get approval for it?

  • ^{Can a participant decide not to participate in training and still remain an active enrollee in the project (or are all participants expected to enroll in some kind of occupational training)?}

  • ^{How does the project limit or guide the training choices available to a participant?}

  • ^{What types of screening does the project do to assess whether a participant could succeed in a given training program (e.g. reading or math skills, mobility, strength)?}

  • ^{What are limits on the duration or cost of training? Can exceptions be made?}

• What are the key factors that participants usually consider in deciding whether to enter training and what type of training to participate in?

• Does the project encourage participants to choose training from a specific provider or in a specific industry? (specific programs created or modified for this grant; providers on the eligible training provider list; other)

8.4 Work Experience or Internships for Hands-On Experience During or After Training

• Are internships or temporary work-experience a part of the AWI program design?
  
  
  

  • ^{If yes, describe the types of internship or work-experience placements (e.g. for-profit or non-profit employer, duration, skills gained or practiced during internships, stipend or training pay during work experience)}

  • ^{Who arranges the internships?}

  • ^{How are employers for internship placements recruited and matched to participants?}

• What is the goal of work-based training? (contact with a potential employer, additional opportunities to practice new skills.)

• What proportion of all AWI participants are involved in work-based learning?

• How does the project rate the quality of its work-based training?

  • ^{How could work experience or internships for older workers be improved?}

  • ^{Does the project think that its work-based training model is effective? Is it worthy of replication by other projects?}

8.5 Assessment of Training Options and Providers

• What are the strengths and weaknesses of the training options and providers used by your project?

  • ^{What are the most popular training occupations? Why are these occupations most attractive to the older workers served by the project?}

  • ^{Who are the most frequently used training providers (partners, eligible training provider list vendors, others?) Why are these training providers most attractive to the project participants?}

  • ^{To what extent have particular training providers and courses adapted their usual course content and training approach to make their courses more attractive to or appropriate for older workers?}

  • ^{What are the most innovative or most effective features of training adapted or designed for older workers? (Provide examples)}

  • ^{What are the most problematic or ineffective features of the training for older workers that you have experienced? Why are they problematic? How could they be improved?}

8.6 Curriculum Designs, Pedagogical Approaches, and Training Tools

What types of training approaches are used as part of the training offered to project participants

^{(Possible examples: contextualized learning, particular methods for upgrading specific occupational skills, comprehensive models with wraparound services such as assessment and follow-up)?}

^{What types of technology-based learning (TBL) do you use in training (i.e. chat rooms, webcasts, internet, and computer-based learning, etc.)? How do older workers react to the uses of technology in instructional methods?}

^{How are these training approaches modified to meet the needs of older learners?}

9. Participation by Employers and Incumbent Workers

Besides participating in the formal project partnerships, employers may be involved in designing participant services and making them responsive to particular industrial or employer needs. Firms may also benefit directly from project services, including advice about strategies to increase their retention of older workers who are approaching retirement age.

9.1 Outreach to/ Recruitment of Employers

• How did/does the project reach out to employers?

  • ^{Did the project undertake a public media campaign to make employers aware of the project or change employer attitudes about older workers? How would you assess the effectiveness of that strategy?}

  • ^{What types of employers did you reach out to as part of the initiative? Did you target employers in certain industries or with a certain number of employees?}

• What are your goals for employer involvement?

  • ^{Were you trying to change employer attitudes, identify employers willing to hire older workers, help them adapt jobs to make them attractive to older workers, help them retain workers as they approached retirement age, or something else?}

• What services do/did you offer to employers?

  • ^{Possible examples: technical assistance on how to deal with older workers; services to their aging employees; assessment of older workers’ skills and skills gaps; recruitment and screening of older workers for available jobs; job coaching/ transitional employment for newly hired older workers)}

• How have employers responded to your outreach?

  • ^{What did employers identify as their most pressing concerns related to hiring aging workers?}

  • ^{Were you successful in reaching as many employers as you wanted and the types of employers you wanted?}

  • ^{What were some of the reasons that employers were receptive to the project? “What is in it for them” as an individual employer?}

  • ^{Were the participating employers interested in retaining their incumbent workers who are approaching retirement age? Hiring older workers as new employees?}

  • ^{Were the participating employers interested in promoting training for aging workers or influencing the content of planned training?}

• What would you do differently in the future to reach employers?

9.2 Project Assistance to Employers

• Please provide some examples of how specific employers have been involved with the project.

  • ^{How many employers did you work with? Describe size, industry, and extent of previous contact with public workforce development system.}

  • ^{How were employers involved in the different stages of the project?}

• How did the project assist employers?

  • ^{How did you work with employers to improve their attitudes about older workers as employees, if at all?}

  • ^{How did you work with employers to adapt jobs to make them more attractive to older workers, if at all?}

  • ^{How did you work with employers to support their recruitment and hiring process, if at all (e.g. refer screened or trained older workers to them)?}

  • ^{How did you work with employers to help them retain their employees as they approached retirement age, if at all?}

9.3 Assessment of Employer Outcomes

• How do/might you measure the project’s employer outcomes?

• How did your involvement influence employer attitudes and practices?

  • ^{How did you influence employer attitudes about older workers as employees?}

  • ^{How did you influence employer recruitment and hiring practices with respect to aging workers in general?}

  • ^{Have employers adapted their hiring procedures or job descriptions to make them more attractive to older workers, and if so how?}

  • ^{Did employers you worked with offer any new opportunities for advancement to their older employees?}

• Did employers you worked with hire any project participants as a result of this outreach?

  • ^{How many?}

  • ^{How satisfied were employers with the skills and job performance of project participants they hired?}

  • ^{What types of accommodations by employers were particularly effective in making the job attractive to older workers (e.g. shorter shifts, less physical exertion, more frequent breaks, etc?)}

• Do you think your project will help satisfy the demand for workers in the targeted industries? Why or why not?

9.4 Services Provided to Incumbent Workers

• Please describe your project’s strategy or design for serving incumbent workers, if any?

  • ^{Does your project include any efforts to serve incumbent worker?}

  • ^{What is the purpose of these services? (e.g. skills up*date, career advancement)}

  • ^{What are the desired outcomes?}

• If applicable, describe the scope and timing of services/training to incumbent workers.

  • ^{How were workers recruited or selected for services/training?}

  • ^{How many participants will be served over what time period?}

  • ^{Describe provider and delivery arrangements.}

  • ^{How is the employer involved in services to employed workers? Are there any cost-sharing arrangements? If so, describe.}

• What are the particular challenges or issues involved in providing services/training to older incumbent workers?

• How does the project rate the quality of its services to incumbent workers?

  • ^{How successful have you been in serving aging workers who are already working? (examples or outcome statistics)}

  • ^{How could the services to incumbent workers be improved?}

  • ^{Does the project think that its design for serving incumbent workers is effective? Is it worthy of replication by other projects?}

10. TECHNICAL ASSISTANCE

10.1 Sources of Technical Assistance

• Who do you look to when you have questions about project design or operations?

  • ^{What issues/questions to you refer to USDOL program office?}

  • ^{What issues/questions to you refer to funded TA provider?}

  • ^{What issues/questions do you refer to project peers?}

10.2 Technical Assistance Needs to Date

• Please describe your technical assistance needs to date?

  • ^{What are some of the key challenges you have faced at each phase of project design, organizational design, implementation and operations?}

  • ^{How have your technical assistance needs evolved over time?}

  • ^{What challenges are you currently facing or do you expect to face in the next year with the AWI project?}

• What are your most important TA needs at this point in your project development?

10.3 Technical Assistance on Organizational and Management Issues

• What are some of the challenges you have faced in dealing with organizational and management issues? (e.g., budgeting, record keeping, reporting, developing MOUs with project partners and defining their roles)

  • ^{To what extent has technical assistance helped you deal with these challenges or issues?}

  • ^{Provide details describing your problems and the assistance you received.}

  • ^{Did the TA help resolve these issues?}

• How has the technical assistance you have received influenced your project organization and management approach? How have these changes have improved your project?

10.4 Technical Assistance on Design and Delivery of Program Services

• What are some of the challenges you have faced in dealing with project design and delivery of program services?

  • ^{To what extent has technical assistance helped you with deal with these challenges or issues?}

  • ^{Provide details describing your problems and the assistance you received.}

  • ^{Did the TA help resolve these issues?}

• How has the technical assistance you have received influenced your project design and service delivery procedures? How have these changes have improved your project?

10.5 Assessment of the TA Received to Date

• How satisfied are you with the technical assistance you have received to date?

  • ^{Is the TA you have received responsive to your perceived capacity building needs?}

  • ^{What are the strengths and limitations of your TA coach?}

  • ^{How satisfied are you with the level of involvement of your TA coach? The frequency of contacts?}

• How could the TA you have received been improved in quality or topics covered?

• What are the most useful things you have learned as a result of the TA and training that you have received?

11. PROGRAM FUNDING

11.1 Program Funding

• Tell me about your project budget and any budget issues you have experienced to date.

  • ^{What is the total amount of project funding?}

  • ^{How are funds allocated among partners?}

  • ^{How are grant funds being used by grantees and sub-grantees? (I.e. what service components are they supporting?)}

  • ^{Have you revised the budget, and if so, why?}

11.2 Monetary Leveraged Resources Available to the Program

• Has the project been able to expand its scope as a result of funds contributed by its partner agencies?

  • ^{Specifically, what leveraged resources did each of the grantee or partners bring to the table?}

  • ^{Are these resources in the form of cash, or some other type of support? Are these funds actually included in the project budget/ spending plan?}

  • ^{What are the actual agreements for spending the leveraged funds? Who controls the expenditure of the leveraged resources?}

  • ^{Have these funds been available as promised? If not, how has this affected project operations?}

• What are the effects of the leveraged resources on the Aging Worker Initiative? (e.g., increased number of participants served or range of services provided)

12. DATA COLLECTION AND REPORTING

12.1 Grant Reporting Requirements

• Please comment on the grant reporting requirements and reporting burden. Requirements include submission of Quarterly Financial Report (ETA Form 9130), Quarterly Performance Report (ETA Form 9134) and Quarterly Project Narrative

  • ^{What are the challenges in meeting these reporting requirements? What appears to be working well?}

  • ^{Does the project receive ongoing informational or technical assistance support in completing the grant reporting requirements?}

  • ^{About how much time does it take each month for staff to monitor participant outcomes and financial activities associated with the grant?}

  • ^{Does the reporting accurately capture characteristics, services, and outcomes for your project?}

12.2 Grantees’ Use of Optional Participant MIS System (AWD) and Other MIS Systems

• Is the project using the AWD performance accountability system to record data on participants and outcomes? If not used, why was this decision made?

  • ^{What problems or challenges have been encountered in using this system? Have these problems been resolved?}

  • ^{What types of technical assistance and training did the project receive on the capabilities of the AWD project reporting system?}

  • ^{From whom? (USDOL High Growth Training Initiative Program Office, TA Contractor) How useful was this training?}

• What other system(s) are used to track program data?

  • ^{Are they used in addition to the AWD system or on a stand-alone basis?}

  • ^{If used together, how does this coordination work?}

• Get copies of data items and definitions for systems other than AWD.

• Is the MIS system used to provide periodic reports useful to the project in managing the grant and assessing staff and partner performance? If so, how is this interim data used?

• What additional challenges have you faced related to data collection and reporting?

  • ^{What kinds of technical assistance have you received to help with these challenges?}

  • ^{What additional kinds of technical assistance would be useful?}

12.3 Procedures for Documenting Participant Services

• How do you monitor and track participant services and outcomes?

  • ^{What tasks are involved in this process?}

  • ^{How much staff time does it take each week to monitor and document participants’ service use? Do participants have to submit attendance records or other documents?}

  • ^{Where are participant services recorded? (i.e. in the automated data system(s), hard copy case files, or both)}

  • ^{How standardized is the outcome information in the MIS system and in participant case files?}

  • ^{What challenges have you faced in tracking participant services? How have you addressed those challenges?}

12.4 Procedures for Documenting Participant and Project Outcomes

• What participant outcomes are measured and recorded? (e.g., training completion, degree attainment, employment, wages, job retention, etc.

  • ^{What data sources do you use to document participants’ employment outcomes? (e.g., UI wage records, participant self-reporting, employer confirmation, pay stubs, etc.)?}

  • ^{Where are participant outcomes recorded? (i.e. in the automated data system(s), hard copy case files, or both)}

  • ^{What client-level outcome measures are recorded in the MIS system?}

  • ^{What challenges have you faced in tracking participant outcomes? How have you addressed those challenges?}

• Have you developed any additional measures to document your project’s outcomes?

  • ^{What additional measures is the project measuring?}

  • ^{Have you identified any outcome measures specific to incumbent workers, employers, or participants targeting self-employment? If so, what are they?}

13. Preliminary Information on Program Outcomes

Review the outcomes in the most recent report, compared to the grantees plan and to other grantees. Discuss outcomes to date with project respondents.

13.1 Program Exiters To Date

• How many and what types of participants have exited the program to date?

  • ^{How many participants have exited to date and for what reasons?}

  • ^{What proportion of those exited to date were drop-outs?}

• Based on exiters to date, please describe typical program duration.

  • ^{What is the average duration of program participation (for trainees, for non-trainees, for all participants excluding drop-outs)?}

  • ^{How much variation is there in program duration? What factors affect duration?}

13.2 Participant Outcomes To Date

• What types of jobs and earnings are participants receiving?

  • ^{Are jobs related to the training received or the career guidance provided?}

  • ^{Do these jobs have established career ladders?}

  • ^{Are these jobs consistent with the project’s targeted occupations and industries?}

  • ^{What are the principal factors affecting outcomes for training and non-training participants?}

• How different are these outcomes from outcomes reported for all WIA participants or outcomes reported for all SCSEP participants in the local area as a whole? How might these differences be explained?

13.3 Outcomes on Any Additional Measures To Date

• Please describe outcomes to date on any additional measures.

  • ^{Have you measured outcomes to date for any additional outcome measures (e.g., for incumbent worker training?)}

  • ^{What are the results?}

14. ACTIVITIES TO INCREASE LOCAL SYSTEM CAPACITY TO SERVE OLDER WORKERS

14.1 Strategies to Expand Availability of Services for Aging Workers

• How has the project worked to expand its own capacity to provide workforce development and training services for aging workers? (improvements in quality and quality)

  • ^{How has the project worked to increase the number of slots for older workers in existing training opportunities? To increase the types of training occupations? (How many additional aging workers will be served indirectly as a result of project efforts)}

  • ^{How has the project worked to make program improvements to better serve older workers?}

• How has the project worked to expand the quality and availability of services for aging workers within the local community?

  • ^{Has the project focused on training One-Stop front-line staff to better serve aging workers?}

  • ^{Has the project focused on disseminating its service designs tailored to the needs of aging workers?}

  • ^{What changes, if any, have occurred to date in how local workforce investment systems serve older individuals?}

  • ^{What changes, if any, have occurred in the number of aging workers served by the local workforce investment system?}

14.2 Capacity-Building Activities, Measures, and Outcomes

• What measures does/will the project use to measure its progress in building the capacity of the local system?

  • ^{Numbers of staff trained?}

  • ^{Numbers of aging workers or employers served with expanded capacity?}

  • ^Other?

• Will these increases in the capacity to serve aging workers last beyond the lifetime of the demonstration grant?

14.3 Planned Capacity-Building Products

• How will the project’s promising practices be packaged for dissemination?

  • ^{What specific activities (products, models, curricula, teaching methods, training-the-trainer, licensure or certification requirements) for serving older workers will be operationalized by the workforce system (and by the grantee, if the grantee is not a WIB) after the grant ends?}

  • ^{How will these products be disseminated for use by other entities after the grant ends?}

  • ^{To what user groups are these products directed (e.g., business groups, community colleges, proprietary training providers, labor-management organizations, One-Stop staff)?}

• How will the grant enhance One-Stop Career Center capacity to serve aging workers?

• What are the different “deliverables,” planned by the project?

  • ^{Who will produce the deliverables?}

  • ^{Who will act as an expert reviewer?}

  • ^{What form will the deliverables take?}

14.4 Progress in Completing Planned Deliverables

• Please describe your progress in developing products to support dissemination of your aging worker approach?

  • ^{What progress has been made in completing planned products?}

  • ^{What challenges have been encountered in producing deliverables?}

  • ^{How have plans for deliverables evolved over time?}

  • ^{What is the current schedule for producing deliverables?}

  • ^{What has been your experience to date in the review of products by independent entities? Has this improved the quality of deliverables?}

• What technical assistance on producing deliverables would be useful?

15. SUCCESSES, CHALLENGES, AND LESSONS LEARNED

15.1 Unmet Needs

• To what extent are the needs of older workers still unaddressed in the local community?

  • ^{What service needs of older workers sometimes are still unaddressed? Which are the highest priority needs?}

  • ^{What resources would be necessary to meet these service needs?}

  • ^{What additional services would be most helpful for aging workers?}

15.2 Summary Assessment of Project Strengths and Limitations

• In summary, what are the primary strengths and the primary limitations of the AWI project with respect to:

  • ^{Integration with regional economic talent development?}

  • ^{Organization and partnerships?}

  • ^{Service delivery?}

  • ^{Sustainabillity?}

• What have program participants found most helpful about services provided by grantees? What services were least helpful?

• What would participants like to see changed about this program?

15.3 Project Successes and Practices Worthy of Replication

• Which of projects services have been designed specifically to meet the needs of older workers?

• What recommendations does the project have about the design of services or aging workers?

  • ^{What recommendations does the project have about specific designs that work well for aging workers?}

  • ^{What recommendations does the project have about specific curricula or guides for workshop content that might be available to other projects serving aging workers?}

• What are the practices of this project that show most promise? What aspects of your program would you recommend that other projects emulate?

  • ^{How successful were grantees in recruiting a diverse array of partners?}

  • ^{Which partner relationships were the most successful and why?}

  • ^{What aspects of the program work particularly well in helping participants find and keep jobs?}

• What were the other main successes of this program initiative? How have you achieved these successes?

15.4 Summary of Project Challenges

• What were the key problems or challenges in administering the project?

  • ^{Did you encounter challenges coordinating the input of all partners into account when making key decisions?}

  • ^{Did any of the partnerships fail during the course of the project, and what were the possible reasons?}

  • ^{Did you encounter challenges in recruiting participants?}

• What challenges did you face in serving participants?

  • ^{What challenges did you face in placing participants in jobs?}

  • ^{What challenges did you encounter in helping participants keep and advance in their jobs?}

  • ^{What challenges did you face in tracking participants and recording their outcomes?}

15.5 Lessons Learned

• What are the most important lessons that you have learned as a result of operating the AWI project?

  • ^{What practical lessons and promising practices for the workforce investment system were identified during this project?}

  • ^{Do these lessons apply only to older workers, or more broadly?}

• In hindsight, what would you do differently if you were to start the project again?

15.6 Plans to Expand, Sustain, or Replicate Project Model Within State

Information about sustainability and replicability will be preliminary in the interim report. However, because the grantees have been operating for almost a full year by the time of the site visits, such questions are appropriate for the Round 1 visit. These topics will be very important in Round 2.

• What are the essential program components of a successful AWI service model?

• Do you plan to replicate your program or service model? If so, where and how?

• What advice do you have for replication of your service design for aging workers?

  • ^{Would you recommend this model to others?}

  • ^{What are the key challenges to replicating this model for serving aging workers?}

  • ^{What changes would you recommend others make before replicating your model?}

• For whom do you think your approach is best suited (e.g. under what economic conditions and with what types of aging workers do you think your approach will be effective)?

• What are your strategies to continue the services provided by the grantee after the program ends?

  • ^{To what extent have grantees sought outside funding to continue providing services? How successful have they been? What is the likelihood that services that the grantee provides to older workers will continue after grant funds run out?}

  • ^{From the project’s perspective, what are the highest priority services and service delivery approaches to try to continue after the grant ends?}

• What specific activities, programs, etc. targeted to older workers do you expect will be institutionalized by the public workforce system (or grantee, if grantee is not a WIB or One-Stop)?

  • ^{Do you expect that the workforce system will use WIA resources to provide services to support older workers after the grant ends?}

  • ^{If so, which agencies or staff are good candidates to provide those services?}

Site Visit Topics for Project Policy and Administrative Staff (Round 2)

1. OVERVIEW OF PROGRAM INITIATIVE

1.2 Goals of the Project

• How have the goals and objectives of the project evolved, if at all, since the beginning of the project?

• How have conditions facing aging workers changed since the beginning of the project, and how has this influenced project design and outcomes?

• What outcomes do you hope to achieve for individual participants?

• How are these outcomes measured (common measures, other ways)?

1.4 Target Population

• How have the groups targeted for this program evolved over time and why?

• What are the characteristics of participants enrolled to date? What are their education and skill levels, and level of work experience?

  • ^{Are these participants consistent with the targeted groups?}

  • ^{Have you been surprised by any of the characteristics of the enrolled participants?}

5. Target Sectors/ Industries

• What sectors/industries are targeted by the initiative?

  • ^{How have these sectors changed, if at all, since the project was initiated and why?}

• Why were specific occupations chosen? Were they selected because they were particularly suitable to the needs of older workers)?

• How do the targeted industries and occupations reflect regional economic development strategies?

5. Program Eligibility Requirements

• ^{Have eligibility requirements posed any challenges to program success? Have there been any changes in the policies about who is eligible for the project?}

2. DESCRIPTION OF PROJECT SERVICE AREA

2.1. Description of Local Service Area(s) Targeted for the Project

• Have there been any changes in the service sites within the project’s service area? Are different project partners active in different parts of the service area? Are different entities managing the project in different parts of the service area?

• Has the initiative been implemented uniformly throughout the service area?

2.2 Description of Local Labor Market in the Designated Service Area

• Please describe the local labor market and how it has changed since the project began.

  • ^{What is the local unemployment rate? What are the major industries/employers?}

  • ^{What types of jobs are older workers most likely to have?}

  • ^{What are the sectors of growth and/or decline in the regional economy? For example, are health care jobs, green jobs, etc. considered high growth sectors?}

• How has the recession affected grant planning and implementation?

  • ^{Has the recession had any effect on employer interest in hiring aging workers; in terms of participant interest in enrolling?}

  • ^{Has the recession had any effect on the types of occupations for which participants are trained, or the number of job openings?}

  • ^{What other challenges were created by the economic recession? How were these challenges addressed?}

• How is the project design and implementation experience influenced by the regional labor market and economic trends?

2.4 Competing Initiatives or Programs for Older Workers

• Since the project has been initiated, is there any change in other programs that offer similar services to older workers in the project service area?

  • ^{Are there programs that target some of the same populations as the AWI project or compete with it for enrollees?}

  • ^{Are services offered by other programs similar to or complementary to the services provided by the AWI project?}

  • ^{Could an individual participate in both programs simultaneously?}

• How does the demand for older worker services compare to the capacity of all local programs offering relevant services. (Are all available programs operating at full capacity or are they competing for customers? Are there waiting lists at the grantee and other similar programs?)

3. DESCRIPTION OF GRANTEE

3.1 Grantee Agency Background

• Please describe how the project fits within the grantee organization.

  • ^{Where does project administration reside within the grantee’s organizational structure?}

  • ^{How does the scope and funding of the AWI grant compare in size to the rest of the grantee budget and activities?}

  • ^{Has the location of the AWI project within the grantee organization changed at all since the beginning of the project?}

3.2 Grantee Administrative and Staffing Structure

• What is the overall staffing structure for administering the AWI grant?

  • ^{How has the staffing structure changed, if at all since the first site visit?}

  • ^{Have there been any major changes in project staffing since the first site visit?}

  • ^{Were new staff members hired for the grant or did they come from One-Stop Career Center or partner staff?}

  • ^{If direct service staff work on other programs or projects as well, what proportion of their time or caseload is devoted to AWI participants?}

  • ^{Describe the qualifications and relevant experience of key staff.}

• Is the staffing plan consistent across all entities (e.g. multiple LWIAs) participating in the grant? If not, describe the variations.

4. DEVELOPING PROJECT PARTNERSHIPS

4.1 Outreach/ Identification of Potential Partners

• What is the range of organizations and individuals who are currently participating in the project (e.g. employers, industry associations, educational institutions, training providers, aging organizations, SCSEP grantees, economic development entities, apprenticeship programs, tribal organizations, philanthropic community, community or faith-based organizations)?

4.2 Formal Relationships among Project Partners

• What types of contractual relationships, if any, are in place between grantee and partner agencies?

  • ^{What do those arrangements encompass?}

• What other agreements or procedures govern the relationships between grantee and partners?

4.3 Project Leadership and Oversight

• Describe the project leadership.

  • ^{Have there been any changes in the project leadership since the first site visit?}

  • ^{Have there been any changes in the composition or role of the project advisory board?}

  • ^{Have there been any changes in the composition or role of the project leadership team since the first site visit.}

4.4 Communication between Grantee and Partners

• How does internal project communication work?

  • ^{How does the grantee’s project manager communicate with project partners? How frequently and about what types of issues? How do partners communicate with each other?}

  • ^{How do partners communicate with each other?}

• Does the grantee hold regular meetings with partners? If so, what is the purpose of these meetings? Are these meetings held individually or as a group? How long do they last?

  • ^{What types of information are discussed during these meetings?}

  • ^{How helpful are the meetings in assessing project status and guiding the project?}

• How has communication patterns changed since the first site visit?

4.5 Assessment of Partner Involvement

• How would you characterize the overall success involving partners to participate in the initiative?

  • ^{How pleased are you with the final composition of partners in the initiative?}

  • ^{Were there some agencies you initially wanted to partner with but were unable to do so? If so, what were the barriers in establishing that arrangement? Was another agency involved to fill that gap, and if not, is the initiative lacking in some way without that partner?}

  • ^{Are there any other partners that in hindsight you wish were involved in the initiative that are not involved?}

• Please describe any challenges you have experienced in developing effective relationships between the project partners.

• What are the most successful aspects of your partnership?

• What advice do you have for other projects serving older workers in terms of partnership formation and partner roles?

4.6 Informal Relationships with Other Entities Serving Project Participants

• Please describe any informal relationships that the project has developed with other entities to expand the services available to project participants.

  • ^{To what extent do the workers recruited for this program receive services from other community agencies or programs?}

  • ^{What other agencies and/or programs are involved—SCSEP, One-Stop Centers (VT, CA and TX), Area Agencies on Aging?}

  • ^{To what extent are participants referred to these agencies? What proportion of participants access these services?}

  • ^{To what extent are participants referred from other agencies?}

• What are the experiences of participants referred to these providers?

• Would the project have benefited from more formal relationships with these agencies?

5. Project Partners And Their Roles

5.1 Involvement of the Public Workforce Investment System (e.g. One-Stop Centers and Constituent Programs)

(If grantee is a local WIB, describe the involvement of One-Stop staff and programs that are not directly part of the funded AWI project.

• What role did/do public workforce investment partners play in designing, managing, or overseeing the grant, if any?

• What roles do public workforce investment partners play in providing services to employer or worker customers?

• How much involvement does the AWI project have with other services provided at the One-Stop Career Centers?

• How does the AWI project fit in with other workforce initiatives, such as, SCSEP or Medicaid Infrastructure Grants (MIG)?

• How have public workforce investment partners contributed to the success of the project?

5.2 Involvement of Organizations with Expertise on Aging

• What role did/do “aging organizations” play in designing, managing, or overseeing the grant, if any?

• What other roles do “aging organizations” play in providing services to employer or worker customers?

• How have “aging organizations” contributed to the success of the project?

5.3 Involvement of Educational Institutions and Training Providers

• What role did/do educational institutions or training providers play in designing, managing, or overseeing the grant, if any?

• What other roles do educational institutions or training providers play in providing services to employer or worker customers?

• How have education and training institutions or training partners contributed to the success of the project?

5.4 Involvement of Economic Development Entities

• What role did/do economic development entities play in designing, managing, or overseeing the grant, if any?

• What other roles do economic development entities play in providing services to employer or worker customers?

• How have economic development entities contributed to the success of the project?

5.5 Involvement of Local Employers, Employer Associations, or Business Intermediaries

• What role did/do employers or business intermediaries play in designing, managing, or overseeing the grant, if any?

• How did you recruit business partners? What were your selection criteria, if any?

• What other roles do employers or business intermediaries play in providing services to employer or worker customers? Are they involved in services to their own incumbent workers; in directly recruiting and hiring project participants, or in some more general way?)

• How have employers or business intermediaries contributed to the success of the project?

• How has employer involvement been affected by the economic recession?

5.6 Involvement of Other Partners
(e.g., faith-based organizations, community organizations, philanthropic institutions, apprenticeship programs, tribal organizations, SCSEPgrantees)

• What role did/do other partners play in designing, managing, or overseeing the grant, if any?

• What other roles do other partners play in providing services to employer or worker customers?

• How have other partners contributed to the success of the project?

6. Overview and Sequencing of Services

6.1 Participant Recruitment and Referral

• How are participants recruited to the program?

  • ^{What proportion of participants are recruited through grant-specific outreach? (How do you advertise the project (e.g. brochures, public service announcements, speakers, requests for referrals)?}

  • ^{What proportion of participants are referred by:}

    • ^{One-Stop Career Centers?}

    • ^{Local education providers?}

    • ^{Other partners in the grant?}

    • ^{Other means}

• What type of special emphasis is there on recruiting disadvantaged populations (veterans, people with disabilities, military spouses, ex-offenders, minorities, new Americans)?

• Did you face any challenges in recruiting participants?

  • ^{Were there any delays in the start of participant enrollment?}

  • ^{If yes, what caused them?}

  • ^{What strategies did you use to overcome those challenges?}

• To what extent and how is recruitment of older workers linked to employer requirements?

6.2 Enrollment in Project

• At what point in the receipt of services is an individual officially “enrolled” in the project (reportable to USDOL as a participant)?

  • ^{Does receipt of a specific service or participation in a specific activity automatically activate “enrollment?”}

  • ^{Is enrollment reserved for individuals who decide to participate in occupational skills training?}

  • ^{If not, what is the ratio of enrollees who participate in training to enrollees who do not participate in training?}

  • ^{Do all training options include occupational skills training? (Or do some participants receive only pre-employment training or only job search training or only basic skills training without occupational skills training?)}

  • ^{Have there been any changes in project enrollment practices since the first site visit?}
    
    
    

6.3 Service Components (including Orientation and Pre-Employment Services)

• What are the different service components developed by the project to meet the needs of aging workers served by the project? (E.g., orientation, assessment, service planning/career counseling, pre-employment training, skills training, academic counseling, internships or temporary work experience, job search/job placement, post-placement services)

  • ^{What is the duration and content of each service?}

  • ^{Who provides each service?}

  • ^{Is each service provided using grant funding or through leveraged funds by the providing agency?}

• What changes have occurred in the service components over time? (new services added, services redesigned, services discontinued) Why and how?

6.4 Sequencing of Services

• What is the typical sequence of project services (variations depending on customer needs?)

  • ^{What services have participants received from other sources before enrolling in AWI? Describe the depth and quality.}

  • ^{How long, on average, do participants remain active in the project?}

  • ^{To what extent do participants drop out of service delivery? At what point in service delivery do clients typically drop out? For what reasons? What type of follow-up is made with drop outs to encourage continuing participation or obtain outcome information?}

  • ^{For participants who stay with the project, what is the point (points) at which an individual is considered to have “completed” the project?}

  • ^{At what point in the service process are individuals considered to have exited the project?}

  • ^{What follow-up services are provided after project exit? When does follow-up end?}

• What is the frequency of different services?

  • ^{What project services, if any, are received by all participants? Do all participants receive occupational skills training? Pre-employment training? Other services?}

  • ^{What services are received by only a portion of all participants? What determines whether a customer will receive a given service?}

  • ^{What other types of services, including WIA-funded services are provided to AWI participants? How many individuals received these services (by type)?}

• What individuals and entities are responsible for the delivery of different services?

  • ^{In the course of participation, what project staff will participants come into contact with? Who does each of these service providers work for?}

  • ^{Do participants have to travel to different locations for different services?}

  • ^{If participants are referred to an education and training, or other project partner, what are the roles and responsibilities of the grantee? Who maintains the case—the grantee, the partner, or both?}

6.5 Co-enrollment of AWI Project Participants in Other Workforce Development Programs

• How frequently are AWI project participants provided with an orientation to the core One-Stop services (e.g. resource room and online labor market information and assessment tools) as part of their participation in the AWI project?

• How frequently are AWI project participants co-enrolled in other programs operated out of One-Stop centers (e.g. WIA, TAA. SCSEP, Employment Services (Wagner-Peyser), or other programs? At what point in the service sequence does co-enrollment occur?

• How is the delivery of other workforce development services coordinated with the delivery of services funded under the AWI grant? What program pays for what services?

• How is case management of AWI participants handled if they are also enrolled in another program?

6.6 Project Exit

• At what point do individuals officially “exit” the project?

  • ^{Is project exit initiated automatically by the MIS system after 90 days without service contact?}

  • ^{What is usually the last service received before exit?}

  • ^{When does exit occur in relation to the completion of training or placement into a job?}

7. DESIGN OF SPECIFIC SERVICES

Note: Training services are covered in section 8. You might want to cover the training services first and then come back to the topics in this section.

Some services may still be in the planning or pilot stage. Distinguish between active and planned services.

7.1 Career Awareness Information

The US Department of Labor has identified promoting career awareness among aging workers as one potentially important component of a strategy to help aging workers enter jobs in high growth sectors.

• Is providing information on careers in the targeted industries an important aspect of your project?

• Please describe any specific techniques or activities you have created to provide career awareness.

• How does the project rate the quality of career awareness services? How does career information influence participant decisions about training, entry occupations, or career paths?

  • ^{Does the project think that its career information service design is effective? Is it worthy of replication by other projects?}

  • ^{How could career information services for older workers be improved?}

7.2 Assessment Practices

Get copies of assessment tools. If on-line assessment, then ask for screenshots and/or a list of information collected.

• What is the goal of project assessment practices?

• What types of assessments are conducted?

  • ^{Assessment based on case manager or counselor’s interview with participant?}

  • ^{Commercial assessment instruments (what products)?}

  • ^{Products designed by state or LWIA}

  • ^{Was any tool or process developed or modified specifically for this program?}

• How was the decision made regarding which tool to use?

• Please describe the assessment process:

• How is this information used to determine whether participants receive readiness training, education/formal training, job placement services, or other services?

• Are these services funded through the grant, or through leveraged resources?

• How does the project rate the quality of its assessment services?

  • ^{Does the project think that its assessment practices are effective? Are they worthy of replication by other projects?}

  • ^{How could assessment practices be improved?}

7.3 Other “Front-End” Services

• Please describe other “front-end” services, such as pre-employment or pre-training workshops for all participants.

  • ^{How do they respond to the special needs of older workers?}

• How does the project rate the quality of its front-end services?

  • ^{How could front-end services be improved?}

  • ^{Does the project think that its front-end workshops or services are effective? Are they worthy of replication by other projects?}

7.4 Planning for Employment and Career Pathways

The key issue underlying this section is how the project develops service plans for individual participants and whether it emphasizes planning for longer-term career pathways (including advancement and lateral moves that build on a worker’s transferrable skills) in addition to finding an immediate job.

• Please describe how service plans and employment goals are established for an individual.

• What are some examples of service plans/occupational goals for typical customers? (Ask a case manager to talk to you about the planning process and/or show you the file of a recent participant. Is there a plan for career advancement beyond initial employment?)

• How is planning for career pathways and career development built into the planning process?

• To what extent can AWI career planning practices provide a model for other programs and services?

7.5 Case Management Practices and Participant Support

• Please describe case management practices for the AWI project.

• How does case management differ for participants who enroll in training and participants who do not participate in training? (Are there any active participants who forgo training?)

• What other types of support (e.g. academic counseling, peer support) are provided to project participants

  • ^{What other types of support do older workers need?}

  • ^{How are other types of support provided, by whom, and with what funding?}

  • ^{What types of support seem to work best for engaging participants and keeping them involved in the program?}

• How does the project rate the quality of its case management services?

  • ^{How could case management practices for older workers be improved?}

  • ^{Does the project think that its case management design is effective? Is it worthy of replication by other projects?}

7.6 Job Search Support and Job Placement Services

• Please describe the job search support and job placement services available to AWI participants.

  • ^{What entity (entities) provide job search support and job placement services to AWI grantees (role of grantee, training provider, other partners)?}

  • ^{How do job search/job placement services vary by occupation and/or participant characteristics?}

• How do job search support and placement services for AWI participants draw on resources available through One-Stop service systems?

  • ^{To what extent do AWI participants use placement services available through the local One-Stop system?}

  • ^{To what extent are grant funds used to support job search and job placement services versus leveraged funds from another source?}

7.7 Post-Placement Services

• What are participants’ service needs after they begin working? How does the project respond to these needs?

  • ^{What post-placement services are available to participants? How long are these services available?}

  • ^{Are post-placement services available both to training participants and participants who did not participate in training?}

7.8 Supportive Services and Service Referrals

• Please describe how the project responds to the supportive service needs of AWI participants?

  • ^{What supportive service needs do participants have when they enroll in the project? (e.g. assistance with health or health insurance issues, financial issues, nutrition, housing, disability, or other social service issues)}

  • ^{How does the project respond to these needs? (formal or informal referral linkages; effectiveness of linkages)}

  • ^{What proportion of participants receive supportive services from the project?}

• To what extent are supportive services paid for from project funds versus funds leveraged from other funding streams?

8. Design and Delivery of education and training services

8.1 Training Options Available to AWI Participants

• What types of education or training opportunities are available to older workers participating in the AWI grant?

• Characteristics of training providers most frequently used:

  • ^{Do any of the training providers offer separate classes or class sections exclusively for AWI participants? If so, how is the curriculum and- training delivery in these classes modified or designed to meet the needs of aging workers? (e.g. part-time training, technology-based training, independent self-paced study, hands-on learning)}

  • ^{Do training providers participate in recruiting students for these classes?}

  • ^{Do the training programs lead to completion of a certificate or credential? Describe the credential(s) and how it is(they are) perceived by local employers}

8.2 Entrepreneurship Training and Services to Support Self-Employment Outcomes

• Does the project provide services to help participants start up a small business?

  • ^{Have the number of participants interested in or participating in entrepreneurship changes since the first site visit?}

  • ^{Is entrepreneurship training linked to occupational content training or is it a stand-alone training option?}

• How many participants are participating in entrepreneurial training? (Examples?)

• How does the project rate the quality of its entrepreneurial training services?

  • ^{How could the entrepreneurial training be improved?}

  • ^{Does the project think that its entrepreneurial training design is effective? Is it worthy of replication by other projects?}

8.3 Individual Decisions about Training

• How does an individual select a training plan and get approval for it?

• What are the key factors that participants usually consider in deciding whether to enter training and what type of training to participate in?

• Does the project encourage participants to choose training from a specific provider or in a specific industry? (specific programs created or modified for this grant; providers on the eligible training provider list; other)

8.4 Work Experience or Internships for Hands-On Experience during or After Training

• Are internships or temporary work-experience a part of the AWI program design?
  
  
  

• What is the goal of work-based training? (Contact with a potential employer, additional opportunities to practice new skills.)

• What proportion of all AWI participants are involved in work-based learning?

• How does the project rate the quality of its work-based training?

  • ^{How could work experience or internships for older workers be improved?}

  • ^{Does the project think that its work-based training model is effective? Is it worthy of replication by other projects?}

8.5 Assessment of Training Options and Providers

• What are the strengths and weaknesses of the training options and providers used by your project?

  • ^{What are the most innovative or most effective features of training adapted or designed for older workers? (Provide examples)}

  • ^{What are the most problematic or ineffective features of the training for older workers that you have experienced? Why are they problematic? How could they be improved?}

8.6 Curriculum Designs, Pedagogical Approaches, and Training Tools

What types of training approaches are used as part of the training offered to project participants

^{(Possible examples: contextualized learning, particular methods for upgrading specific occupational skills, comprehensive models with wraparound services such as assessment and follow-up)?}

^{What types of technology-based learning (TBL) do you use in training (i.e. chat rooms, webcasts, internet, and computer-based learning, etc.)? How do older workers react to the uses of technology in instructional methods?}

^{How are these training approaches modified to meet the needs of older learners?}

9. Participation by Employers and Incumbent Workers

Besides participating in the formal project partnerships, employers may be involved in designing participant services and making them responsive to particular industrial or employer needs. Firms may also benefit directly from project services, including advice about strategies to increase their retention of older workers who are approaching retirement age.

9.1 Outreach to/ Recruitment of Employers

• How did/does the project reach out to employers?

  • ^{Did the project undertake a public media campaign to make employers aware of the project or change employer attitudes about older workers? How would you assess the effectiveness of that strategy?}

  • ^{What types of employers did you reach out to as part of the initiative? Did you target employers in certain industries or with a certain number of employees?}

• What are your goals for employer involvement?

  • ^{Were you trying to change employer attitudes, identify employers willing to hire older workers, help them adapt jobs to make them attractive to older workers, help them retain workers as they approached retirement age, or something else?}

• What services do/did you offer to employers?

  • ^{Possible examples: technical assistance on how to deal with older workers; services to their aging employees; assessment of older workers’ skills and skills gaps; recruitment and screening of older workers for available jobs; job coaching/ transitional employment for newly hired older workers)}

• How have employers responded to your outreach?

• What would you do differently in the future to reach employers?

9.2 Project Assistance to Employers

• Please provide some examples of how specific employers have been involved with the project.

  • ^{How many employers did you work with? Describe size, industry, and extent of previous contact with public workforce development system.}

  • ^{How were employers involved in the different stages of the project?}

• How did the project assist employers?

9.3 Assessment of Employer Outcomes

• How do/might you measure the project’s employer outcomes?

• How did your involvement influence employer attitudes and practices?

  • ^{How did you influence employer attitudes about older workers as employees?}

  • ^{How did you influence employer recruitment and hiring practices with respect to aging workers in general?}

  • ^{Have employers adapted their hiring procedures or job descriptions to make them more attractive to older workers, and if so how?}

  • ^{Did employers you worked with offer any new opportunities for advancement to their older employees?}

• Did employers you worked with hire any project participants as a result of this outreach?

  • ^{How many?}

  • ^{How satisfied were employers with the skills and job performance of project participants they hired?}

  • ^{What types of accommodations by employers were particularly effective in making the job attractive to older workers (e.g. shorter shifts, less physical exertion, more frequent breaks, etc?)}

• Do you think your project will help satisfy the demand for workers in the targeted industries? Why or why not?

9.4 Services Provided to Incumbent Workers

• Please describe your project’s strategy or design for serving incumbent workers, if any?

• If applicable, describe the scope and timing of services/training to incumbent workers.

  • ^{How were workers recruited or selected for services/training?}

  • ^{How many participants will be served over what time period?}

  • ^{Describe provider and delivery arrangements.}

  • ^{How is the employer involved in services to employed workers? Are there any cost-sharing arrangements? If so, describe.}

• What are the particular challenges or issues involved in providing services/training to older incumbent workers?

• How does the project rate the quality of its services to incumbent workers?

  • ^{How successful have you been in serving aging workers who are already working? (examples or outcome statistics)}

  • ^{How could the services to incumbent workers be improved?}

  • ^{Does the project think that its design for serving incumbent workers is effective? Is it worthy of replication by other projects?}

10. TECHNICAL ASSISTANCE

10.1 Sources of Technical Assistance

• Who do you look to when you have questions about project design or operations?

  • ^{What issues/questions to you refer to USDOL program office?}

  • ^{What issues/questions to you refer to funded TA provider?}

  • ^{What issues/questions do you refer to project peers?}

10.2 Technical Assistance Needs to Date

• Please describe your technical assistance needs to date?

  • ^{What are some of the key challenges you have faced at each phase of project design, organizational design, implementation and operations?}

  • ^{How have your technical assistance needs evolved over time?}

  • ^{What challenges are you currently facing or do you expect to face in the next year with the AWI project?}

• What are your most important TA needs at this point in your project development?

10.3 Technical Assistance on Organizational and Management Issues

• What are some of the challenges you have faced in dealing with organizational and management issues? (e.g., budgeting, record keeping, reporting, developing MOUs with project partners and defining their roles)

  • ^{To what extent has technical assistance helped you deal with these challenges or issues?}

  • ^{Provide details describing your problems and the assistance you received.}

  • ^{Did the TA help resolve these issues?}

• How has the technical assistance you have received influenced your project organization and management approach? How have these changes have improved your project?

10.4 Technical Assistance on Design and Delivery of Program Services

• What are some of the challenges you have faced in dealing with project design and delivery of program services?

  • ^{To what extent has technical assistance helped you with deal with these challenges or issues?}

  • ^{Provide details describing your problems and the assistance you received.}

  • ^{Did the TA help resolve these issues?}

• How has the technical assistance you have received influenced your project design and service delivery procedures? How have these changes have improved your project?

10.5 Assessment of the TA Received to Date

• How satisfied are you with the technical assistance you have received to date?

  • ^{Is the TA you have received responsive to your perceived capacity building needs?}

  • ^{What are the strengths and limitations of your TA coach?}

  • ^{How satisfied are you with the level of involvement of your TA coach? The frequency of contacts?}

• How could the TA you have received been improved in quality or topics covered?

• What are the most useful things you have learned as a result of the TA and training that you have received?

11. PROGRAM FUNDING

11.1 Program Funding

• Tell me about your project budget and any budget issues you have experienced to date.

  • ^{What is the total amount of project funding?}

  • ^{How are funds allocated among partners?}

  • ^{How are grant funds being used by grantees and sub-grantees? (I.e. what service components are they supporting?)}

  • ^{Have you revised the budget, and if so, why?}

11.2 Monetary Leveraged Resources Available to the Program

• Has the project been able to expand its scope as a result of funds contributed by its partner agencies?

  • ^{Specifically, what leveraged resources did each of the grantee or partners bring to the table?}

  • ^{Are these resources in the form of cash, or some other type of support? Are these funds actually included in the project budget/ spending plan?}

  • ^{What are the actual agreements for spending the leveraged funds? Who controls the expenditure of the leveraged resources?}

  • ^{Have these funds been available as promised? If not, how has this affected project operations}

12. DATA COLLECTION AND REPORTING

12.1 Grant Reporting Requirements

• Please comment on the grant reporting requirements and reporting burden. Requirements include submission of Quarterly Financial Report (ETA Form 9130), Quarterly Performance Report (ETA Form 9134) and Quarterly Project Narrative

  • ^{What are the challenges in meeting these reporting requirements? What appears to be working well?}

  • ^{Does the project receive ongoing informational or technical assistance support in completing the grant reporting requirements?}

  • ^{About how much time does it take each month for staff to monitor participant outcomes and financial activities associated with the grant?}

  • ^{Does the reporting accurately capture characteristics, services, and outcomes for your project?}

12.2 Grantees’ Use of Optional Participant MIS System (AWD) and Other MIS Systems

• Is the project using the AWD performance accountability system to record data on participants and outcomes? If not used, why was this decision made?

  • ^{What problems or challenges have been encountered in using this system? Have these problems been resolved?}

  • ^{What types of technical assistance and training did the project receive on the capabilities of the AWD project reporting system?}

  • ^{From whom? (USDOL High Growth Training Initiative Program Office, TA Contractor) How useful was this training?}

• What other system(s) are used to track program data?

  • ^{Are they used in addition to the AWD system or on a stand-alone basis?}

  • ^{If used together, how does this coordination work?}

• Get copies of data items and definitions for systems other than AWD.

• Is the MIS system used to provide periodic reports useful to the project in managing the grant and assessing staff and partner performance? If so, how is this interim data used?

• What additional challenges have you faced related to data collection and reporting?

  • ^{What kinds of technical assistance have you received to help with these challenges?}

  • ^{What additional kinds of technical assistance would be useful?}

12.3 Procedures for Documenting Participant Services

• How do you monitor and track participant services and outcomes?

  • ^{What tasks are involved in this process?}

  • ^{How much staff time does it take each week to monitor and document participants’ service use? Do participants have to submit attendance records or other documents?}

  • ^{Where are participant services recorded? (i.e. in the automated data system(s), hard copy case files, or both)}

  • ^{How standardized is the outcome information in the MIS system and in participant case files?}

  • ^{What challenges have you faced in tracking participant services? How have you addressed those challenges?}

13. Preliminary Information on Program Outcomes

Review the outcomes in the most recent report, compared to the grantees plan and to other grantees. Discuss outcomes to date with project respondents.

13.1 Program Exiters To Date

• How many and what types of participants have exited the program to date?

  • ^{How many participants have exited to date and for what reasons?}

  • ^{What proportion of those exited to date were drop-outs?}

• Based on exiters to date, please describe typical program duration.

  • ^{What is the average duration of program participation (for trainees, for non-trainees, for all participants excluding drop-outs)?}

  • ^{How much variation is there in program duration? What factors affect duration?}

13.2 Participant Outcomes To Date

• What types of jobs and earnings are participants receiving?

  • ^{Are jobs related to the training received or the career guidance provided?}

  • ^{Do these jobs have established career ladders?}

  • ^{Are these jobs consistent with the project’s targeted occupations and industries?}

  • ^{What are the principal factors affecting outcomes for training and non-training participants?}

• How different are these outcomes from outcomes reported for all WIA participants or outcomes reported for all SCSEP participants in the local area as a whole? How might these differences be explained?

13.3 Outcomes on Any Additional Measures To Date

• Please describe outcomes to date on any additional measures.

  • ^{Have you measured outcomes to date for any additional outcome measures (e.g., for incumbent worker training?)}

  • ^{What are the results?}

14. ACTIVITIES TO INCREASE LOCAL SYSTEM CAPACITY TO SERVE OLDER WORKERS

14.1 Strategies to Expand Availability of Services for Aging Workers

• How has the project worked to expand its own capacity to provide workforce development and training services for aging workers? (improvements in quality and quality)

  • ^{How has the project worked to increase the number of slots for older workers in existing training opportunities? To increase the types of training occupations? (How many additional aging workers will be served indirectly as a result of project efforts)}

  • ^{How has the project worked to make program improvements to better serve older workers?}

• How has the project worked to expand the quality and availability of services for aging workers within the local community?

  • ^{What changes, if any, have occurred to date in how local workforce investment systems serve older individuals?}

  • ^{What changes, if any, have occurred in the number of aging workers served by the local workforce investment system?}

14.2 Capacity-Building Activities, Measures, and Outcomes

• What measures does/will the project use to measure its progress in building the capacity of the local system?

  • ^{Numbers of staff trained?}

  • ^{Numbers of aging workers or employers served with expanded capacity?}

  • ^Other?

• Will these increases in the capacity to serve aging workers last beyond the lifetime of the demonstration grant?

14.3 Planned Capacity-Building Products

• How will the project’s promising practices be packaged for dissemination?

  • ^{What specific activities (products, models, curricula, teaching methods, training-the-trainer, licensure or certification requirements) for serving older workers will be operationalized by the workforce system (and by the grantee, if the grantee is not a WIB) after the grant ends?}

  • ^{How will these products be disseminated for use by other entities after the grant ends?}

  • ^{To what user groups are these products directed (e.g., business groups, community colleges, proprietary training providers, labor-management organizations, One-Stop staff)?}

• How will the grant enhance One-Stop Career Center capacity to serve aging workers?

• What are the different “deliverables,” planned by the project?

  • ^{Who will produce the deliverables?}

  • ^{Who will act as an expert reviewer?}

  • ^{What form will the deliverables take?}

14.4 Progress in Completing Planned Deliverables

• Please describe your progress in developing products to support dissemination of your aging worker approach?

  • ^{What progress has been made in completing planned products?}

  • ^{What challenges have been encountered in producing deliverables?}

  • ^{How have plans for deliverables evolved over time?}

  • ^{What is the current schedule for producing deliverables?}

  • ^{What has been your experience to date in the review of products by independent entities? Has this improved the quality of deliverables?}

• What technical assistance on producing deliverables would be useful?

15. SUCCESSES, CHALLENGES, AND LESSONS LEARNED

15.1 Unmet Needs

• To what extent are the needs of older workers still unaddressed in the local community?

  • ^{What service needs of older workers sometimes are still unaddressed? Which are the highest priority needs?}

  • ^{What resources would be necessary to meet these service needs?}

  • ^{What additional services would be most helpful for aging workers?}

15.2 Summary Assessment of Project Strengths and Limitations

• In summary, what are the primary strengths and the primary limitations of the AWI project with respect to:

  • ^{Integration with regional economic talent development?}

  • ^{Organization and partnerships?}

  • ^{Service delivery?}

  • ^{Sustainabillity?}

• What have program participants found most helpful about services provided by grantees? What services were least helpful?

• What would participants like to see changed about this program?

15.3 Project Successes and Practices Worthy of Replication

• Which of projects services have been designed specifically to meet the needs of older workers?

• What recommendations does the project have about the design of services for aging workers?

  • ^{What recommendations does the project have about specific designs that work well for aging workers?}

  • ^{What recommendations does the project have about specific curricula or guides for workshop content that might be available to other projects serving aging workers?}

• What are the practices of this project that show most promise? What aspects of your program would you recommend that other projects emulate?

  • ^{How successful were grantees in recruiting a diverse array of partners?}

  • ^{Which partner relationships were the most successful and why?}

  • ^{What aspects of the program work particularly well in helping participants find and keep jobs?}

• What were the other main successes of this program initiative? How have you achieved these successes?

15.4 Summary of Project Challenges

• What were the key problems or challenges in administering the project?

• What challenges did you face in serving participants?

  • ^{What challenges did you face in placing participants in jobs?}

  • ^{What challenges did you encounter in helping participants keep and advance in their jobs?}

  • ^{What challenges did you face in tracking participants and recording their outcomes?}

15.5 Lessons Learned

• What are the most important lessons that you have learned as a result of operating the AWI project?

  • ^{What practical lessons and promising practices for the workforce investment system were identified during this project?}

  • ^{Do these lessons apply only to older workers, or more broadly?}

• In hindsight, what would you do differently if you were to start the project again?

15.6 Plans to Expand, Sustain, or Replicate Project Model Within State

Information about sustainability and replicability will be preliminary in the interim report. However, because the grantees have been operating for almost a full year by the time of the site visits, such questions are appropriate for the Round 1 visit. These topics will be very important in Round 2.

• What are the essential program components of a successful AWI service model?

• Do you plan to replicate your program or service model? If so, where and how?

• What advice do you have for replication of your service design for aging workers?

  • ^{Would you recommend this model to others?}

  • ^{What are the key challenges to replicating this model for serving aging workers?}

  • ^{What changes would you recommend others make before replicating your model?}

• For whom do you think your approach is best suited (e.g. under what economic conditions and with what types of aging workers do you think your approach will be effective)?

• What are your strategies to continue the services provided by the grantee after the program ends?

  • ^{To what extent have grantees sought outside funding to continue providing services? How successful have they been? What is the likelihood that services that the grantee provides to older workers will continue after grant funds run out?}

  • ^{From the project’s perspective, what are the highest priority services and service delivery approaches to try to continue after the grant ends?}

• What specific activities, programs, etc. targeted to older workers do you expect will be institutionalized by the public workforce system (or grantee, if grantee is not a WIB or One-Stop)?

  • ^{Do you expect that the workforce system will use WIA resources to provide services to support older workers after the grant ends?}

  • ^{If so, which agencies or staff are good candidates to provide those services?}

Site Visit Topics for Direct Service Delivery Staff and Supervisors (Round 1)

1. OVERVIEW OF PROGRAM INITIATIVE

1.2 Goals of the Project

• How would you describe your project philosophy or approach?

• What program goals do you hope to achieve? To what extent are these goals quantifiable?

  • ^{How could these goals be assessed? (How will you know if you are successful?)}

  • ^{Have the goals of the project evolved or changed since the grant began?}

• What outcomes do you hope to achieve for individual participants?

• How are these outcomes measured (common measures, other ways)?

1.4 Target Population

• What specific groups of aging workers does your project target (age, employment status, previous work history, other characteristics)?

  • ^{Why were these particular groups targeted for this program in your area?}

  • ^{Among the target participants, what are the most common challenges to getting and keeping a job?}

• What are the characteristics of participants enrolled to date? What are their education and skill levels, and level of work experience?

  • ^{Are these participants consistent with the targeted groups?}

  • ^{Have you been surprised by any of the characteristics of the enrolled participants?}

1.6. Program Eligibility Requirements

• What are the eligibility requirements for participation in the program?

  • ^{Are there any additional restrictions or goals beyond the grant requirement (age 55+)? Do participants need to be currently unemployed or employed in a particular industry?}

  • ^{Have eligibility requirements posed any challenges to program success?}

2. DESCRIPTION OF PROJECT SERVICE AREA

2.2 Description of Local Labor Market in the Designated Service Area

• Please describe the local labor market.

  • ^{What is the local unemployment rate? What are the major industries/employers?}

  • ^{What types of jobs are older workers most likely to have?}

  • ^{What are the sectors of growth and/or decline in the regional economy? For example, are health care jobs, green jobs, etc. considered high growth sectors?}

• How has the recession affected grant planning and implementation?

  • ^{Has the recession had any effect on employer interest in hiring aging workers; in terms of participant interest in enrolling?}

  • ^{Has the recession had any effect on the types of occupations for which participants are trained, or the number of job openings?}

  • ^{What other challenges were created by the economic recession? How were these challenges addressed?}

• How is the project design and implementation experience influenced by the regional labor market and economic trends?

2.3 Description of Other Services Available to Older Workers in the Service Area

• Before this grant, what employment and training services existed for older workers?

  • ^{What services for older workers were available from public workforce development funds, e.g., from WIA adult or dislocated worker funding stream; SCSEP; WIA or other incumbent worker training?}

  • ^{Did other agencies or funding streams also support employment and training services for older workers? If so, provide details.}

  • ^{Were there any public training funds targeted to currently employed or retired workers?}

• Did existing training services make any special arrangements to make their services appropriate for older workers? (e.g. accommodations for people with disabilities; changes in working hours or conditions?)

• What factors, if any, limited the services available to older workers from the general workforce development system?

4. DEVELOPING PROJECT PARTNERSHIPS

4.4 Communication Between Grantee and Partners

• How does internal project communication work?

  • ^{How does the grantee’s project manager communicate with project partners? How frequently and about what types of issues? How do partners communicate with each other?}

  • ^{How do partners communicate with each other?}

• Does the grantee hold regular meetings with partners? If so, what is the purpose of these meetings? Are these meetings held individually or as a group? How long do they last?

  • ^{What types of information are discussed during these meetings?}

  • ^{How helpful are the meetings in assessing project status and guiding the project?}

4.5 Assessment of Partner Involvement

• How would you characterize the overall success involving partners to participate in the initiative?

  • ^{How pleased are you with the final composition of partners in the initiative?}

  • ^{Were there some agencies you initially wanted to partner with but were unable to do so? If so, what were the barriers in establishing that arrangement? Was another agency involved to fill that gap, and if not, is the initiative lacking in some way without that partner?}

  • ^{Are there any other partners that in hindsight you wish were involved in the initiative that are not involved?}

• Please describe any challenges you have experienced in developing effective relationships between the project partners.

  • ^{What challenges did you experience in recruiting partners or in arranging for their specific role in the project?}

  • ^{What challenges did you experience in the quality of the contributions made by individual partners?}

• What are the most successful aspects of your partnership?

• What advice do you have for other projects serving older workers in terms of partnership formation and partner roles?

4.6 Informal Relationships with Other Entities Serving Project Participants

• Please describe any informal relationships that the project has developed with other entities to expand the services available to project participants.

  • ^{To what extent do the workers recruited for this program receive services from other community agencies or programs?}

  • ^{What other agencies and/or programs are involved—SCSEP, One-Stop Centers (VT, CA and TX), Area Agencies on Aging?}

  • ^{To what extent are participants referred to these agencies? What proportion of participants access these services?}

  • ^{To what extent are participants referred from other agencies?}

• What are the experiences of participants referred to these providers?

• Would the project have benefited from more formal relationships with these agencies?

5. PROJECT PARTNERS AND THEIR ROLES

5.1 Involvement of the Public Workforce Investment System (e.g. One-Stop Centers and Constituent Programs)

(If grantee is a local WIB, describe the involvement of One-Stop staff and programs that are not directly part of the funded AWI project.

• What role did/do public workforce investment partners play in designing, managing, or overseeing the grant, if any?

• What roles do public workforce investment partners play in providing services to employer or worker customers?

• How much involvement does the AWI project have with other services provided at the One-Stop Career Centers?

  • ^{Do AWI project participants utilize WIA core and intensive services? If so, for what types of services? Are workers served under AWI co-enrolled in the One-Stop system (or SCSEP) under WIA?}

  • ^{Are other workforce investment partners involved in providing services to AWI project participants? What is the nature of that involvement?}

• How does the AWI project fit in with other workforce initiatives, such as, SCSEP or Medicaid Infrastructure Grants (MIG)?

  • ^{Are the services offered across these initiatives substitutes or complements?}

  • ^{Do AWI participants need services available through SCSEP? Can AWI participants be co-enrolled in SCSEP or participate in any of its services?}

  • ^{Describe the level of coordination and communication across the agencies administering these initiatives (AWI, SCSEP and Medicaid Infrastructure Grants (MCI)?}

• How have public workforce investment partners contributed to the success of the project?

^{Note: MCI grants from the US Department of Health and Human Services are intended to develop a comprehensive system of employment supports for people with disabilities.}

5.2 Involvement of Organizations with Expertise on Aging

• What role did/do “aging organizations” play in designing, managing, or overseeing the grant, if any?

• What other roles do “aging organizations” play in providing services to employer or worker customers?

• How have “aging organizations” contributed to the success of the project?

5.3 Involvement of Educational Institutions and Training Providers

• What role did/do educational institutions or training providers play in designing, managing, or overseeing the grant, if any?

• What other roles do educational institutions or training providers play in providing services to employer or worker customers?

• How have education and training institutions or training partners contributed to the success of the project?

5.4 Involvement of Economic Development Entities

• What role did/do economic development entities play in designing, managing, or overseeing the grant, if any?

• What other roles do economic development entities play in providing services to employer or worker customers?

• How have economic development entities contributed to the success of the project?

5.5 Involvement of Local Employers, Employer Associations, or Business Intermediaries

• What role did/do employers or business intermediaries play in designing, managing, or overseeing the grant, if any?

• How did you recruit business partners? What were your selection criteria, if any?

• What other roles do employers or business intermediaries play in providing services to employer or worker customers? Are they involved in services to their own incumbent workers; in directly recruiting and hiring project participants, or in some more general way?)

• How have employers or business intermediaries contributed to the success of the project?

• How has employer involvement been affected by the economic recession?

5.6 Involvement of Other Partners
(e.g., faith-based organizations, community organizations, philanthropic institutions, apprenticeship programs, tribal organizations, SCSEPgrantees)

• What role did/do other partners play in designing, managing, or overseeing the grant, if any?

• What other roles do other partners play in providing services to employer or worker customers?

• How have other partners contributed to the success of the project?

6. OVERVIEW AND SEQUENCING OF SERVICES

6.1 Participant Recruitment and Referral

• How are participants recruited to the program?

  • ^{What proportion of participants are recruited through grant-specific outreach? (How do you advertise the project (e.g. brochures, public service announcements, speakers, requests for referrals)?}

  • ^{What proportion of participants are referred by:}

    • ^{One-Stop Career Centers?}

    • ^{Local education providers?}

    • ^{Other partners in the grant?}

    • ^{Other means}

• What type of special emphasis is there on recruiting disadvantaged populations (veterans, people with disabilities, military spouses, ex-offenders, minorities, new Americans)?

• Did you face any challenges in recruiting participants?

  • ^{Were there any delays in the start of participant enrollment?}

  • ^{If yes, what caused them?}

  • ^{What strategies did you use to overcome those challenges?}

• To what extent and how is recruitment of older workers linked to employer requirements?

6.2 Enrollment in Project

• At what point in the receipt of services is an individual officially “enrolled” in the project (reportable to USDOL as a participant)?

  • ^{Does receipt of a specific service or participation in a specific activity automatically activate “enrollment?”}

  • ^{Is enrollment reserved for individuals who decide to participate in occupational skills training?}

  • ^{If not, what is the ratio of enrollees who participate in training to enrollees who do not participate in training?}

  • ^{Do all training options include occupational skills training? (Or do some participants receive only pre-employment training or only job search training or only basic skills training without occupational skills training?)}

6.3 Service Components (including Orientation and Pre-Employment Services)

• What are the different service components developed by the project to meet the needs of aging workers served by the project? (E.g., orientation, assessment, service planning/career counseling, pre-employment training, skills training, academic counseling, internships or temporary work experience, job search/job placement, post-placement services)

  • ^{What is the duration and content of each service?}

  • ^{Who provides each service?}

  • ^{Is each service provided using grant funding or through leveraged funds by the providing agency?}

• What changes have occurred in the service components over time? (new services added, services redesigned, services discontinued) Why and how?

6.4 Sequencing of Services

• What is the typical sequence of project services (variations depending on customer needs?)

  • ^{What services have participants received from other sources before enrolling in AWI? Describe the depth and quality.}

  • ^{How long, on average, do participants remain active in the project?}

  • ^{To what extent do participants drop out of service delivery? At what point in service delivery do clients typically drop out? For what reasons? What type of follow-up is made with drop outs to encourage continuing participation or obtain outcome information?}

  • ^{For participants who stay with the project, what is the point (points) at which an individual is considered to have “completed” the project?}

  • ^{At what point in the service process are individuals considered to have exited the project?}

  • ^{What follow-up services are provided after project exit? When does follow-up end?}

• What is the frequency of different services?

  • ^{What project services, if any, are received by all participants? Do all participants receive occupational skills training? Pre-employment training? Other services?}

  • ^{What services are received by only a portion of all participants? What determines whether a customer will receive a given service?}

  • ^{What other types of services, including WIA-funded services are provided to AWI participants? How many individuals received these services (by type)?}

• What individuals and entities are responsible for the delivery of different services?

  • ^{In the course of participation, what project staff will participants come into contact with? Who does each of these service providers work for?}

  • ^{Do participants have to travel to different locations for different services?}

  • ^{If participants are referred to an education and training, or other project partner, what are the roles and responsibilities of the grantee? Who maintains the case—the grantee, the partner, or both?}

6.5 Co-enrolment of AWI Project Participants in Other Workforce Development Programs

• How frequently are AWI project participants provided with an orientation to the core One-Stop services (e.g. resource room and online labor market information and assessment tools) as part of their participation in the AWI project?

• How frequently are AWI project participants co-enrolled in other programs operated out of One-Stop centers (e.g. WIA, TAA. SCSEP, Employment Services (Wagner-Peyser), or other programs? At what point in the service sequence does co-enrollment occur?

• How is the delivery of other workforce development services coordinated with the delivery of services funded under the AWI grant? What program pays for what services?

• How is case management of AWI participants handled if they are also enrolled in another program?

6.6 Project Exit

• At what point do individuals officially “exit” the project?

  • ^{Is project exit initiated automatically by the MIS system after 90 days without service contact?}

  • ^{What is usually the last service received before exit?}

  • ^{When does exit occur in relation to the completion of training or placement into a job?}

7. DESIGN OF SPECIFIC SERVICES

Note: Training services are covered in section 8. You might want to cover the training services first and then come back to the topics in this section.

Some services may still be in the planning or pilot stage. Distinguish between active and planned services.

7.1 Career Awareness Information

The US Department of Labor has identified promoting career awareness among aging workers as one potentially important component of a strategy to help aging workers enter jobs in high growth sectors.

• Is providing information on careers in the targeted industries an important aspect of your project?

• Please describe any specific techniques or activities you have created to provide career awareness.

  • ^{How is career awareness integrated into the training and non-training services?}

  • ^{What types of resources (web sites, videos, etc.) are used to develop this awareness?}

  • ^{Are job shadowing or informational session opportunities available to promote career awareness?}

• How does the project rate the quality of career awareness services? How does career information influence participant decisions about training, entry occupations, or career paths?

  • ^{Does the project think that its career information service design is effective? Is it worthy of replication by other projects?}

  • ^{How could career information services for older workers be improved?}

7.2 Assessment Practices

Get copies of assessment tools. If on-line assessment, then ask for screenshots and/or a list of information collected.

• What is the goal of project assessment practices?

• What types of assessments are conducted?

  • ^{Assessment based on case manager or counselor’s interview with participant?}

  • ^{Commercial assessment instruments (what products)?}

  • ^{Products designed by state or LWIA}

  • ^{Was any tool or process developed or modified specifically for this program?}

• How was the decision made regarding which tool to use?

• Please describe the assessment process:

  • ^{Who conducts the assessment?}

  • ^{How long does it take?}

  • ^{When is it conducted?}

  • ^{What information is gathered?}

  • ^{How does the assessment compare to its counterpart under WIA and SCSEP?}

• How is this information used to determine whether participants receive readiness training, education/formal training, job placement services, or other services?

• Are these services funded through the grant, or through leveraged resources?

• How does the project rate the quality of its assessment services?

  • ^{Does the project think that its assessment practices are effective? Are they worthy of replication by other projects?}

  • ^{How could assessment practices be improved?}

7.3 Other “Front-End” Services

• Please describe other “front-end” services, such as pre-employment or pre-training workshops for all participants.

  • ^{What do front-end services consist of?}

  • ^{What is the goal of these services?}

  • ^{How were they developed?}

  • ^{How do they respond to the special needs of older workers?}

• How does the project rate the quality of its front-end services?

  • ^{How could front-end services be improved?}

  • ^{Does the project think that its front-end workshops or services are effective? Are they worthy of replication by other projects?}

7.4 Planning for Employment and Career Pathways

The key issue underlying this section is how the project develops service plans for individual participants and whether it emphasizes planning for longer-term career pathways (including advancement and lateral moves that build on a worker’s transferrable skills) in addition to finding an immediate job.

• Please describe how service plans and employment goals are established for an individual.

  • ^{Are these reflected in a written plan? Do all participants develop a service plan/employment *plan?}

  • ^{Are service plans/employment plans developed for participants who do not participate in training (if any)?}

  • ^{What information is used to develop the employment plan?}

  • ^{Does the plan for services describe both a short term and a long term career goal?}

  • ^{What occupational training and placement goals are available? Are all project participants prepared for the same occupation/industry or is there customer choice involved?}

• What are some examples of service plans/occupational goals for typical customers? (Ask a case manager to talk to you about the planning process and/or show you the file of a recent participant. Is there a plan for career advancement beyond initial employment?)

• How is planning for career pathways and career development built into the planning process?

  • ^{Are participants concerned about career advancement?}

  • ^{Are participants encouraged to move vertically up the career ladder, or laterally across occupations/industries? How does that vary by participant characteristics?}

  • ^{Do participants feel that they are being supported in developing skills for advancement in the targeted occupation?}

  • ^{How has the USDOL emphasis on developing career pathways influenced the project’s approach to service planning and employment goals?}

• To what extent can AWI career planning practices provide a model for other programs and services?

  • ^{For grantees that also serve WIA or SCSEP participants, how does service planning for AWI differ from other practices?}

  • ^{How could service planning for older workers be improved? What could other programs learn from the AWI service planning model?}

  • ^{Are any of the project’s career planning designs and practices more generally applicable to other groups receiving workforce development services?}

7.5 Case Management Practices and Participant Support

• Please describe case management practices for the AWI project.

  • ^{What are the goals of case management for this project?}

  • ^{Are there designated individuals (s) who provide case management services?}

  • ^{When does case management begin and when does it end?}

  • ^{How often and why does the case manager interact with the participant?}

  • ^{What are common topics discussed at case management meetings}

  • ^{What is the most common form of communication between case managers and program participants—in person, telephone?}

  • ^{Are case management and participant support funded through the grant, or through leveraged resources?}

• How does case management differ for participants who enroll in training and participants who do not participate in training? (Are there any active participants who forgo training?)

• What other types of support (e.g. academic counseling, peer support) are provided to project participants

  • ^{What other types of support do older workers need?}

  • ^{How are other types of support provided, by whom, and with what funding?}

  • ^{What types of support seem to work best for engaging participants and keeping them involved in the program?}

• How does the project rate the quality of its case management services?

  • ^{How could case management practices for older workers be improved?}

  • ^{Does the project think that its case management design is effective? Is it worthy of replication by other projects?}

7.6 Job Search Support and Job Placement Services

• Please describe the job search support and job placement services available to AWI participants.

  • ^{What entity (entities) provide job search support and job placement services to AWI grantees (role of grantee, training provider, other partners)?}

  • ^{How do job search/job placement services vary by occupation and/or participant characteristics?}

• How do job search support and placement services for AWI participants draw on resources available through One-Stop service systems?

  • ^{To what extent do AWI participants use placement services available through the local One-Stop system?}

  • ^{To what extent are grant funds used to support job search and job placement services versus leveraged funds from another source?}

7.7 Post-Placement Services

• What are participants’ service needs after they begin working? How does the project respond to these needs?

  • ^{What post-placement services are available to participants? How long are these services available?}

  • ^{Are post-placement services available both to training participants and participants who did not participate in training?}

7.8 Supportive Services and Service Referrals

• Please describe how the project responds to the supportive service needs of AWI participants?

  • ^{What supportive service needs do participants have when they enroll in the project? (e.g. assistance with health or health insurance issues, financial issues, nutrition, housing, disability, or other social service issues)}

  • ^{How does the project respond to these needs? (formal or informal referral linkages; effectiveness of linkages)}

  • ^{What proportion of participants receive supportive services from the project?}

• To what extent are supportive services paid for from project funds versus funds leveraged from other funding streams?

8. DESIGN AND DELIVERY OF EDUCATION AND TRAINING SERVICES

8.1 Training Options Available to AWI Participants

• (Get menu of training options, and information to fill in Training Options Matrix included as Section C).

• What types of education or training opportunities are available to older workers participating in the AWI grant?

  • ^{What is the range of short-term versus longer term training options?}

  • ^{How were these training opportunities developed or selected for se by the project? Did industry representatives or partners participate in selecting the targeted training opportunities?}

  • ^{How do these training choices relate to high-demand occupations, as required by WIA; H1-B industries and occupations, as required for this grant; state or local high growth/high wage criteria?}

  • ^{How are participants notified of the available education and training programs? If there are multiple training options, how do participants select among them?}

  • ^{Do project participants have to pass entry screening tests for specific training programs?}

• Characteristics of training providers most frequently used:

  • ^{What is the nature of the MOU between the training provider(s) and the project administrator?}

  • ^{What is the capacity of the training provider to serve aging workers enrolled in the AWI project?}

  • ^{Do any of the training providers offer separate classes or class sections exclusively for AWI participants? If so, how is the curriculum and- training delivery in these classes modified or designed to meet the needs of aging workers? (e.g. part-time training, technology-based training, independent self-paced study, hands-on learning)}

  • ^{Do training providers participate in recruiting students for these classes?}

  • ^{Do the training programs lead to completion of a certificate or credential? Describe the credential(s) and how it is (they are) perceived by local employers.}

• What payments, if any, can the project make for additional training costs?

  • ^{Does the project pay for additional costs associated with training or first job (e.g. equipment, post-training services such as qualifying exams, externships, etc?)}

  • ^{Does the project offer any living stipend or financial payment during training?}

8.2 Entrepreneurship Training and Services to Support Self-Employment Outcomes

• Does the project provide services to help participants start up a small business?

  • ^{How many participants are interested in this outcome?}

  • ^{Is the grantee partnered or have any type of relationship with Small Business Administration programs? If so, which specific programs and how are they utilized?}

  • ^{What types of entrepreneurial training are available to older workers through the grant?}

  • ^{Is this training available to all participants, or only those thinking of businesses in certain industries/sectors?}

  • ^{Is entrepreneurship training linked to occupational content training or is it a stand-alone training option?}

• How many participants are participating in entrepreneurial training? (Examples?)

• How does the project rate the quality of its entrepreneurial training services?

  • ^{How could the entrepreneurial training be improved?}

  • ^{Does the project think that its entrepreneurial training design is effective? Is it worthy of replication by other projects?}

8.3 Individual Decisions about Training

• How does an individual select a training plan and get approval for it?

  • ^{Can a participant decide not to participate in training and still remain an active enrollee in the project (or are all participants expected to enroll in some kind of occupational training)?}

  • ^{How does the project limit or guide the training choices available to a participant?}

  • ^{What types of screening does the project do to assess whether a participant could succeed in a given training program (e.g. reading or math skills, mobility, strength)?}

  • ^{What are limits on the duration or cost of training? Can exceptions be made?}

• What are the key factors that participants usually consider in deciding whether to enter training and what type of training to participate in?

• Does the project encourage participants to choose training from a specific provider or in a specific industry? (specific programs created or modified for this grant; providers on the eligible training provider list; other)

8.4 Work Experience or Internships for Hands-On Experience During or After Training

• Are internships or temporary work-experience a part of the AWI program design?
  
  
  

  • ^{If yes, describe the types of internship or work-experience placements (e.g. for-profit or non-profit employer, duration, skills gained or practiced during internships, stipend or training pay during work experience)}

  • ^{Who arranges the internships?}

  • ^{How are employers for internship placements recruited and matched to participants?}

• What is the goal of work-based training? (contact with a potential employer, additional opportunities to practice new skills.)

• What proportion of all AWI participants are involved in work-based learning?

• How does the project rate the quality of its work-based training?

  • ^{How could work experience or internships for older workers be improved?}

  • ^{Does the project think that its work-based training model is effective? Is it worthy of replication by other projects?}

8.5 Assessment of Training Options and Providers

• What are the strengths and weaknesses of the training options and providers used by your project?

  • ^{What are the most popular training occupations? Why are these occupations most attractive to the older workers served by the project?}

  • ^{Who are the most frequently used training providers (partners, eligible training provider list vendors, others?) Why are these training providers most attractive to the project participants?}

  • ^{To what extent have particular training providers and courses adapted their usual course content and training approach to make their courses more attractive to or appropriate for older workers?}

  • ^{What are the most innovative or most effective features of training adapted or designed for older workers? (Provide examples)}

  • ^{What are the most problematic or ineffective features of the training for older workers that you have experienced? Why are they problematic? How could they be improved?}

8.6 Curriculum Designs, Pedagogical Approaches, and Training Tools

What types of training approaches are used as part of the training offered to project participants

^{(Possible examples: contextualized learning, particular methods for upgrading specific occupational skills, comprehensive models with wraparound services such as assessment and follow-up)?}

^{What types of technology-based learning (TBL) do you use in training (i.e. chat rooms, webcasts, internet, and computer-based learning, etc.)? How do older workers react to the uses of technology in instructional methods?}

^{How are these training approaches modified to meet the needs of older learners?}

9. Participation by Employers and Incumbent Workers

9.1 Outreach to/ Recruitment of Employers

• How did/does the project reach out to employers?

  • ^{Did the project undertake a public media campaign to make employers aware of the project or change employer attitudes about older workers? How would you assess the effectiveness of that strategy?}

  • ^{What types of employers did you reach out to as part of the initiative? Did you target employers in certain industries or with a certain number of employees?}

• What are your goals for employer involvement?

  • ^{Were you trying to change employer attitudes, identify employers willing to hire older workers, help them adapt jobs to make them attractive to older workers, help them retain workers as they approached retirement age, or something else?}

• What services do/did you offer to employers?

  • ^{Possible examples: technical assistance on how to deal with older workers; services to their aging employees; assessment of older workers’ skills and skills gaps; recruitment and screening of older workers for available jobs; job coaching/ transitional employment for newly hired older workers)}

• How have employers responded to your outreach?

  • ^{What did employers identify as their most pressing concerns related to hiring aging workers?}

  • ^{Were you successful in reaching as many employers as you wanted and the types of employers you wanted?}

  • ^{What were some of the reasons that employers were receptive to the project? “What is in it for them” as an individual employer?}

  • ^{Were the participating employers interested in retaining their incumbent workers who are approaching retirement age? Hiring older workers as new employees?}

  • ^{Were the participating employers interested in promoting training for aging workers or influencing the content of planned training?}

• What would you do differently in the future to reach employers?

9.2 Project Assistance to Employers

• Please provide some examples of how specific employers have been involved with the project.

  • ^{How many employers did you work with? Describe size, industry, and extent of previous contact with public workforce development system.}

  • ^{How were employers involved in the different stages of the project?}

• How did the project assist employers?

  • ^{How did you work with employers to improve their attitudes about older workers as employees, if at all?}

  • ^{How did you work with employers to adapt jobs to make them more attractive to older workers, if at all?}

  • ^{How did you work with employers to support their recruitment and hiring process, if at all (e.g. refer screened or trained older workers to them)?}

  • ^{How did you work with employers to help them retain their employees as they approached retirement age, if at all?}

9.3 Assessment of Employer Outcomes

• How do/might you measure the project’s employer outcomes?

• How did your involvement influence employer attitudes and practices?

  • ^{How did you influence employer attitudes about older workers as employees?}

  • ^{How did you influence employer recruitment and hiring practices with respect to aging workers in general?}

  • ^{Have employers adapted their hiring procedures or job descriptions to make them more attractive to older workers, and if so how?}

  • ^{Did employers you worked with offer any new opportunities for advancement to their older employees?}

• Did employers you worked with hire any project participants as a result of this outreach?

  • ^{How many?}

  • ^{How satisfied were employers with the skills and job performance of project participants they hired?}

  • ^{What types of accommodations by employers were particularly effective in making the job attractive to older workers (e.g. shorter shifts, less physical exertion, more frequent breaks, etc?)}

• Do you think your project will help satisfy the demand for workers in the targeted industries? Why or why not?

9.4 Services Provided to Incumbent Workers

• Please describe your project’s strategy or design for serving incumbent workers, if any?

  • ^{Does your project include any efforts to serve incumbent worker?}

  • ^{What is the purpose of these services? (e.g. skills up*date, career advancement)}

  • ^{What are the desired outcomes?}

• If applicable, describe the scope and timing of services/training to incumbent workers.

  • ^{How were workers recruited or selected for services/training?}

  • ^{How many participants will be served over what time period?}

  • ^{Describe provider and delivery arrangements.}

  • ^{How is the employer involved in services to employed workers? Are there any cost-sharing arrangements? If so, describe.}

• What are the particular challenges or issues involved in providing services/training to older incumbent workers?

• How does the project rate the quality of its services to incumbent workers?

  • ^{How successful have you been in serving aging workers who are already working? (examples or outcome statistics)}

  • ^{How could the services to incumbent workers be improved?}

  • ^{Does the project think that its design for serving incumbent workers is effective? Is it worthy of replication by other projects?}

10. TECHNICAL ASSISTANCE

10.1 Sources of Technical Assistance

• Who do you look to when you have questions about project design or operations?

  • ^{What issues/questions to you refer to USDOL program office?}

  • ^{What issues/questions to you refer to funded TA provider?}

  • ^{What issues/questions do you refer to project peers?}

10.2 Technical Assistance Needs to Date

• Please describe your technical assistance needs to date?

  • ^{What are some of the key challenges you have faced at each phase of project design, organizational design, implementation and operations?}

  • ^{How have your technical assistance needs evolved over time?}

  • ^{What challenges are you currently facing or do you expect to face in the next year with the AWI project?}

• What are your most important TA needs at this point in your project development?

10.3 Technical Assistance on Organizational and Management Issues

• What are some of the challenges you have faced in dealing with organizational and management issues? (e.g., budgeting, record keeping, reporting, developing MOUs with project partners and defining their roles)

  • ^{To what extent has technical assistance helped you deal with these challenges or issues?}

  • ^{Provide details describing your problems and the assistance you received.}

  • ^{Did the TA help resolve these issues?}

• How has the technical assistance you have received influenced your project organization and management approach? How have these changes have improved your project?

10.4 Technical Assistance on Design and Delivery of Program Services

• What are some of the challenges you have faced in dealing with project design and delivery of program services?

  • ^{To what extent has technical assistance helped you with deal with these challenges or issues?}

  • ^{Provide details describing your problems and the assistance you received.}

  • ^{Did the TA help resolve these issues?}

• How has the technical assistance you have received influenced your project design and service delivery procedures? How have these changes have improved your project?

10.5 Assessment of the TA Received to Date

• How satisfied are you with the technical assistance you have received to date?

  • ^{Is the TA you have received responsive to your perceived capacity building needs?}

  • ^{What are the strengths and limitations of your TA coach?}

  • ^{How satisfied are you with the level of involvement of your TA coach? The frequency of contacts?}

• How could the TA you have received been improved in quality or topics covered?

• What are the most useful things you have learned as a result of the TA and training that you have received?

12. DATA COLLECTION AND REPORTING

12.3 Procedures for Documenting Participant Services

• How do you monitor and track participant services and outcomes?

  • ^{What tasks are involved in this process?}

  • ^{How much staff time does it take each week to monitor and document participants’ service use? Do participants have to submit attendance records or other documents?}

  • ^{Where are participant services recorded? (i.e. in the automated data system(s), hard copy case files, or both)}

  • ^{How standardized is the outcome information in the MIS system and in participant case files?}

  • ^{What challenges have you faced in tracking participant services? How have you addressed those challenges?}

12.4 Procedures for Documenting Participant and Project Outcomes

• What participant outcomes are measured and recorded? (e.g., training completion, degree attainment, employment, wages, job retention, etc.

  • ^{What data sources do you use to document participants’ employment outcomes? (e.g., UI wage records, participant self-reporting, employer confirmation, pay stubs, etc.)?}

  • ^{Where are participant outcomes recorded? (i.e. in the automated data system(s), hard copy case files, or both)}

  • ^{What client-level outcome measures are recorded in the MIS system?}

  • ^{What challenges have you faced in tracking participant outcomes? How have you addressed those challenges?}

• Have you developed any additional measures to document your project’s outcomes?

  • ^{What additional measures is the project measuring?}

  • ^{Have you identified any outcome measures specific to incumbent workers, employers, or participants targeting self-employment? If so, what are they?}

13. PRELIMINARY INFORMATION ON PROGRAM OUTCOMES

Review the outcomes in the most recent report, compared to the grantees plan and to other grantees. Discuss outcomes to date with project respondents.

13.1 Program Exiters To Date

• How many and what types of participants have exited the program to date?

  • ^{How many participants have exited to date and for what reasons?}

  • ^{What proportion of those exited to date were drop-outs?}

• Based on exiters to date, please describe typical program duration.

  • ^{What is the average duration of program participation (for trainees, for non-trainees, for all participants excluding drop-outs)?}

  • ^{How much variation is there in program duration? What factors affect duration?}

13.2 Participant Outcomes To Date

• What types of jobs and earnings are participants receiving?

  • ^{Are jobs related to the training received or the career guidance provided?}

  • ^{Do these jobs have established career ladders?}

  • ^{Are these jobs consistent with the project’s targeted occupations and industries?}

  • ^{What are the principal factors affecting outcomes for training and non-training participants?}

• How different are these outcomes from outcomes reported for all WIA participants or outcomes reported for all SCSEP participants in the local area as a whole? How might these differences be explained?

13.3 Outcomes on Any Additional Measures To Date

• Please describe outcomes to date on any additional measures.

  • ^{Have you measured outcomes to date for any additional outcome measures (e.g., for incumbent worker training?)}

  • ^{What are the results?}

14. ACTIVITIES TO INCREASE LOCAL SYSTEM CAPACITY TO SERVE OLDER WORKERS

14.1 Strategies to Expand Availability of Services for Aging Workers

• How has the project worked to expand its own capacity to provide workforce development and training services for aging workers? (improvements in quality and quality)

  • ^{How has the project worked to increase the number of slots for older workers in existing training opportunities? To increase the types of training occupations? (How many additional aging workers will be served indirectly as a result of project efforts)}

  • ^{How has the project worked to make program improvements to better serve older workers?}

• How has the project worked to expand the quality and availability of services for aging workers within the local community?

  • ^{Has the project focused on training One-Stop front-line staff to better serve aging workers?}

  • ^{Has the project focused on disseminating its service designs tailored to the needs of aging workers?}

  • ^{What changes, if any, have occurred to date in how local workforce investment systems serve older individuals?}

  • ^{What changes, if any, have occurred in the number of aging workers served by the local workforce investment system?}

14.2 Capacity-Building Activities, Measures, and Outcomes

• What measures does/will the project use to measure its progress in building the capacity of the local system?

  • ^{Numbers of staff trained?}

  • ^{Numbers of aging workers or employers served with expanded capacity?}

  • ^Other?

• Will these increases in the capacity to serve aging workers last beyond the lifetime of the demonstration grant?

14.3 Planned Capacity-Building Products

• How will the project’s promising practices be packaged for dissemination?

  • ^{What specific activities (products, models, curricula, teaching methods, training-the-trainer, licensure or certification requirements) for serving older workers will be operationalized by the workforce system (and by the grantee, if the grantee is not a WIB) after the grant ends?}

  • ^{How will these products be disseminated for use by other entities after the grant ends?}

  • ^{To what user groups are these products directed (e.g., business groups, community colleges, proprietary training providers, labor-management organizations, One-Stop staff)?}

• How will the grant enhance One-Stop Career Center capacity to serve aging workers?

• What are the different “deliverables,” planned by the project?

  • ^{Who will produce the deliverables?}

  • ^{Who will act as an expert reviewer?}

  • ^{What form will the deliverables take?}

14.4 Progress in Completing Planned Deliverables

• Please describe your progress in developing products to support dissemination of your aging worker approach?

  • ^{What progress has been made in completing planned products?}

  • ^{What challenges have been encountered in producing deliverables?}

  • ^{How have plans for deliverables evolved over time?}

  • ^{What is the current schedule for producing deliverables?}

  • ^{What has been your experience to date in the review of products by independent entities? Has this improved the quality of deliverables?}

• What technical assistance on producing deliverables would be useful?

15. SUCCESSES, CHALLENGES, AND LESSONS LEARNED

15.1 Unmet Needs

• To what extent are the needs of older workers still unaddressed in the local community?

  • ^{What service needs of older workers sometimes are still unaddressed? Which are the highest priority needs?}

  • ^{What resources would be necessary to meet these service needs?}

  • ^{What additional services would be most helpful for aging workers?}

15.2 Summary Assessment of Project Strengths and Limitations

• In summary, what are the primary strengths and the primary limitations of the AWI project with respect to:

  • ^{Integration with regional economic talent development?}

  • ^{Organization and partnerships?}

  • ^{Service delivery?}

  • ^{Sustainabillity?}

• What have program participants found most helpful about services provided by grantees? What services were least helpful?

• What would participants like to see changed about this program?

15.3 Project Successes and Practices Worthy of Replication

• Which of projects services have been designed specifically to meet the needs of older workers?

• What recommendations does the project have about the design of services or aging workers?

  • ^{What recommendations does the project have about specific designs that work well for aging workers?}

  • ^{What recommendations does the project have about specific curricula or guides for workshop content that might be available to other projects serving aging workers?}

• What are the practices of this project that show most promise? What aspects of your program would you recommend that other projects emulate?

  • ^{How successful were grantees in recruiting a diverse array of partners?}

  • ^{Which partner relationships were the most successful and why?}

  • ^{What aspects of the program work particularly well in helping participants find and keep jobs?}

• What were the other main successes of this program initiative? How have you achieved these successes?

15.4 Summary of Project Challenges

• What were the key problems or challenges in administering the project?

  • ^{Did you encounter challenges coordinating the input of all partners into account when making key decisions?}

  • ^{Did any of the partnerships fail during the course of the project, and what were the possible reasons?}

  • ^{Did you encounter challenges in recruiting participants?}

• What challenges did you face in serving participants?

  • ^{What challenges did you face in placing participants in jobs?}

  • ^{What challenges did you encounter in helping participants keep and advance in their jobs?}

  • ^{What challenges did you face in tracking participants and recording their outcomes?}

15.5 Lessons Learned

• What are the most important lessons that you have learned as a result of operating the AWI project?

  • ^{What practical lessons and promising practices for the workforce investment system were identified during this project?}

  • ^{Do these lessons apply only to older workers, or more broadly?}

• In hindsight, what would you do differently if you were to start the project again?

15.6 Plans to Expand, Sustain, or Replicate Project Model Within State

Information about sustainability and replicability will be preliminary in the interim report. However, because the grantees have been operating for almost a full year by the time of the site visits, such questions are appropriate for the Round 1 visit. These topics will be very important in Round 2.

• What are the essential program components of a successful AWI service model?

• Do you plan to replicate your program or service model? If so, where and how?

• What advice do you have for replication of your service design for aging workers?

  • ^{Would you recommend this model to others?}

  • ^{What are the key challenges to replicating this model for serving aging workers?}

  • ^{What changes would you recommend others make before replicating your model?}

• For whom do you think your approach is best suited (e.g. under what economic conditions and with what types of aging workers do you think your approach will be effective)?

• What are your strategies to continue the services provided by the grantee after the program ends?

  • ^{To what extent have grantees sought outside funding to continue providing services? How successful have they been? What is the likelihood that services that the grantee provides to older workers will continue after grant funds run out?}

  • ^{From the project’s perspective, what are the highest priority services and service delivery approaches to try to continue after the grant ends?}

• What specific activities, programs, etc. targeted to older workers do you expect will be institutionalized by the public workforce system (or grantee, if grantee is not a WIB or One-Stop)?

  • ^{Do you expect that the workforce system will use WIA resources to provide services to support older workers after the grant ends?}

  • ^{If so, which agencies or staff are good candidates to provide those services?}

Site Visit Topics for Direct Service Delivery Staff and Supervisors (Round 2)

1. OVERVIEW OF PROGRAM INITIATIVE

1.4 Target Population

• What specific groups of aging workers does your project target (age, employment status, previous work history, other characteristics)?

  • ^{Have there been any changes in the groups targeted by the project since the first site visit?}

• What are the characteristics of participants enrolled to date? What are their education and skill levels, and level of work experience?

  • ^{Are these participants consistent with the targeted groups?}

  • ^{Have you been surprised by any of the characteristics of the enrolled participants?}

1.6. Program Eligibility Requirements

• What are the eligibility requirements for participation in the program?

  • ^{Are there any additional restrictions or goals beyond the grant requirement (age 55+)? Do participants need to be currently unemployed or employed in a particular industry?}

  • ^{Have eligibility requirements posed any challenges to program success?}

  • ^{Have there been any chances in eligibility requirements since the first site visit?}

2. DESCRIPTION OF PROJECT SERVICE AREA

2.2 Description of Local Labor Market in the Designated Service Area

• How have changes in the economy since the first site visit affected grant planning and implementation?

• How is the project design and implementation experience influenced by the regional labor market and economic trends?

2.3 Description of Other Services Available to Older Workers in the Service Area

• Before this grant, what employment and training services existed for older workers?

  • ^{Have there been any changes in the services for older workers available from other agencies since the first site visit?}

• What factors, if any, limit the services available to older workers from the general workforce development system?

4. DEVELOPING PROJECT PARTNERSHIPS

4.4 Communication Between Grantee and Partners

• How does internal project communication work?

  • ^{How does the grantee’s project manager communicate with project partners? How frequently and about what types of issues? How do partners communicate with each other?}

  • ^{How do partners communicate with each other?}

• Does the grantee hold regular meetings with partners? If so, what is the purpose of these meetings? Are these meetings held individually or as a group? How long do they last?

  • ^{Have communication patterns changed since the first site visit?}

4.5 Assessment of Partner Involvement

• How would you characterize the overall success involving partners to participate in the initiative?

  • ^{How pleased are you with the final composition of partners in the initiative?}

  • ^{Were there some agencies you initially wanted to partner with but were unable to do so? If so, what were the barriers in establishing that arrangement? Was another agency involved to fill that gap, and if not, is the initiative lacking in some way without that partner?}

  • ^{Are there any other partners that in hindsight you wish were involved in the initiative that are not involved?}

• Please describe any challenges you have experienced in developing effective relationships between the project partners.

  • ^{What challenges did you experience in recruiting partners or in arranging for their specific role in the project?}

  • ^{What challenges did you experience in the quality of the contributions made by individual partners?}

• What are the most successful aspects of your partnership?

• What advice do you have for other projects serving older workers in terms of partnership formation and partner roles?

4.6 Informal Relationships with Other Entities Serving Project Participants

• Please describe any informal relationships that the project has developed with other entities to expand the services available to project participants.

  • ^{Have there been any changes in informal relationships since the first site visit?}

• What are the experiences of participants referred to these providers?

• Would the project have benefited from more formal relationships with these agencies?

5. PROJECT PARTNERS AND THEIR ROLES

5.1 Involvement of the Public Workforce Investment System (e.g. One-Stop Centers and Constituent Programs)

(If grantee is a local WIB, describe the involvement of One-Stop staff and programs that are not directly part of the funded AWI project.

• What role did/do public workforce investment partners play in designing, managing, or overseeing the grant, if any?

• What roles do public workforce investment partners play in providing services to employer or worker customers?

• How much involvement does the AWI project have with other services provided at the One-Stop Career Centers?

  • ^{Do AWI project participants utilize WIA core and intensive services? If so, for what types of services? Are workers served under AWI co-enrolled in the One-Stop system (or SCSEP) under WIA?}

  • ^{Are other workforce investment partners involved in providing services to AWI project participants? What is the nature of that involvement?}

• How does the AWI project fit in with other workforce initiatives, such as, SCSEP or Medicaid Infrastructure Grants (MIG)?

  • ^{Are the services offered across these initiatives substitutes or complements?}

  • ^{Do AWI participants need services available through SCSEP? Can AWI participants be co-enrolled in SCSEP or participate in any of its services?}

  • ^{Describe the level of coordination and communication across the agencies administering these initiatives (AWI, SCSEP and Medicaid Infrastructure Grants (MCI)?}

• How have public workforce investment partners contributed to the success of the project?

5.2 Involvement of Organizations with Expertise on Aging

• What role did/do “aging organizations” play in designing, managing, or overseeing the grant, if any?

• What other roles do “aging organizations” play in providing services to employer or worker customers?

• How have “aging organizations” contributed to the success of the project?

5.3 Involvement of Educational Institutions and Training Providers

• What role did/do educational institutions or training providers play in designing, managing, or overseeing the grant, if any?

• What other roles do educational institutions or training providers play in providing services to employer or worker customers?

• How have education and training institutions or training partners contributed to the success of the project?

5.4 Involvement of Economic Development Entities

• What role did/do economic development entities play in designing, managing, or overseeing the grant, if any?

• What other roles do economic development entities play in providing services to employer or worker customers?

• How have economic development entities contributed to the success of the project?

5.5 Involvement of Local Employers, Employer Associations, or Business Intermediaries

• What role did/do employers or business intermediaries play in designing, managing, or overseeing the grant, if any?

• How did you recruit business partners? What were your selection criteria, if any?

• What other roles do employers or business intermediaries play in providing services to employer or worker customers? Are they involved in services to their own incumbent workers; in directly recruiting and hiring project participants, or in some more general way?)

• How have employers or business intermediaries contributed to the success of the project?

• How has employer involvement been affected by the economic recession?

5.6 Involvement of Other Partners
(e.g., faith-based organizations, community organizations, philanthropic institutions, apprenticeship programs, tribal organizations, SCSEPgrantees)

• What role did/do other partners play in designing, managing, or overseeing the grant, if any?

• What other roles do other partners play in providing services to employer or worker customers?

• How have other partners contributed to the success of the project?

6. OVERVIEW AND SEQUENCING OF SERVICES

6.1 Participant Recruitment and Referral

• How are participants recruited to the program?

  • ^{What proportion of participants are recruited through grant-specific outreach? (How do you advertise the project (e.g. brochures, public service announcements, speakers, requests for referrals)?}

  • ^{What proportion of participants are referred by:}

    • ^{One-Stop Career Centers?}

    • ^{Local education providers?}

    • ^{Other partners in the grant?}

    • ^{Other means}

• What type of special emphasis is there on recruiting disadvantaged populations (veterans, people with disabilities, military spouses, ex-offenders, minorities, new Americans)?

• Have you faced any challenges in recruiting participants?

• To what extent and how is recruitment of older workers linked to employer requirements?

6.2 Enrollment in Project

• At what point in the receipt of services is an individual officially “enrolled” in the project (reportable to USDOL as a participant)?

  • ^{Does receipt of a specific service or participation in a specific activity automatically activate “enrollment?”}

  • ^{Is enrollment reserved for individuals who decide to participate in occupational skills training?}

  • ^{If not, what is the ratio of enrollees who participate in training to enrollees who do not participate in training?}

  • ^{Do all training options include occupational skills training? (Or do some participants receive only pre-employment training or only job search training or only basic skills training without occupational skills training?)}

6.3 Service Components (including Orientation and Pre-Employment Services)

• Have there been any changes in the service components offered by the project since the first site visit?

  • ^{Is each service provided using grant funding or through leveraged funds by the providing agency?}

• What changes have occurred in the service components over time? (new services added, services redesigned, services discontinued) Why and how?

6.4 Sequencing of Services

• What is the typical sequence of project services (variations depending on customer needs?)

• What is the frequency of different services?

• What individuals and entities are responsible for the delivery of different services?

6.5 Co-enrolment of AWI Project Participants in Other Workforce Development Programs

• How frequently are AWI project participants provided with an orientation to the core One-Stop services (e.g. resource room and online labor market information and assessment tools) as part of their participation in the AWI project?

• How frequently are AWI project participants co-enrolled in other programs operated out of One-Stop centers (e.g. WIA, TAA. SCSEP, Employment Services (Wagner-Peyser), or other programs? At what point in the service sequence does co-enrollment occur?

• How is the delivery of other workforce development services coordinated with the delivery of services funded under the AWI grant? What program pays for what services?

• How is case management of AWI participants handled if they are also enrolled in another program?

6.6 Project Exit

• At what point do individuals officially “exit” the project?

  • ^{Is project exit initiated automatically by the MIS system after 90 days without service contact?}

  • ^{What is usually the last service received before exit?}

  • ^{When does exit occur in relation to the completion of training or placement into a job?}

7. DESIGN OF SPECIFIC SERVICES

Note: Training services are covered in section 8. You might want to cover the training services first and then come back to the topics in this section.

Some services may still be in the planning or pilot stage. Distinguish between active and planned services.

7.1 Career Awareness Information

The US Department of Labor has identified promoting career awareness among aging workers as one potentially important component of a strategy to help aging workers enter jobs in high growth sectors.

• Is providing information on careers in the targeted industries an important aspect of your project?

• Please describe any specific techniques or activities you have created to provide career awareness.

• How does the project rate the quality of career awareness services? How does career information influence participant decisions about training, entry occupations, or career paths?

7.2 Assessment Practices

Get copies of assessment tools. If on-line assessment, then ask for screenshots and/or a list of information collected.

• What is the goal of project assessment practices?

• What types of assessments are conducted?

• Please describe the assessment process:

  • ^{How does the assessment compare to its counterpart under WIA and SCSEP?}

• How is this information used to determine whether participants receive readiness training, education/formal training, job placement services, or other services?

• Are these services funded through the grant, or through leveraged resources?

• How does the project rate the quality of its assessment services?

  • ^{Does the project think that its assessment practices are effective? Are they worthy of replication by other projects?}

  • ^{How could assessment practices be improved?}

7.3 Other “Front-End” Services

• Please describe other “front-end” services, such as pre-employment or pre-training workshops for all participants.

  • ^{What do front-end services consist of?}

  • ^{How do they respond to the special needs of older workers?}

• How does the project rate the quality of its front-end services?

  • ^{How could front-end services be improved?}

  • ^{Does the project think that its front-end workshops or services are effective? Are they worthy of replication by other projects?}

7.4 Planning for Employment and Career Pathways

The key issue underlying this section is how the project develops service plans for individual participants and whether it emphasizes planning for longer-term career pathways (including advancement and lateral moves that build on a worker’s transferrable skills) in addition to finding an immediate job.

• Please describe how service plans and employment goals are established for an individual.

• What are some examples of service plans/occupational goals for typical customers? (Ask a case manager to talk to you about the planning process and/or show you the file of a recent participant. Is there a plan for career advancement beyond initial employment?)

• How is planning for career pathways and career development built into the planning process?

• To what extent can AWI career planning practices provide a model for other programs and services?

  • ^{For grantees that also serve WIA or SCSEP participants, how does service planning for AWI differ from other practices?}

  • ^{How could service planning for older workers be improved? What could other programs learn from the AWI service planning model?}

  • ^{Are any of the project’s career planning designs and practices more generally applicable to other groups receiving workforce development services?}

7.5 Case Management Practices and Participant Support

• Please describe case management practices for the AWI project.

• How does case management differ for participants who enroll in training and participants who do not participate in training? (Are there any active participants who forgo training?)

• What other types of support (e.g. academic counseling, peer support) are provided to project participants

• How does the project rate the quality of its case management services?

  • ^{How could case management practices for older workers be improved?}

  • ^{Does the project think that its case management design is effective? Is it worthy of replication by other projects?}

7.6 Job Search Support and Job Placement Services

• Please describe the job search support and job placement services available to AWI participants.

  • ^{What entity (entities) provide job search support and job placement services to AWI grantees (role of grantee, training provider, other partners)?}

  • ^{How do job search/job placement services vary by occupation and/or participant characteristics?}

• How do job search support and placement services for AWI participants draw on resources available through One-Stop service systems?

  • ^{To what extent do AWI participants use placement services available through the local One-Stop system?}

  • ^{To what extent are grant funds used to support job search and job placement services versus leveraged funds from another source?}

7.7 Post-Placement Services

• What are participants’ service needs after they begin working? How does the project respond to these needs?

  • ^{What post-placement services are available to participants? How long are these services available?}

  • ^{Are post-placement services available both to training participants and participants who did not participate in training?}

7.8 Supportive Services and Service Referrals

• Please describe how the project responds to the supportive service needs of AWI participants?

  • ^{What supportive service needs do participants have when they enroll in the project? (e.g. assistance with health or health insurance issues, financial issues, nutrition, housing, disability, or other social service issues)}

  • ^{How does the project respond to these needs? (formal or informal referral linkages; effectiveness of linkages)}

  • ^{What proportion of participants receive supportive services from the project?}

• To what extent are supportive services paid for from project funds versus funds leveraged from other funding streams?

8. DESIGN AND DELIVERY OF EDUCATION AND TRAINING SERVICES

8.1 Training Options Available to AWI Participants

• What types of education or training opportunities are available to older workers participating in the AWI grant?

• Characteristics of training providers most frequently used:

• What payments, if any, can the project make for additional training costs?

  • ^{Does the project pay for additional costs associated with training or first job (e.g. equipment, post-training services such as qualifying exams, externships, etc.)}

  • ^{Does the project offer any living stipend or financial payment during training?}

8.2 Entrepreneurship Training and Services to Support Self-Employment Outcomes

• Does the project provide services to help participants start up a small business?

• How many participants are participating in entrepreneurial training? (Examples?)

• How does the project rate the quality of its entrepreneurial training services?

  • ^{How could the entrepreneurial training be improved?}

  • ^{Does the project think that its entrepreneurial training design is effective? Is it worthy of replication by other projects?}

8.3 Individual Decisions about Training

• How does an individual select a training plan and get approval for it?

  • ^{Can a participant decide not to participate in training and still remain an active enrollee in the project (or are all participants expected to enroll in some kind of occupational training)?}

  • ^{How does the project limit or guide the training choices available to a participant?}

  • ^{What types of screening does the project do to assess whether a participant could succeed in a given training program (e.g. reading or math skills, mobility, strength)?}

  • ^{What are limits on the duration or cost of training? Can exceptions be made?}

• What are the key factors that participants usually consider in deciding whether to enter training and what type of training to participate in?

• Does the project encourage participants to choose training from a specific provider or in a specific industry? (specific programs created or modified for this grant; providers on the eligible training provider list; other)

8.4 Work Experience or Internships for Hands-On Experience During or After Training

• Are internships or temporary work-experience a part of the AWI program design?
  
  
  

  • ^{If yes, describe the types of internship or work-experience placements (e.g. for-profit or non-profit employer, duration, skills gained or practiced during internships, stipend or training pay during work experience)}

  • ^{Who arranges the internships?}

  • ^{How are employers for internship placements recruited and matched to participants?}

• What is the goal of work-based training? (contact with a potential employer, additional opportunities to practice new skills.)

• What proportion of all AWI participants are involved in work-based learning?

• How does the project rate the quality of its work-based training?

  • ^{How could work experience or internships for older workers be improved?}

  • ^{Does the project think that its work-based training model is effective? Is it worthy of replication by other projects?}

8.5 Assessment of Training Options and Providers

• What are the strengths and weaknesses of the training options and providers used by your project?

  • ^{What are the most popular training occupations? Why are these occupations most attractive to the older workers served by the project?}

  • ^{Who are the most frequently used training providers (partners, eligible training provider list vendors, others?) Why are these training providers most attractive to the project participants?}

  • ^{To what extent have particular training providers and courses adapted their usual course content and training approach to make their courses more attractive to or appropriate for older workers?}

  • ^{What are the most innovative or most effective features of training adapted or designed for older workers? (Provide examples)}

  • ^{What are the most problematic or ineffective features of the training for older workers that you have experienced? Why are they problematic? How could they be improved?}

8.6 Curriculum Designs, Pedagogical Approaches, and Training Tools

• What types of training approaches are used as part of the training offered to project participants

  • ^{How are these training approaches modified to meet the needs of older learners?}

9. Participation by Employers and Incumbent Workers

9.1 Outreach to/ Recruitment of Employers

• How did/does the project reach out to employers?

• What are your goals for employer involvement?

• What services have you offered to employers?

• How have employers responded to your outreach?

  • ^{What did employers identify as their most pressing concerns related to hiring aging workers?}

  • ^{Were you successful in reaching as many employers as you wanted and the types of employers you wanted?}

  • ^{What were some of the reasons that employers were receptive to the project? “What is in it for them” as an individual employer?}

  • ^{Were the participating employers interested in retaining their incumbent workers who are approaching retirement age? Hiring older workers as new employees?}

  • ^{Were the participating employers interested in promoting training for aging workers or influencing the content of planned training?}

• What would you do differently in the future to reach employers?

9.2 Project Assistance to Employers

• Please provide some examples of how specific employers have been involved with the project.

  • ^{How many employers did you work with? Describe size, industry, and extent of previous contact with public workforce development system.}

  • ^{How were employers involved in the different stages of the project?}

• How did the project assist employers?

  • ^{How did you work with employers to improve their attitudes about older workers as employees, if at all?}

  • ^{How did you work with employers to adapt jobs to make them more attractive to older workers, if at all?}

  • ^{How did you work with employers to support their recruitment and hiring process, if at all (e.g. refer screened or trained older workers to them)?}

  • ^{How did you work with employers to help them retain their employees as they approached retirement age, if at all?}

9.3 Assessment of Employer Outcomes

• How do/might you measure the project’s employer outcomes?

• How did your involvement influence employer attitudes and practices?

  • ^{How did you influence employer attitudes about older workers as employees?}

  • ^{How did you influence employer recruitment and hiring practices with respect to aging workers in general?}

  • ^{Have employers adapted their hiring procedures or job descriptions to make them more attractive to older workers, and if so how?}

  • ^{Did employers you worked with offer any new opportunities for advancement to their older employees?}

• Did employers you worked with hire any project participants as a result of this outreach?

  • ^{How many?}

  • ^{How satisfied were employers with the skills and job performance of project participants they hired?}

  • ^{What types of accommodations by employers were particularly effective in making the job attractive to older workers (e.g. shorter shifts, less physical exertion, more frequent breaks, etc?)}

• Do you think your project will help satisfy the demand for workers in the targeted industries? Why or why not?

9.4 Services Provided to Incumbent Workers

• Please describe your project’s strategy or design for serving incumbent workers, if any?

  • ^{Does your project include any efforts to serve incumbent worker?}

  • ^{What is the purpose of these services? (e.g. skills up*date, career advancement)}

  • ^{What are the desired outcomes?}

• If applicable, describe the scope and timing of services/training to incumbent workers.

  • ^{How were workers recruited or selected for services/training?}

  • ^{How many participants will be served over what time period?}

  • ^{Describe provider and delivery arrangements.}

  • ^{How is the employer involved in services to employed workers? Are there any cost-sharing arrangements? If so, describe.}

• What are the particular challenges or issues involved in providing services/training to older incumbent workers?

• How does the project rate the quality of its services to incumbent workers?

  • ^{How successful have you been in serving aging workers who are already working? (examples or outcome statistics)}

  • ^{How could the services to incumbent workers be improved?}

  • ^{Does the project think that its design for serving incumbent workers is effective? Is it worthy of replication by other projects?}

10. TECHNICAL ASSISTANCE

10.2 Technical Assistance Needs to Date

• Please describe your technical assistance needs to date?

  • ^{What are some of the key challenges you have faced at each phase of project design, organizational design, implementation and operations?}

  • ^{How have your technical assistance needs evolved over time?}

  • ^{What challenges are you currently facing or do you expect to face in the next year with the AWI project?}

• What are your most important TA needs at this point in your project development?

10.3 Technical Assistance on Organizational and Management Issues

• What are some of the challenges you have faced in dealing with organizational and management issues? (e.g., budgeting, record keeping, reporting, developing MOUs with project partners and defining their roles)

  • ^{To what extent has technical assistance helped you deal with these challenges or issues?}

  • ^{Provide details describing your problems and the assistance you received.}

  • ^{Did the TA help resolve these issues?}

• How has the technical assistance you have received influenced your project organization and management approach? How have these changes have improved your project?

10.4 Technical Assistance on Design and Delivery of Program Services

• What are some of the challenges you have faced in dealing with project design and delivery of program services?

  • ^{To what extent has technical assistance helped you with deal with these challenges or issues?}

  • ^{Provide details describing your problems and the assistance you received.}

  • ^{Did the TA help resolve these issues?}

• How has the technical assistance you have received influenced your project design and service delivery procedures? How have these changes have improved your project?

10.5 Assessment of the TA Received to Date

• How satisfied are you with the technical assistance you have received to date?

  • ^{Is the TA you have received responsive to your perceived capacity building needs?}

  • ^{What are the strengths and limitations of your TA coach?}

  • ^{How satisfied are you with the level of involvement of your TA coach? The frequency of contacts?}

• How could the TA you have received been improved in quality or topics covered?

• What are the most useful things you have learned as a result of the TA and training that you have received?

12. DATA COLLECTION AND REPORTING

12.3 Procedures for Documenting Participant Services

• How do you monitor and track participant services and outcomes?

  • ^{What challenges have you faced in tracking participant services? How have you addressed those challenges?}

12.4 Procedures for Documenting Participant and Project Outcomes

• What participant outcomes are measured and recorded? (e.g., training completion, degree attainment, employment, wages, job retention, etc.

• Have you developed any additional measures to document your project’s outcomes?

  • ^{What additional measures is the project measuring?}

  • ^{Have you identified any outcome measures specific to incumbent workers, employers, or participants targeting self-employment? If so, what are they?}

13. PRELIMINARY INFORMATION ON PROGRAM OUTCOMES

Review the outcomes in the most recent report, compared to the grantees plan and to other grantees. Discuss outcomes to date with project respondents.

13.1 Program Exiters To Date

• How many and what types of participants have exited the program to date?

  • ^{How many participants have exited to date and for what reasons?}

  • ^{What proportion of those exited to date were drop-outs?}

• Based on exiters to date, please describe typical program duration.

  • ^{What is the average duration of program participation (for trainees, for non-trainees, for all participants excluding drop-outs)?}

  • ^{How much variation is there in program duration? What factors affect duration?}

13.2 Participant Outcomes To Date

• What types of jobs and earnings are participants receiving?

  • ^{Are jobs related to the training received or the career guidance provided?}

  • ^{Do these jobs have established career ladders?}

  • ^{Are these jobs consistent with the project’s targeted occupations and industries?}

  • ^{What are the principal factors affecting outcomes for training and non-training participants?}

• How different are these outcomes from outcomes reported for all WIA participants or outcomes reported for all SCSEP participants in the local area as a whole? How might these differences be explained?

13.3 Outcomes on Any Additional Measures To Date

• Please describe outcomes to date on any additional measures.

  • ^{Have you measured outcomes to date for any additional outcome measures (e.g., for incumbent worker training?)}

  • ^{What are the results?}

14. ACTIVITIES TO INCREASE LOCAL SYSTEM CAPACITY TO SERVE OLDER WORKERS

14.1 Strategies to Expand Availability of Services for Aging Workers

• How has the project worked to expand its own capacity to provide workforce development and training services for aging workers? (improvements in quality and quality)

  • ^{How has the project worked to increase the number of slots for older workers in existing training opportunities? To increase the types of training occupations? (How many additional aging workers will be served indirectly as a result of project efforts)}

  • ^{How has the project worked to make program improvements to better serve older workers?}

• How has the project worked to expand the quality and availability of services for aging workers within the local community?

  • ^{Has the project focused on training One-Stop front-line staff to better serve aging workers?}

  • ^{Has the project focused on disseminating its service designs tailored to the needs of aging workers?}

  • ^{What changes, if any, have occurred to date in how local workforce investment systems serve older individuals?}

  • ^{What changes, if any, have occurred in the number of aging workers served by the local workforce investment system?}

14.2 Capacity-Building Activities, Measures, and Outcomes

• What measures does/will the project use to measure its progress in building the capacity of the local system?

  • ^{Numbers of staff trained?}

  • ^{Numbers of aging workers or employers served with expanded capacity?}

  • ^Other?

• Will these increases in the capacity to serve aging workers last beyond the lifetime of the demonstration grant?

14.3 Planned Capacity-Building Products

• How will the project’s promising practices be packaged for dissemination?

  • ^{What specific activities (products, models, curricula, teaching methods, training-the-trainer, licensure or certification requirements) for serving older workers will be operationalized by the workforce system (and by the grantee, if the grantee is not a WIB) after the grant ends?}

  • ^{How will these products be disseminated for use by other entities after the grant ends?}

  • ^{To what user groups are these products directed (e.g., business groups, community colleges, proprietary training providers, labor-management organizations, One-Stop staff)?}

• How will the grant enhance One-Stop Career Center capacity to serve aging workers?

• What are the different “deliverables,” planned by the project?

  • ^{Who will produce the deliverables?}

  • ^{Who will act as an expert reviewer?}

  • ^{What form will the deliverables take?}

14.4 Progress in Completing Planned Deliverables

• Please describe your progress in developing products to support dissemination of your aging worker approach?

  • ^{What progress has been made in completing planned products?}

  • ^{What challenges have been encountered in producing deliverables?}

  • ^{How have plans for deliverables evolved over time?}

  • ^{What is the current schedule for producing deliverables?}

  • ^{What has been your experience to date in the review of products by independent entities? Has this improved the quality of deliverables?}

• What technical assistance on producing deliverables would be useful?

15. SUCCESSES, CHALLENGES, AND LESSONS LEARNED

15.1 Unmet Needs

• To what extent are the needs of older workers still unaddressed in the local community?

  • ^{What service needs of older workers sometimes are still unaddressed? Which are the highest priority needs?}

  • ^{What resources would be necessary to meet these service needs?}

  • ^{What additional services would be most helpful for aging workers?}

15.2 Summary Assessment of Project Strengths and Limitations

• In summary, what are the primary strengths and the primary limitations of the AWI project with respect to:

  • ^{Integration with regional economic talent development?}

  • ^{Organization and partnerships?}

  • ^{Service delivery?}

  • ^{Sustainabillity?}

• What have program participants found most helpful about services provided by grantees? What services were least helpful?

• What would participants like to see changed about this program?

15.3 Project Successes and Practices Worthy of Replication

• Which of projects services have been designed specifically to meet the needs of older workers?

• What recommendations does the project have about the design of services or aging workers?

  • ^{What recommendations does the project have about specific designs that work well for aging workers?}

  • ^{What recommendations does the project have about specific curricula or guides for workshop content that might be available to other projects serving aging workers?}

• What are the practices of this project that show most promise? What aspects of your program would you recommend that other projects emulate?

  • ^{How successful were grantees in recruiting a diverse array of partners?}

  • ^{Which partner relationships were the most successful and why?}

  • ^{What aspects of the program work particularly well in helping participants find and keep jobs?}

• What were the other main successes of this program initiative? How have you achieved these successes?

15.4 Summary of Project Challenges

• What were the key problems or challenges in administering the project?

  • ^{Did you encounter challenges coordinating the input of all partners into account when making key decisions?}

  • ^{Did any of the partnerships fail during the course of the project, and what were the possible reasons?}

  • ^{Did you encounter challenges in recruiting participants?}

• What challenges did you face in serving participants?

  • ^{What challenges did you face in placing participants in jobs?}

  • ^{What challenges did you encounter in helping participants keep and advance in their jobs?}

  • ^{What challenges did you face in tracking participants and recording their outcomes?}

15.5 Lessons Learned

• What are the most important lessons that you have learned as a result of operating the AWI project?

  • ^{What practical lessons and promising practices for the workforce investment system were identified during this project?}

  • ^{Do these lessons apply only to older workers, or more broadly?}

• In hindsight, what would you do differently if you were to start the project again?

15.6 Plans to Expand, Sustain, or Replicate Project Model Within State

• What are the essential program components of a successful AWI service model?

• Do you plan to replicate your program or service model? If so, where and how?

• What advice do you have for replication of your service design for aging workers?

  • ^{Would you recommend this model to others?}

  • ^{What are the key challenges to replicating this model for serving aging workers?}

  • ^{What changes would you recommend others make before replicating your model?}

• For whom do you think your approach is best suited (e.g. under what economic conditions and with what types of aging workers do you think your approach will be effective)?

• What are your strategies to continue the services provided by the grantee after the program ends?

  • ^{To what extent have grantees sought outside funding to continue providing services? How successful have they been? What is the likelihood that services that the grantee provides to older workers will continue after grant funds run out?}

  • ^{From the project’s perspective, what are the highest priority services and service delivery approaches to try to continue after the grant ends?}

• What specific activities, programs, etc. targeted to older workers do you expect will be institutionalized by the public workforce system (or grantee, if grantee is not a WIB or One-Stop)?

  • ^{Do you expect that the workforce system will use WIA resources to provide services to support older workers after the grant ends?}

  • ^{If so, which agencies or staff are good candidates to provide those services?}

I.Site Visit Topics for Representatives of Partner Agencies (Round 1)

1. OVERVIEW OF PROGRAM INITIATIVE

1.1 Impetus for/ Purpose of the Project

• How did this project come about? What individual(s) or organization(s) were the chief instigators or initiators of the project proposal?

  • ^{Who were the key individuals and entities involved in writing the initial grant application to USDOL?}

  • ^{How did you select the grant recipient (grantee organization)?}

• What is the purpose of your project?

  • ^{What are the key challenges facing aging workers who would like to work for pay in your region? Which of these challenges is the project particularly focused on addressing?}

  • ^{What are the key challenges facing employers who need workers with the skill set to match current and anticipated jobs in high growth sectors?}

  • ^{Which of these challenges is the project particularly focused on addressing?}

• Why was the AWI grant announcement attractive to you?

  • ^{What were the perceived opportunities of the grant?}

  • ^{Were there any aspects of the grant requirements that were not such a good fit with your needs and interests?}

1.2 Goals of the Project

• How would you describe your project philosophy or approach?

• What program goals do you hope to achieve? To what extent are these goals quantifiable?

  • ^{How could these goals be assessed? (How will you know if you are successful?)}

  • ^{Have the goals of the project evolved or changed since the grant began?}

• What outcomes do you hope to achieve for individual participants?

• How are these outcomes measured (common measures, other ways)?

1.3 Planning Process

• What were the key steps in planning for this initiative? How much of the design was developed in the grant application? What was the planning process like after the grant was awarded?

• Who were the key players involved in the design of the project and what organizations/ entities did they represent?

  • ^{How engaged were different project partners (including LWIB board and staff, employers and employer associations, organizations specializing in aging worker services, and education and training providers in the planning process? How often did you meet?}

  • ^{Are there organizations that you wish had been at the table, in hindsight, and why?}

• How would you characterize the overall planning process? Did it go smoothly?

  • ^{What were the main issues of focus and/or concern during the planning process?}

  • ^{Were there any particular challenges during the planning of the project?}

• What was the effect of the economic recession on the project?

  • ^{How did the onset of the economic recession influence project planning and design?}

  • ^{How did the onset of the economic recession influence contribution of leveraged resources by project partners?}

  • ^{If partners have received additional funds from the Recovery Act, what effect have they had on contribution of leveraged resources?}

2. DESCRIPTION OF PROJECT SERVICE AREA

2.4 Competing Initiatives or Programs for Older Workers

• Are there other programs that offer similar services to older workers in the project service area?

  • ^{Are there programs that target some of the same populations as the AWI project or compete with it for enrollees?}

  • ^{Are services offered by other programs similar to or complementary to the services provided by the AWI project?}

  • ^{Could an individual participate in both programs simultaneously?}

• How does the demand for older worker services compare to the capacity of all local programs offering relevant services. (Are all available programs operating at full capacity or are they competing for customers? Are there waiting lists at the grantee and other similar programs?)

• If they can only participate in one program, why would/do individuals select the AWI program over others? Or why would/do they choose another program instead?

4. Developing Project Partnerships

4.4 Communication Between Grantee and Partners

• How does internal project communication work?

  • ^{How does the grantee’s project manager communicate with project partners? How frequently and about what types of issues? How do partners communicate with each other?}

  • ^{How do partners communicate with each other?}

• Does the grantee hold regular meetings with partners? If so, what is the purpose of these meetings? Are these meetings held individually or as a group? How long do they last?

  • ^{What types of information are discussed during these meetings?}

  • ^{How helpful are the meetings in assessing project status and guiding the project?}

4.5 Assessment of Partner Involvement

• How would you characterize the overall success involving partners to participate in the initiative?

  • ^{How pleased are you with the final composition of partners in the initiative?}

  • ^{Were there some agencies you initially wanted to partner with but were unable to do so? If so, what were the barriers in establishing that arrangement? Was another agency involved to fill that gap, and if not, is the initiative lacking in some way without that partner?}

  • ^{Are there any other partners that in hindsight you wish were involved in the initiative that are not involved?}

• Please describe any challenges you have experienced in developing effective relationships between the project partners.

  • ^{What challenges did you experience in recruiting partners or in arranging for their specific role in the project?}

  • ^{What challenges did you experience in the quality of the contributions made by individual partners?}

• What are the most successful aspects of your partnership?

• What advice do you have for other projects serving older workers in terms of partnership formation and partner roles?

4.6 Informal Relationships with Other Entities Serving Project Participants

• Please describe any informal relationships that the project has developed with other entities to expand the services available to project participants.

  • ^{To what extent do the workers recruited for this program receive services from other community agencies or programs?}

  • ^{What other agencies and/or programs are involved—SCSEP, One-Stop Centers (VT, CA and TX), Area Agencies on Aging?}

  • ^{To what extent are participants referred to these agencies? What proportion of participants access these services?}

  • ^{To what extent are participants referred from other agencies?}

• What are the experiences of participants referred to these providers?

• Would the project have benefited from more formal relationships with these agencies?

5. PROJECT PARTNERS AND THEIR ROLES

5.1 Involvement of the Public Workforce Investment System (e.g. One-Stop Centers and Constituent Programs)

(If grantee is a local WIB, describe the involvement of One-Stop staff and programs that are not directly part of the funded AWI project.

• What role did/do public workforce investment partners play in designing, managing, or overseeing the grant, if any?

• What roles do public workforce investment partners play in providing services to employer or worker customers?

• How much involvement does the AWI project have with other services provided at the One-Stop Career Centers?

  • ^{Do AWI project participants utilize WIA core and intensive services? If so, for what types of services? Are workers served under AWI co-enrolled in the One-Stop system (or SCSEP) under WIA?}

  • ^{Are other workforce investment partners involved in providing services to AWI project participants? What is the nature of that involvement?}

• How does the AWI project fit in with other workforce initiatives, such as, SCSEP or Medicaid Infrastructure Grants (MIG)?

  • ^{Are the services offered across these initiatives substitutes or complements?}

  • ^{Do AWI participants need services available through SCSEP? Can AWI participants be co-enrolled in SCSEP or participate in any of its services?}

  • ^{Describe the level of coordination and communication across the agencies administering these initiatives (AWI, SCSEP and Medicaid Infrastructure Grants (MCI)?}

• How have public workforce investment partners contributed to the success of the project?

^{Note: MCI grants from the US Department of Health and Human Services are intended to develop a comprehensive system of employment supports for people with disabilities.}

5.2 Involvement of Organizations with Expertise on Aging

• What role did/do “aging organizations” play in designing, managing, or overseeing the grant, if any?

• What other roles do “aging organizations” play in providing services to employer or worker customers?

• How have “aging organizations” contributed to the success of the project?

5.3 Involvement of Educational Institutions and Training Providers

• What role did/do educational institutions or training providers play in designing, managing, or overseeing the grant, if any?

• What other roles do educational institutions or training providers play in providing services to employer or worker customers?

• How have education and training institutions or training partners contributed to the success of the project?

5.4 Involvement of Economic Development Entities

• What role did/do economic development entities play in designing, managing, or overseeing the grant, if any?

• What other roles do economic development entities play in providing services to employer or worker customers?

• How have economic development entities contributed to the success of the project?

5.5 Involvement of Local Employers, Employer Associations, or Business Intermediaries

• What role did/do employers or business intermediaries play in designing, managing, or overseeing the grant, if any?

• How did you recruit business partners? What were your selection criteria, if any?

• What other roles do employers or business intermediaries play in providing services to employer or worker customers? Are they involved in services to their own incumbent workers; in directly recruiting and hiring project participants, or in some more general way?)

• How have employers or business intermediaries contributed to the success of the project?

• How has employer involvement been affected by the economic recession?

5.6 Involvement of Other Partners
(e.g., faith-based organizations, community organizations, philanthropic institutions, apprenticeship programs, tribal organizations, SCSEPgrantees)

• What role did/do other partners play in designing, managing, or overseeing the grant, if any?

• What other roles do other partners play in providing services to employer or worker customers?

• How have other partners contributed to the success of the project?

II.Site Visit Topics for Representatives of Partner Agencies (Round 2)

1. OVERVIEW OF PROGRAM INITIATIVE

1.1 Impetus for/ Purpose of the Project

• How has the purpose of the project evolved over time, if at all?

  • ^{What are the key challenges facing aging workers who would like to work for pay in your region? Which of these challenges is the project particularly focused on addressing?}

  • ^{What are the key challenges facing employers who need workers with the skill set to match current and anticipated jobs in high growth sectors?}

  • ^{Which of these challenges is the project particularly focused on addressing?}

1.2 Goals of the Project

• How would you describe your project philosophy or approach?

• What program goals do you hope to achieve? To what extent are these goals quantifiable?

  • ^{How could these goals be assessed? (How will you know if you are successful?)}

  • ^{Have the goals of the project evolved or changed since the grant began?}

• What outcomes do you hope to achieve for individual participants?

• How are these outcomes measured (common measures, other ways)?

2. DESCRIPTION OF PROJECT SERVICE AREA

2.4 Competing Initiatives or Programs for Older Workers

• Are there other programs that offer similar services to older workers in the project service area?

  • ^{Are there programs that target some of the same populations as the AWI project or compete with it for enrollees?}

  • ^{Are services offered by other programs similar to or complementary to the services provided by the AWI project?}

  • ^{Could an individual participate in both programs simultaneously?}

• How does the demand for older worker services compare to the capacity of all local programs offering relevant services. (Are all available programs operating at full capacity or are they competing for customers? Are there waiting lists at the grantee and other similar programs?)

4. Developing Project Partnerships

4.4 Communication Between Grantee and Partners

• How does internal project communication work?

  • ^{How does the grantee’s project manager communicate with project partners? How frequently and about what types of issues? How do partners communicate with each other?}

  • ^{How do partners communicate with each other?}

• Does the grantee hold regular meetings with partners? If so, what is the purpose of these meetings? Are these meetings held individually or as a group? How long do they last?

  • ^{What types of information are discussed during these meetings?}

  • ^{How helpful are the meetings in assessing project status and guiding the project?}

  • ^{How has communication between grantees and partners changed since the first site visit?}

4.5 Assessment of Partner Involvement

• How would you characterize the overall success involving partners to participate in the initiative?

  • ^{How pleased are you with the final composition of partners in the initiative?}

  • ^{Were there some agencies you initially wanted to partner with but were unable to do so? If so, what were the barriers in establishing that arrangement? Was another agency involved to fill that gap, and if not, is the initiative lacking in some way without that partner?}

  • ^{Are there any other partners that in hindsight you wish were involved in the initiative that are not involved?}

• Please describe any challenges you have experienced in developing effective relationships between the project partners.

  • ^{What challenges did you experience in recruiting partners or in arranging for their specific role in the project?}

  • ^{What challenges did you experience in the quality of the contributions made by individual partners?}

• What are the most successful aspects of your partnership?

• What advice do you have for other projects serving older workers in terms of partnership formation and partner roles?

4.6 Informal Relationships with Other Entities Serving Project Participants

• Please describe any informal relationships that the project has developed with other entities to expand the services available to project participants.

  • ^{To what extent do the workers recruited for this program receive services from other community agencies or programs?}

  • ^{What other agencies and/or programs are involved—SCSEP, One-Stop Centers (VT, CA and TX), Area Agencies on Aging?}

  • ^{To what extent are participants referred to these agencies? What proportion of participants access these services?}

  • ^{To what extent are participants referred from other agencies?}

• What are the experiences of participants referred to these providers?

• Would the project have benefited from more formal relationships with these agencies?

5. PROJECT PARTNERS AND THEIR ROLES

5.1 Involvement of the Public Workforce Investment System (e.g. One-Stop Centers and Constituent Programs)

(If grantee is a local WIB, describe the involvement of One-Stop staff and programs that are not directly part of the funded AWI project.

• What role did/do public workforce investment partners play in designing, managing, or overseeing the grant, if any?

• What roles do public workforce investment partners play in providing services to employer or worker customers?

• How much involvement does the AWI project have with other services provided at the One-Stop Career Centers?

  • ^{Do AWI project participants utilize WIA core and intensive services? If so, for what types of services? Are workers served under AWI co-enrolled in the One-Stop system (or SCSEP) under WIA?}

  • ^{Are other workforce investment partners involved in providing services to AWI project participants? What is the nature of that involvement?}

• How does the AWI project fit in with other workforce initiatives, such as, SCSEP or Medicaid Infrastructure Grants (MIG)?

  • ^{Are the services offered across these initiatives substitutes or complements?}

  • ^{Do AWI participants need services available through SCSEP? Can AWI participants be co-enrolled in SCSEP or participate in any of its services?}

  • ^{Describe the level of coordination and communication across the agencies administering these initiatives (AWI, SCSEP and Medicaid Infrastructure Grants (MCI)?}

• How have public workforce investment partners contributed to the success of the project?

^{Note: MCI grants from the US Department of Health and Human Services are intended to develop a comprehensive system of employment supports for people with disabilities.}

5.2 Involvement of Organizations with Expertise on Aging

• What role did/do “aging organizations” play in designing, managing, or overseeing the grant, if any?

• What other roles do “aging organizations” play in providing services to employer or worker customers?

• How have “aging organizations” contributed to the success of the project?

5.3 Involvement of Educational Institutions and Training Providers

• What role did/do educational institutions or training providers play in designing, managing, or overseeing the grant, if any?

• What other roles do educational institutions or training providers play in providing services to employer or worker customers?

• How have education and training institutions or training partners contributed to the success of the project?

5.4 Involvement of Economic Development Entities

• What role did/do economic development entities play in designing, managing, or overseeing the grant, if any?

• What other roles do economic development entities play in providing services to employer or worker customers?

• How have economic development entities contributed to the success of the project?

5.5 Involvement of Local Employers, Employer Associations, or Business Intermediaries

• What role did/do employers or business intermediaries play in designing, managing, or overseeing the grant, if any?

• How did you recruit business partners? What were your selection criteria, if any?

• What other roles do employers or business intermediaries play in providing services to employer or worker customers? Are they involved in services to their own incumbent workers; in directly recruiting and hiring project participants, or in some more general way?)

• How have employers or business intermediaries contributed to the success of the project?

• How has employer involvement been affected by the economic recession?

5.6 Involvement of Other Partners
(e.g., faith-based organizations, community organizations, philanthropic institutions, apprenticeship programs, tribal organizations, SCSEPgrantees)

• What role did/do other partners play in designing, managing, or overseeing the grant, if any?

• What other roles do other partners play in providing services to employer or worker customers?

• How have other partners contributed to the success of the project?

III.Topics for Data Reconnaissance Phone Calls to Projects (Fall 2011)

Data Collection System

• Are you using the Aging Worker Data (AWD) system to collect client-level data on Aging Worker Initiative (AWI) participants?

  • ^{If using AWD:}

    • ^{Why did you decide to use AWD?}

    • ^{Are you using any other systems to collect data for AWI participants? If so, why? What kinds of data are collected in this/these system(s)?}

  • ^{If not using AWD:}

    • ^{Did you attempt to use AWD?}

    • ^{Why are you not using AWD?}

    • ^{What data system(s) are you using to collect data for AWI? (Describe type of system(s) and their uses.)}

Data Collection Challenges

• (If applicable) What challenges did you face in using AWD?

  • ^{Were you able to overcome these challenges? How so?}

• What could be done to make AWD more user-friendly?

Data Collected

• What types of participants are you collecting participant-level data on? Only trainees? Trainees and non-trainees? Others?

• To what extent are you capturing data on participants’ use of services? Is this happening consistently?

  • ^{Are you entering all services in the participants receives into AWD? If not, what ones are excluded?}

  • ^{Are you entering the dates of completion of training?}

  • ^{Are you entering the receipt of a degree/certificate?}

• To what extent are you capturing data on participants’ employment outcomes? Is this happening consistently?

  • ^{Employer information?}

  • ^{Employment start dates?}

  • ^{Unsubsidized employment?}

  • ^{Training-related employment?}

    • ^{NAICS code?}

  • ^{Hourly wage?}

  • ^{Hours worked weekly?}

• Are you capturing educational and employment outcomes that occur after the quarter in which training is completed? [Because the 9134 captures only educational and employment outcomes that occur in the quarter in which training is completed, it is possible that some grantees may not be capturing outcomes after this period.]

  • ^{If so, where and how is this being done? In AWD or some other location?}

• What are your “follow-up” procedures? How frequently do you follow-up with participants?

  • ^{Do you enter data on employment outcomes after they have finished training?}

  • ^{Do you enter data on employment after participants’ have exited? How frequently do you do this? Every quarter?}

  • ^{Do you verify employment? If so, how?}

Availability of Data

[Per the evaluation contract, we will be collecting data from all grantees on AWI participants close to the end of the grant period.]

• If using AWD:

  • ^{We will need you to produce an “Evaluation Extract” from AWD. This feature is located in the Administrator Menu. It will produce an Excel spreadsheet of all AWI participant records recorded in AWD.}

• If using another system in addition to AWD:

  • ^{What kinds of data can you provide us from this other data system?}

  • ^{Is there another entity that we would need to coordinate with to obtain these data? Can you help facilitate that coordination?}

  • ^{What format are these data in?}

• If using only a system besides AWD:

  • ^{What kinds of data can you provide us from this data system?}

  • ^{How closely do these data mirror the data collected in AWD? [Respondent may not be aware of the data elements in AWD, in which case would need to review all sections of data collected in AWD.]}

  • ^{Is there another entity that we would need to coordinate with to obtain these data? Can you help facilitate that coordination?}

  • ^{What format are these data in?}

• Before sending us the data, we will need you to strip out any personally identifiable information (PII), including:

  • ^{Participants’ Social Security Numbers (“UID”)
    [DO NOT delete the “Client ID” number.]}

  • ^{Participant names}

  • ^{Participants addresses and phone numbers}

Participants’ employers’ names, addresses and phone numbersTopics for Final Telephone Call to Projects (June 2012)

1. To what extent would you say your project has achieved its goals?

• In what areas was the project most successful in achieving its goals? Why and how?

• In what areas was the project least successful in achieving its goals? Why and how?

2. What are the factors that helped and hindered project success (e.g. local context, partnerships, availability of training resources, effective service designs)?

3. How has the project influenced the quality and availability of services for older workers throughout the project region?

• Which of these influences are most likely to be sustained after the end of the grant?

• What would be needed to continue to prioritize services to older workers (e.g. dedicated resources, continued staff training, specialized curricula or specialized workshops targeted to older workers, continued interagency partnerships)?

4. What products have you developed to support dissemination of your approach in other areas?

• What particular aspects of your services or approach do these products document?

• How have you reviewed these products (e.g. expert review, review by AWI peers)?

• What is your strategy for sharing these products with other local areas?

5. What are the most important unmet needs of older workers in your region? What would be needed to address these needs?

6. What advice do you have for other local areas interested in improving responsiveness of One-Stop Career Center services for aging workers?

39