Form 7 P&A: Executive Director Interview

EXECUTIVE DIRECTOR INTERVIEW (WITH SELECTED STAFF, AS APPROPRIATE)

[Interviewer]: Thank you for taking the time to meet with us today. My name is [interviewer’s name] and this is [second interviewer’s name]. We both work for _______, a private research company in _______.

As we’ve already discussed, the Administration on Developmental Disabilities (ADD) is conducting an independent evaluation of Developmental Disabilities Network programs and collaboration among them. The purpose of this evaluation is to examine the impact of the Developmental Disabilities (DD) Network programs on the lives of people with developmental disabilities and their families. This evaluation is intended to provide feedback to the ADD to help improve the effectiveness of its programs and policies, to serve as a decision-making tool for ADD, and to promote accountability to the public.

Your program is one of ____ P&A programs in ____ states that has been randomly selected to participate in the evaluation. The evaluation will consist of interviews with key stakeholders (e.g., Executive Directors, P&A staff, P&A Board of Directors members, policymakers and P&A collaborators, recipients of P&A public education, and P&A clients). We will also be conducting the evaluation at UCEDDs and State Councils on Developmental Disabilities. We will summarize the information provided by interviewees (yourself included) in all programs to help us to examine the impact of the P&A system at the national level.

As the P&A executive director, you are an important source of information regarding the operations and achievements of this program. We have some specific questions to ask you about the different key functions of your P&A. The interview should last approximately 3-4 hours.

Before we start, we would like to ask your permission to record this interview so that we do not miss any of your responses to our questions. The recording will be used by ________ only; it will not be provided to ADD or anyone else.

Do you have any questions before we start?

PART A. PLANNING AND PRIORITY SETTING

We would like to begin by discussing the planning and priority setting process at your P&A and your Statement of Goals and Priorities (SGP). Specifically, we would like to talk about how you obtain input, the different kinds of input you use to develop the SGP, and how you use the SGP.

Process for developing goals and priorities

Let’s start with the process your P&A uses for developing the SGP each year, thinking about what happened during the last planning cycle.

1. What methods did you use to obtain input on gaps in and barriers to supports and services? Please describe.

Probe:

• Public meetings

• Focus groups

• Surveys

• Analysis of telephone calls to the P&A

• Input from staff

• Input from the P&A Advisory Council

• Other _________________________________________________________

2. Whom did you contact to participate in providing input?

Groups/individuals	Contacted ()	Provided input ()
People with developmental disabilities
Family members
Advocates
Service providers
Policy makers
P&A Advisory Council
DD Council
UCEDD(s)
Other, please specify

3. What groups/organizations/individuals provided your P&A with input for the last planning cycle?

4. What did you do to accommodate people with developmental disabilities so they were able to participate in the planning and priority setting process? Please describe.

Probe:

• Methods of contact

• Materials used

• Accessibility at meetings

• Assistance with materials

• Other, please specify ____________________________________________

5. How did you make sure you provided opportunities to participate to those who never participated before in the process? For example, please describe input sought from:

• People from both urban and rural communities;

• People throughout all parts of the state;

• People from ethnic/minority groups in your state;

• People who have difficulty understanding English;

• Others, please specify____________________________________________

6. What do you do to provide accommodation to those who need it in order to participate? Please give examples.

7. What type of existing data or reports did you use to identify the needs and interests of people with developmental disabilities? Please give examples.

Probe:

• Census data

• Findings from UCEDD reports [Obtain copy or references.]

• Findings from published literature [Obtain references.]

• Findings from DD Council Comprehensive Review and Analysis of the State Plan

[Obtain copy.]

• Findings from DD Council-supported demonstration projects [Obtain copy.]

• State agency reports [Obtain copy.]

• Other _________________________________________________________

[Obtain copy or references.]

8. In the last planning cycle, how did you use feedback about current activities to inform the planning and priority setting process? Please describe.

1. What kind of information did you collect about current activities (e.g., information on recipients’ satisfaction with P&A supported programs)?

2. How did you collect it?

3. From whom (e.g.,staff and contractor feedback)?

4. How did you use it? Give examples.

Results of planning process

9. What is the process you use to move from stakeholder input to statements of goals and priorities in your SGP? Please describe.

1. What do you do?

2. Who is involved?

Probe:

• Staff

• Members of the board of directors (commissioners)

• State officials

• People with developmental disabilities

• Family members

• Representatives from other DD Network programs

• Other _____________________________________________________

3. How do you make sure the end result is consistent with both stakeholder input and existing data regarding the needs of people with developmental disabilities? Please explain.

d. How do you make sure the end result is truly reflective of the findings from the planning process? Please describe.

I see the goals and priorities for this year are _______________________ (Refer to SGP).

10. How do these relate to the feedback you received from stakeholders? Please give examples.

Use of SGP to shape P&A activities

We’ve been told that the SGP is used to shape the activities of the P&A for the year.

11. How does your P&A use the SGP? Please describe.

Probe:

• To determine which individual advocacy cases to take

• To determine which systemic advocacy activities to support or sponsor

• To determine which community outreach activities to support or sponsor

• To determine which community education activities to support or sponsor

• Other

12. How do P&A forms and materials reflect the goals and priorities in the SGP? Please give examples.

Probe:

• Intake form

• Intake training materials

• Board of Directors/Commission orientation materials

• Outreach materials made publicly available

• Website

• Other ___________________________________________________________

13. How did your P&A deal with emerging and/or emergency issues that arose in the past year? Please give examples.

14. What was the rationale for taking on these emerging or emergency issuesnot covered in the SGP? Please explain.

PART B. INTAKE AND ASSISTANCE

The next section of our interview has to do with the intake process and decisions made about the best way to meet the needs of people who contact the P&A. Questions in this section involve procedures of your intake process and how you make sure they are being followed, the P&A’s ability to meet the needs of people who contact the P&A, and client satisfaction. Let’s start with the intake process.

Procedures of the intake process

15. Please describe the procedures that regulate the intake process at your P&A. Are the procedures written?

[Obtain copies of procedures and intake form.]

Probe:

• Requirements on the collection of basic intake information

• Criteria for determining the amount of information to collect on each type of caller (e.g., those for whom a file is opened; those who require information and/or referral only)

• Maximum response times

• Circumstances under which a person is referred to a P&A lawyer/advocate to receive individual advocacy

• Other _________________________________________________________

16. Do P&A staff members receive training on P&A intake procedures? Please describe.

Probe:

• Arranged training on the intake procedures for all new intake staff

• Included people with developmental disabilities and family members in training

• Included training on cultural competency¹

• Arranged for existing intake staff to receive review training periodically

• Other _________________________________________________________

17. How does P&A management monitor the extent to which intake staff appropriately follows the intake procedures?

Probe:

• Nature of monitoring process (e.g., data review; performance review)

• Frequency of monitoring

• Other

18. How well do intake staff members adhere to the P&A policy and/or procedures? Please explain.

Probe:

• Completion of forms

• Maximum response times to requests for information

• Statistics kept

Satisfaction of P&A clients

We understand that some P&As collect information on satisfaction of their clients, so we’d like to understand what this P&A does with regard to assessing the satisfaction of those who are served by the P&A.

19. How do you measure client satisfaction? Please describe.

1. How does the P&A collect satisfaction data?

Probe:

• Survey (mail out or telephone)

• Form given to every client who exits? Given randomly?

• Unsolicited letters

• Other _________________________________________________________

2. What kind of satisfaction is ascertained (i.e., satisfaction with what)?

[Obtain forms, questionnaires, or other documentation.]

20. How and when do you inform callers and clients about the grievance procedure?

Probe:

• Provision of information on the grievance process to those who are denied P&A services beyond information and referral.

• Provision of information on the grievance process to those who are taken on as clients.

PART C. INDIVIDUAL ADVOCACY

We would like to turn now to the topic of individual advocacy. Let’s start by talking about client objectives and needs, how they are determined and monitored, and the extent to which they are achieved.

Client objectives and needs

21. What are some of the things P&A staff, attorneys, advocates, or contractors did in the past year to adhere to clients’ preferred mode of communication? Please give examples.

Probe:

• Used interpreters (including sign language interpreters)

• Provided personal assistance for those with a cognitive disability

• Communicated in person

• Communicated both in person and in writing

• Spoke/wrote in a language other than English

• Used large print

• Other _________________________________________________________

P&A representatives need to make sure their clients understand the issues that need to be resolved and what the P&A is going to do to resolve them.

22. In the past year, how did the P&A make sure the client understood the process and results throughout the representation process? Please give examples.

23. What does the P&A do to make sure it is meeting the objectives of its clients?

1. Does the P&A use a client representation agreement to identify and document client objectives? Please describe. Does the agreement follow a written, standard format? Please describe.

Probe:

• Lists issues P&A will address for the client

• Lists client’s objectives

• Describes process by which P&A will attempt to meet client’s objectives

• Provides an estimated timeframe

• Other

[Obtain 3 examples.]

2. Is a closing letter included in clients’ file to document the processes and outcomes of the assistance clients received? Please describe.

Probe:

• Under what circumstances is a closing letter prepared?

• What information is included?

Probe:

• documented objectives

• agreements made by P&A

• what the P&A did

• estimated timeframe

[Obtain 3 examples.]

3. In the past year, did staff members discuss cases and issues during staff meetings? Please describe.

Probe:

• Frequency of discussions

• Format (formal or informal)

• Other _________________________________________________________

Confidentiality

25. How does the P&A maintain the confidentiality of individual advocacy clients? Please describe.

Probe:

• Use of private offices or rooms

• Password protection of electronic files

• Secure storage of files that contain personal/confidential information

• Other _________________________________________________________

b. How does the P&A ensure that staff members are adequately familiar with confidentiality policies and procedures? Please describe.

1. How does the P&A ensure that staff members are complying with confidentiality policies and procedures? Please describe.

PART D. SYSTEMATIC ADVOCACY

The next section of the interview has to do with systemic advocacy. Questions in this section will address systemic advocacy strategies, collaboration to meet systemic advocacy goals, and the outcomes of P&A systemic advocacy efforts. Let’s start by talking about systemic advocacy strategies used by the P&A.

Systemic Advocacy Strategies

28. First, please give us 3 examples of a systemic advocacy project or activity you took part in over the past year.

[Use table. Interviewer: Work through one project or activity at a time.]

1. Please explain what the systemic advocacy issue was.

2. Please describe the type of projects or activities (strategies) that were involved.

3. Who were the collaborators on each issue?

4. What did each collaborator contribute?

5. What role did the P&A play (i.e., lead role or participant)?

Examples of systemic advocacy issues

Ex.	Issue (Q. 28a)	Systemic Advocacy Project/Activity (Q. 28b)	Collaborators (Q. 28c)	Contributions of Collaborators (Q. 28d)	P&A Lead (L) or Participant (P) (Q. 28e)
1.
2.

Probe:

• Issue

• Activity

• Collaborators (groups, individuals)

• Contributions of each collaborator

• Whether P&A took a lead or supportive role

[INTERVIEWER: REPEAT QUESTION 28 FOR 2 MORE EXAMPLES.]

29. Thinking about your systemic advocacy activities in the past year, what different strategies did your P&A use to meet systemic advocacy goals in addition to ones mentioned as examples? Please describe.

Probe:

• Individual litigation

• Writing position papers

• Obtaining press coverage

• Informing policy makers

• Including people with developmental disabilities/family members in the process

• Providing testimony

• Securing pro bono law firms to assist

• Drafting legislation

• Other ___________________________________________________________

30. In supporting the systemic activities of other organizations in the past year, what did the P&A contribute? Please give examples.

Probe:

• Staff time

• Funding

• Location for meetings

• Other_________________________________________________________

Outcomes of systemic advocacy efforts

We would now like to talk about some of the outcomes of P&A systemic advocacy activities.

31. In what ways have systemic advocacy activities facilitated system changes or prevented system changes from occurring over the past 3-5 years; 5-10 years? Please describe.

Probe:

• Supported, enhanced, or prevented changes to legislation or statewide or local public policy

• Supported, enhanced or maintained changes to statewide or local community practice or services

• Other _________________________________________________________

PART E. COMMUNITY OUTREACH AND EDUCATION

This section of the interview is about P&A community outreach² and community education.³

Community Outreach

We’ll start with questions on community outreach first – what your outreach activities look like, the audiences these activities target, and your success in reaching the audiences you intended to reach.

Community outreach consists of efforts made to connect or bring awareness of the P&A and P&A services to organizations, groups, or the general public. One goal of community outreach is to identify typically unserved and underserved populations who might benefit from P&A services and programs.

32. In the past year, how did your P&A reach out to the public or potential clients to increase access to P&A services for traditionally unserved or underserved populations? Please describe.

Probe:

• Use of media

• Use of P&A web site

• Presentations at meetings/conferences

• Distribution of brochures [Obtain a copy of brochures/announcements used in the past year.]

• Other _________________________________________________________

33. What audiences did the P&A target last year for outreach?

Probe:

• African-American communities or other minority populations

• Specific ethnic populations

• Native American Indian populations

• People with specific types of disabilities and their families

• Specific communities (e.g., low income, rural)

• Other _________________________________________________________

33. How were these targets selected? Please explain.

33. What does your P&A do to make sure your outreach reaches traditionally unserved or underserved populations or communities?

• Collaborates with other individuals or groups in order to reach unserved and underserved populations (e.g., disability-related organizations/agencies, local tribal leaders, local political or community leaders, etc.)?

Probe:

• Which organizations/individuals?

• What do these organizations/individuals do?

• Other _______________________________________________________________

Community Education

Let’s move on to your community education activities. We think of community education as informing the public at large and people with developmental disabilities about the rights and value of people with developmental disabilities in their community and empowering people with developmental disabilities to advocate for themselves and others.

We’ll ask you what your community education activities consisted of, the target audiences of these activities, how you determined which organizations or individuals to target for community education, and how you made sure that your community education activities were culturally competent and appropriate to each target audience.

33. Last year, what training and education did you provide or support for people with developmental disabilities and family members? Please give examples.

Probe:

• Provided training and education for people with developmental disabilities and family members to advocate for themselves

• Referred people with developmental disabilities to local or state self-advocacy organizations

• Identified potential participants in training and education for people with developmental disabilities and family members (e.g., Partners in Policymaking)

• Other __________________________________________________

33. What activities does your P&A provide or support to the community at large to improve access, inclusion, and choice? Please give examples.

Probe:

• Presentations to employers on hiring people with developmental disabilities or reasonable accommodation

• Awareness training to service providers on client or patient rights

• Technical assistance to attorneys on disability-related legislation

• Other ____________________________________________________________

33. In the past year, what audiences were addressed?

Probe:

• People with developmental disabilities

• Family members of people with developmental disabilities

• Media

• Policy makers

• Service providers

• Disability-related organizations

• Non-disability related organizations that interact with people with disabilities and their families

• Employers

• Health care providers

• Teachers

• Others _________________________________________________________

3. What does your P&A do to make sure community education activities are conducted or provided in a culturally competent and appropriate manner? Please give examples.

[Obtain examples of materials.]

39. How do you make sure public education materials are appropriate to the target audience? Please describe.

Now we’d like to ask you some questions on the results of your community education activities.

40. How does the P&A learn about the effectiveness of its community education activities? Please describe.

41. How do recipients of P&A community education use the information provided? Please give examples.

Probe:

• People with developmental disabilities obtain services for themselves.

• Recipients provide accommodations for employees with developmental disabilities.

• Recipients make use of tax incentives for assistive technology for employees.

• Recipients make public buildings more accessible.

• Recipients improve the quality of their interactions with people with developmental disabilities and their families (e.g., classroom instruction and policies, accessibility of their workplace; accessibility of community events).

• Recipients seek and make changes to policies that benefit people with developmental disabilities.

• Recipients increase the number of people with developmental people they serve or hire.

• Other _________________________________________________________

PART F. GOVERNANCE AND MANAGEMENT

Now we would like to talk about how your P&A is governed and managed.

Board of Governors (Commissions)

First, we have questions about your board of directors (commissions) and its role in your P&A. Even if you do not have a board of directors, we are interested in learning about how your P&A is governed and how certain functions are covered without a board.

As we understand it, your P&A does (does not) have a board of directors.

42. Please describe the form of governance under which your P&A operates (e.g., part of a state agency, a non-profit organization, no board of directors/commission).

If no board of directors:

43. Is there another type of group that fills some of the roles of a board of directors (e.g., commissioners)? Please describe.

[Interviewer: if no board of directors or commissioners, go to question 46.]

For P&As that have a board of directors or commissioners:

43. Does the P&A provide any supports to the members of the board of directors (commissioners) to help them fulfill their roles and responsibilities? Please describe.

Probe for:

• Regular reporting to the Board of Directors or Commission

• Orientation

• Accommodations for people with developmental disabilities

• Reimbursement for P&A-related travel when requested

• Provision of written background materials in advance of meetings

• Provision of a support person to help with preparation and understanding efore and during meetings for those who required it

• Provided other accommodations

• Provided an orientation to new Board members on their responsibilities, the DD Act, and other governing Federal requirements

• Other _________________________________________________________

If no board of directors or commissioners:

45. We recognize that there is no board of directors or commissioners at your P&A. How does your P&A cover some of the functions typically fulfilled by a board of directors or commissioners?

Probe:

• Oversight

• Policy advice

• Business advice

• Understanding of needs of people with developmental disabilities and family members

P&A Infrastructures for Efficiency and Effectiveness

46. How does the P&A make sure it has qualified staff to effectively provide services, including services to typically unserved or underserved populations and communities?

Probe:

• Conducts staff performance review each year

• Uses experienced staff to assist or mentor less experienced staff

• Hires people with personal experience with people or communities that are typically unserved or underserved (e.g., racial and ethnic minorities; people whose first language is not English; people with a developmental disability, including a cognitive disability)

• Other _________________________________________________________________

46. How does the P&A facilitate responsiveness to emerging issues and emergency issues that arise suddenly?

1. What structures and procedures do you have in place? Please describe.

2. In the past year, how did the P&A deal with emerging and emergency issues that arose suddenly? Please give examples.

46. In the past year, how did the P&A ensure sound fiscal management? Please describe.

46. What evidence do you have to demonstrate that the P&A is financially sound?

[Obtain results of fiscal audit from the past calendar year.]

P&A Adherence to the Principles and Goals of the DD Act

46. What are some of the things you do to maintain the P&A’s independence from the governor and other senior state officials? Please give examples.

Probe:

• No restrictions on P&As ability or authority to litigate;

• P&A is not financially encumbered by office of the Governor or state developmental disabilities service system;

• Other __________________________________________________________________

46. To what extent are P&A facilities (in main and satellite offices) physically accessible? Please describe the ways in which P&A facilities are physically accessible.

Probe:

• Ramps at front entrance

• Wide doorways to accommodate wheelchairs

• Proximity to public transportation

• Other _________________________________________________________

52. In what ways is the P&A programmatically accessible? Please give examples.

Probe:

• Provision of technology and translators to communicate with clients

• Forms in Braille as needed

• Use of interpreters (including sign language interpreters) to communicate with clients

• Bilingual staff/volunteers

• Other _________________________________________________________

PART G. COLLABORATION

According to the DD Act, DD Network programs are required to report the extent to which they collaborate with one another. We’ve talked a little bit about collaboration as part of your systemic advocacy activities. Now we would like to focus on your collaborative efforts with the DD Network programs, as well as collaboration on other types of activities besides systemic advocacy.

For this evaluation, we define collaboration as a well-defined relationship entered into by two or more individuals, groups, and/or organizations to achieve common goals of the Developmental Disabilities Assistance and Bill of Rights Act of 2000 (the DD Act).

Let’s start first with how the DD Network programs identify issues and common goals on which to collaborate.

53. What process do the programs use for identifying, planning, and implementing collaborative efforts? Please describe.

Probe:

• Nature of process

• Participants

• Meeting frequency

• Formal vs. informal

• Other_______________________________________________________________

54. What are the issues and goals on which the DD Network programs are currently collaborating? Please describe.

[See following table.]

[Obtain written goals/objectives.]

Collaborations

No.	Project/Issue (Q. 54a)	Goals/Objectives (Q. 54b)	Collaborators (including P&A) (Q. 55)	Collaborators’ role (including role of P&A) (Q. 56)	Leadership (Single/joint) (Q. 57)	Resources Contributed (Q. 58)
a.
b.
c.

[INTERVIEWER: REPEAT QUESTIONS 81 THROUGH 86 FOR TWO MORE EXAMPLES.]

1. What project or issues are you collaborating on with DD Network programs? Would you please explain each issue?

2. What are the goals you and your collaborators were trying to achieve?

55. Which organizations or groups (including other DD Network programs) are you collaborating with on these issues?

56. What is the role of each collaborator, including the P&A and other DD Network programs?

56. How would you characterize the type of leadership for each project?

Probe:

• Single

• Joint

• Other ____________________________________________________________

Resources for Collaboration

56. Collaboration suggests shared resources being available and used. What resources did each group or organization contribute? Please describe.

Probe:

• Funding

• Staff time

• Meeting place

• In kind contributions

• Other _________________________________________________________

Process of collaboration – regular communication

Let’s talk about the process of communication by using the same examples.

56. What were some of the ways you achieved regular communication during your collaborative efforts? Please describe.

1. Sharing information and progress on ongoing efforts

Probe:

• Examples

• Frequency

• Recipients

• Other

2. Regular updates to participants in the collaborative effort

Probe:

• Examples

• Frequency

• Recipients

• Other

3. Reports on milestones achieved

Probe:

• Examples

• Frequency

• Recipients

• Other

4. Discussion of issues that arose

Probe:

• Examples

• Frequency

• Participants

• Other

• Debriefing at the end of a collaborative effort (if appropriate)

Probe:

• Examples

• Frequency

• Recipients

• Other

56. In the past year, were there any issues that arose in the community on which the DD Network programs responded together? Please give examples.

56. What was the joint response of the DD Network programs?

56. What is the process the DD Network programs use to monitor progress and barriers of collaborative efforts? Please describe.

1. Please give examples of ways in which the programs monitor and identify barriers.

2. For collaborative efforts implemented in the past 2 years, what process was used to identify leadership? What decisions were made on leadership of each effort?

3. What barriers have been identified?

4. Have goals and objectives been changed as a result of monitoring and identifying barriers? Please give examples.

Results of collaboration

Finally, let’s talk about achieving your goals and objectives -- the results of your collaborative efforts.

56. Were there any products derived from your collaborative efforts? Please give examples and describe?

Probe:

• White papers

• Wording for legislation

• Brochure on joint program

• Program enrollment form

• Other _________________________________________________________

[Obtain examples of products developed in past year, if applicable.]

56. What has been the result of your collaboration with DD Network programs and other organizations? Please give examples from the past 5 years.

Probe:

• Changes to (or maintenance of) State and local systems (e.g., changes or maintenance in practices and legislation)

• System changes (or maintenance) that affected people with developmental disabilities and their families, including people who are traditionally unserved and underserved (e.g., racial or ethnic minorities, people with little or no use of English, or people who live in remote or rural areas or live in poverty)

• Increases in the capacity of the community to provide supports and services to people with developmental disabilities and their families (e.g., accommodations are provided in public services such as transportation, education, and employment)

• Ways participants in DDN collaborative efforts have improved their knowledge or skills in ways that benefit people with developmental disabilities and their families

• Other _________________________________________________________

PART H: DEBRIEFING [AT END OF SITE VISIT]

56. Before we leave, we would like to give you the opportunity to ask any questions you might have, raise any issues you might want to raise, or talk about anything related to your program that you would like to expand upon.

1. First, do you have any general comments to make?

2. Issues?

3. Further explanation of your P&A?

56. There are also a few things I want to make sure we don’t forget.

Obtain materials requested, if easily available.

Confirm products to be sent. Leave FedEx form.

Obtain contact information for additional individuals who need to be contacted.

Confirm process for telephone interviews and availability of interviewees.

Leave business card.

We’d like to thank you again for taking the time to meet with us these past 2 days. We also appreciate you making your staff available and arranging for us to meet with many of your stakeholders.

1 According to the DD Act, cultural competency means “services, supports, or other assistance are conducted or provided in a manner that is responsive to the beliefs, interpersonal styles, attitudes, language, and behaviors of individuals who are receiving the services, supports, or other assistance. These services, supports, or other assistance are conducted or provided in a manner that has the greatest likelihood of ensuring the maximum participation of people with developmental disabilities and their families in the program involved” [sec. 102(7) of the DD Act].

2Community outreach consists of efforts made to connect or bring awareness of the P&A and P&A services to organizations, groups, or the general public. One goal of community outreach is to identify typically unserved and underserved populations who might benefit from P&A services and programs.

3Community education consists of informing the public at large and people with developmental disabilities about the rights and value of people with developmental disabilities in their community and empowering people with developmental disabilities to advocate for themselves and others.

S2D-12