PAIMI Eval Supporting Statement 07 13 07

National Evaluation of the Protection and Advocacy for

Individuals with Mental Illness (PAIMI) Program

OMB Submission

THE SUPPORTING STATEMENT

A. Justification.

1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Mental Health Services (CMHS), has funded a National Evaluation of the Protection and Advocacy for Individuals with Mental Illness (PAIMI) Program. This evaluation is the first independent, external evaluation of the program since its inception in 1986. Consistent with federal efforts to address issues of accountability, capacity and effectiveness (such as the Program Assessment Rating Tool (PART)), the Human Services Research Institute (HSRI) has undertaken a three stage approach to independently evaluate the PAIMI Program: conduct an Evaluability Assessment, prepare an Evaluation Plan, conduct the evaluation and write a report of findings and recommendations.

This evaluation requests clearance of five separate data collection instruments: 1) Interview with P&A Executive Directors; 2) Survey of PAIMI Advisory Council Chairs; 3) Survey of PAIMI Clients; 4) Survey of State Mental Health Authority Directors; 5) Survey of Directors of Other Mental Health Advocacy Organizations (the latter to be nominated by the P&A Director, one of which will be a consumer run organization).

PAIMI Legislation

In 1975, the Developmental Disabilities Assistance and Bill of Rights Act (the DD Act)

[42 U. S. C 6041, et seq.], established the State protection and advocacy (P&A) system to protect and advocate the rights of persons with developmental disabilities. In 1986, the Protection and Advocacy for Mentally Ill Individuals (PAIMI) Act [42 U.S.C. 10801 et seq.] extended the DD Act protections to eligible individuals with significant (severe) mental illnesses (adults) and significant emotional impairments (children), at risk for abuse, neglect, and civil rights violations while residing in public or private residential care and treatment facilities.

The PAIMI Act provides formula grant support to 57 governor-designated P&A systems in each State, the District of Columbia, and the territories (the American Indian Consortium, American Samoa, Guam, Northern Mariana Islands, Puerto Rico, and the U.S. Virgin Islands). Administrative responsibility at the Federal level rests with the Administration on Developmental Disabilities (ADD) in the Administration for Children and Families (ACF), Department of Health and Human Services (DHHS) for the DD Act. The Center for Mental Health Services (CMHS) in the Substance Abuse and Mental Health Services Administration (SAMHSA), DHHS is responsible for carrying out the provisions of the PAIMI Act [ 42 U.S.C. 290bb-31].

The PAIMI Program provides funding to establish and operate a protection and advocacy (P&A) system for individuals with mental illness which will protect and advocate the rights of these individuals through activities that ensure enforcement of the Constitution and Federal and State statutes. Each P&A system is designated by the governor of the State, the District of Columbia (the Mayor), and the territories to receive formula grants to support legal-based advocacy services for PAIMI-eligible clients -- persons with significant mental illnesses (adults) and significant emotional impairments (children), at risk for, or in danger of, abuse, neglect, and civil rights violations while residing in public and private, treatment or care facilities.

2. Purpose and Use of Information

The practical utility of this new project is to comply with the federal government’s PART requirement to provide for an independent program evaluation. This independent evaluation is the first one in the program’s history, and these surveys constitute the predominance of information to be gathered about the PAIMI Program.

Examples of the kinds of information to be collected, analyzed and distributed include information about PAIMI Program federal guidance and oversight, program operations, quality management, outreach to the community (including communities of color and other minorities), and outcomes at the individual client level and systems levels. Such information is not currently available to determine the impact of the program. All survey instruments will be shared with CMHS and can be used for future program evaluation.

Findings from the evaluation will be disseminated in a way that addresses the various needs of multiple stakeholders and maximizes the multiple uses to which the findings may apply. The range of stakeholders who have active interests in the PAIMI Program include:

* PAIMI program staff in CMHS who are charged with implementing the program, monitoring adherence, and quality improvement;

* Other federal agencies with substantive or evaluative interests overlapping with those of the PAIMI Program, including other P&A Programs; and

* PAIMI Programs themselves along with allied P&As (Governing Boards, Advisory Councils, Executive Directors, and staff) to promote program improvement at the organizational level.

Additional stakeholders also have active interests in the PAIMI Program:

* Executive and legislative staff who are concerned with oversight and management, including how the PAIMI program fits in with other DHHS programs and the degree to which the program contributes to the achievement of broad policy goals;

* Consumers and allied advocacy groups who are concerned about access to mental health services, protection of rights and the degree to which PAIMI Programs contribute to goals around recovery and quality of life; and

* SMHA commissioners and staff who are concerned with local policy development and the impact of the PAIMI Program at the state level.

Funds for this evaluation have already been set aside by CMHS; the contractor fully intends to complete the evaluation with the remaining funds.

3. Use of Information Technology

To minimize the burden to the PAIMI Programs and other stakeholders, extant data available will be gathered from CMHS and its contractor McFarland & Associates. New data collection will be gathered using a mixed media approach that is planned to improve data quality, agency efficiency, and responsiveness to the public:

• Telephone interviews with P&A agency Executive Directors

• On line surveys for electronic submission of data using Survey Monkey webware for surveys of PAIMI Advisory Council Chairs, State Mental Health Authority Directors, and Directors of Other Mental Health Advocacy Organizations

• Focus group meetings and surveys with PAIMI Clients

All web based surveys will comply with the Section 508 of the Rehabilitation Act to permit accessibility to people with disabilities. All data will be kept confidential and entered and stored electronically in secured files backed up on a regular basis and maintained by HSRI. HSRI expects a response rate of above 85%.

Surveys other than those with PAIMI Clients will be delivered using the electronic web surveys. For these respondents, the ease of an online survey that can be completed either in one or several sittings is attractive and respectful of their schedules. Surveys with PAIMI Clients will be conducted in face to face meetings in locations comfortable to the respondents. Meetings will be facilitated by experienced HSRI team members who can assist Clients with filling out surveys as needed. Accommodations will be offered to complete surveys verbally to assist those who may not have access or comfort with computers, or prefer to respond verbally, or need language interpretation. (Refer to Attachment I for an example web-based survey page displaying the OMB control number, expiration date, and burden statement.)

CMHS will determine the Evaluation Report’s distribution to the public and can make this report available on various websites including the contractor’s website.

4. Efforts to Identify Duplication

In conducting the Evaluability Assessment, HSRI explored the extent to which data exists in the system for evaluating performance. This assessment included analysis of duplication of information as well. The data collection proposed for this evaluation is not available elsewhere, is not duplicative, and is seen as critically valuable for assessing the effectiveness of this national program by the PAIMI Evaluation Advisory Workgroup.

Existing data that this evaluation has already collected and will evaluate are the annual Program Performance Reports (PPR) prepared by each PAIMI grantee. HSRI has collected PPRs for 2003 and 2004. HSRI’s analysis of these reports however, is evaluating both qualitative and quantitative information that has not been previously analyzed. (See Attachment II, Evaluation Plan, pages 15-24, for a detailed description of planned PPR analysis.)

5. Involvement of Small Entities (small business)

This item does not apply as small businesses or other small entities are neither involved in nor impacted by the program or its evaluation.

6. Consequences if Information Collected Less Frequently

Each selected respondent is asked to respond one time only. Participation is voluntary. If information is not selected from identified respondents, then the information collected could bias the sample and impact conclusions drawn.

7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

HSRI fully believes that this information collection fully complies with 5 CFR 1320.5(d)(2), though we will note an outside chance that a special circumstances exists due to the nature of sampling. This evaluation will sample 20 of the 57 PAIMI grantees. While all efforts will be made to draw a representative sample based on parameters identified through out the Evaluability Assessment and Evaluation Plan stages, there is a slight risk that any particular sample may not fully represent all programs and therefore could impact the generalizability of the data.

8. Consultation Outside the Agency

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on Thursday, January 26, 2006, Vol. 71, No 17, page 4367. To review written comments received please refer to Attachment IV, Public Comments Received as per FRN. To review an itemization of recommended changes to the surveys and changes incorporated by the contractor, please refer to Attachment III, Recommendations Received through FRN Process & Resulting Changes to PAIMI Evaluation Surveys.

Below are summaries of the outside consultations regarding proposed PAIMI evaluation data collection instruments and methodology:

National Association of State Mental Health Program Directors (NASMHPD)

NASMHPD’s Legal Division created a committee to provide comments on the proposed evaluation, which they note is being directed by the Advocates for Human Potential, not the Human Services Research Institute.  NASMHPD’s comments consist of 3 suggested changes and 1 request. Suggested changes are: 1) Inclusion of a state mental health agency’s legal counsel as part of the agency staff being questioned; 2) add check box to Question 17 to indicate that the issue between the PAIMI organization and the agency was dealt with through formal negotiations and a settlement agreement short of litigation; 3) Data be supplied to reflect the PAIMI organizations role in the initiation and settlement of an issue and/or lawsuit, including but not limited to, whether the PAIMI organization is a party or representing a party as either legal counsel, advocate or both. The request is to receive a copy of the evaluation report. Phone: 703-739-9333. The contractor adopts all recommendations.

Barbara Hill Francis, AAG, Counsel, Maryland Mental Hygiene Administration

Assistant Attorney General representing Maryland’s Mental Hygiene Administration suggests that attorneys who interface with the PAIMI Program and who represent the mental health authorities be included in the survey. This recommendation parallels one from the NASMHPD comment above. Phone: 800-735-2258. HSRI will expand the evaluation to include a survey of attorneys representing mental health authorities.

Nan Ellen D. East, Executive Director, Disability Rights Center, Inc., the Arkansas P&A

Comment from the Executive Director of the Arkansas P&A on behalf of the PAIMI Advisory Council Chair, Joyce Soularie. Advisory Council Chairperson’s comment appears to confuse this independent evaluation with other review processes conducted by CMHS, probably the periodic site reviews and the now annual PPR Peer Review process. As for the survey instruments, Ms. Soularie notes that the questions and the evaluation process sound like a “fair way to get a good understanding of the PAIMI programs”. Phone: 501-296-1775 extension 104.

PAIMI Program Peer Reviewers: In person feedback on all surveys was provided to the contractor on 2/28/06 during the PPR Peer Reviewers meeting facilitated by CMHS.

Because we were uncertain about the response to the FRN, we determined that it would be useful to seek comment from an additional group of persons who have a range of experiences with the PAIMI program in the field. As a meeting of Peer Reviewers of the PAIMI Annual Program Performance Reports (PPR) was planned, HSRI initiated face to face interviews with 10 reviewers, including experienced P&A PAIMI staff, PAIMI Advisory Council Chairs, and consumer advocates. These reviewers were sent a copy of the survey in advance of the meeting. In the discussions, HSRI went through the surveys item-by-item soliciting comments, and also reviewed the overall conception of the survey and the likely burden.

Peer Reviewer Affiliation Phone

John Jones former P&A staff, AK 501-753-5332

Miriam Righter former PAIMI Advisory Council Chair, IA 319-646-5668

Aaryce Hayes P&A staff, TX 512-454-4816

Howard Trachtman consumer advocate, MA 781-642-0368

Dana Farmer P&A staff, FL 850-488-9071

Jeff McCloud PAIMI Advisory Council Chair, NC 252-527-0604

Joe Bevilacqua former MH commissioner in 3 states 401-635-2792

Pat Risser consumer advocate, OH (former PAC chair) 503-655-2530

Debbie Toler P&A staff, WV 304-346-0867

Kent Earnhardt PAIMI Advisory Council Member 919-832-0638

Evaluation Advisory Workgroup

The Evaluation Plan containing all data collection instruments has been reviewed by CMHS staff and the National Evaluation of the PAIMI Program’s Evaluation Advisory Workgroup (EAW), made up of 15 members representing program stakeholders of diverse backgrounds, interests, and experience with various components of the PAIMI program both as to local programs and nationally. EAW membership includes people with mental illness, family members of people with mental illness, P&A Executive Directors, former Commissioners of state mental health departments, PAIMI Advisory Council Chair, attorneys, technical assistance staff to the PAIMI Programs, and Federal staff. The EAW reviewed the instruments and provided feedback in writing and during two conference calls at the end of September 2005 held specifically to review the planned data collection processes and instruments. EAW members commented specifically on each survey and overall that the evaluation plan is well constructed and makes connections systematically. (For specific recommendations to changes in the surveys and HSRI’s response see Attachment III, Recommendations Received through FRN Process and Resulting Changes to PAIMI Evaluation Surveys.) Consultation with the EAW is an ongoing activity at significant stages of this evaluation. EAW members who provided feedback on the Evaluation Plan and surveys:

Bonnie Pate, Director SC Share, consumer representative, (803) 739-5712

Bob Glover, National Assoc. of State MH Directors, (703) 739-9333

Catherine Blakemore, Executive Director, California Protection & Advocacy, Inc.,

(916) 488-9955

Laura Zeigler, psychiatric survivor representative, (802) 454-1354

Joseph Bevilacqua, former commissioner of 3 state mental health agencies, (401) 635-2792

Peter Cubra, attorney representing individuals with psychiatric disabilities, (505) 256-7690

Stacie Hiramoto, minorities and persons of color representative, (916) 876-5149

Clarence Sundram, attorney, former P&A Director when Commissioner of NY’s Commission on the Quality of Care, (518) 527-1918

New Jersey Protection and Advocacy, Inc. Site Visit

Persons interviewed during the NJ P&A site visit also participated as survey respondents and critiqued survey tools. Suggestions for changes to the survey tools, particularly the Executive Director survey, predominantly were to add or refine questions, not to eliminate any items. Respondents noted the surveys were well crafted with generally plain language and written to the level of each type of respondent. (For specific recommendations to changes in the surveys and HSRI’s response see Attachment III, Recommendations Received through FRN Process and Resulting Changes to PAIMI Evaluation Surveys.)

NJ P&A staff: Phone: (609) 292-9742

Sarah Wiggins Mitchell, Executive Director

Joe Young, Assistant Director

Rachel Parsio-Corso, PAIMI staff

Louanne Lukens, PAIMI Director

Helen Dodick, PAIMI staff

Emmett Dwyer, Chief of Litigation

Other Site Visit Respondents:

John Wright, PAIMI Advisory Council Chair, (973) 778-8810

Ann Wright, Executive Director, Coalition of Mental Health Consumer Organizations (COMHCO), (973) 778-8810

Maria Verner, Director of Consumer Advocacy, Mental Health Association

Jack Boucher, Director, Collaborative Support Programs of NJ, (732) 780-1175

Syliva Axelrod, Executive Director, National Association of Mental Illness (NAMI), NJ, (732) 940-0991

Carol Beauchamp, Executive Director, Mental Health Association in NJ, (973) 571-4400

Terri Wilson, Deputy Commissioner, State Mental Health Authority, NJ Department of Human Services, (609) 292-1034

9. Payment to Respondents

This evaluation has a two tiered system with respect to respondent payment. With the exception of PAIMI Clients, no payments or gifts will be provided to respondents for their participation in this data collection. In the case of PAIMI Clients, those participating in the survey and focus group meetings will be reimbursed for their travel expenses and a token payment of $20.00 for their time to enable their participation. It is important to provide financial support to PAIMI clients as they typically live on much lower incomes than most Americans, are much more likely to have reduced access to transportation, and to be underemployed or unemployed due to disability. Reimbursement serves thus a practical reason, but in addition, serves as an acknowledgement that their time is valuable and is not significant enough to be considered as influencing opinion.

10. Assurance of Confidentiality

Participation in this evaluation is voluntary. All survey introductions include the purpose of the information collection and that this activity is sponsored by the Federal Government. Additionally, survey introductions inform respondents that their information will be kept strictly private, and further that they are free to skip any question that they do not wish to answer.

11. Questions of a Sensitive Nature

There are a couple of questions that could be identified as sensitive in nature, e.g., inquiring of respondents whether they self-identify as having a disability. These questions include a reminder that answering the question is voluntary. PAIMI Clients will not only read this instruction on their surveys but will be advised of this by the HSRI team member facilitating focus group meetings.

Information regarding disability status is relevant to this evaluation as the purpose of the PAIMI Program is to advocate as to the legal rights of individuals with psychiatric disability. Important to the Federal Government when passing the PAIMI legislation was that individuals with mental disability and their families have input into establishing program priorities. Thus, the extent to which people with disabilities are in positions of influence and contributing to the direction of the program is important to survey.

All surveys provide assurances that respondents have the right to refuse to answer any questions, and that the information they provide to us is confidential.

12. Estimates of Annualized Hour Burden

The basis for hour and cost burden estimates is based on testing of survey instruments and previous experience with similar data gathering activities.

Form Name	No. of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost ($)
Executive Director Interview	20	1	20	2	40	$52.88	$2115.20
PAIMI Advisory Chair Survey	20	1	20	1	20	0	N/A
State Mental Health Director Survey	20	1	20	1	20	$76.92	$1538.40
State Mental Health Legal Counsel	20	1	20	1	20	$67.30	$1346
Other Mental Health Advocacy Org Director Survey	20	1	20	1	20	$31.25	$625
PAIMI Client Survey	72	1	72	.50	36	0	N/A
Total	172		172		156		$5624.60

13. Estimates of Annualized Cost Burden to Respondents

There are no capital or start-up, operations, maintenance or purchase of services costs that exceed standard business expenses associated with participating in this evaluation.

14. Estimates of Annualized Cost to the Government

SAMHSA/CMHS has planned and allocated resources for the efficient and effective management and use of the information to be collected, including the processing of the information in a manner which shall enhance, where appropriate, the utility of the information to agencies and the public. The total cost of this evaluation, being performed under a Task Order, is $399,981, with the remaining budget of $115,000 allocated to data collection and reporting.

SAMHSA /CMHS estimates the GS-13 Government Project/Task Order Officer (GPO/TOO) principally involved in the oversight and analysis of this contracted evaluation has spent on average approximately 10% of his time (4 hours weekly) overseeing various components of this project. On an annualized basis this would be the equivalent of $12,000 in federal employee personnel costs (based on an annualized GS-13 salary of $80,000) over the last year and-a-half that the project has been underway. The GS-13 Government Project Officer has been working under the supervision of a GS-15 since the contract began. For this particular project, the year-and-a-half cost estimate associated with supervision is $1500 (1 hour weekly). Therefore, the contract costs and the personnel costs associated with data collection total $128,500.

The contractor has been reimbursed on a monthly basis for operational expenses consistent with the terms of their cost-reimbursement contract.

15. Changes in Burden

This is a new project.

16. Time Schedule, Publication and Analysis Plans

The current contract ends January 2, 2007. A final report is to be delivered by this date to CMHS. The attached Evaluation Plan includes a Task and Timing Chart specifying timelines for data collection, analysis, report delivery and presentation to stakeholders. (See Attachment II, Evaluation Plan, page 37 to review the Task & Timing Chart.)

Analysis Plan

The questions to be answered by the evaluation are outlined in the Evaluation Plan. (See Attachment II, Evaluation Plan, Evaluation Questions pages 6 through 14.) Almost all survey items are pre-coded; many are "yes" or "no" items. (Respondents will be given the opportunity to write clarifying comments regarding their responses.) There are no correct or anticipated answers to questions. Because of differences in program size, organization and culture, we expect to see differences among grantees. Furthermore, because HSRI intentionally has chosen respondents with different perspectives, we expect to see differences due to respondent type as well. Analysis will largely be through cross-tabulation of responses, comparing frequencies on similar items by the type of grantee and the type of respondent. Two sample table shells are shown below focusing on one aspect of the question of consumer involvement in planning PAIMI priorities for goals and objectives.

Sample Table Shells

To what extent are consumers of mental health services involved in the annual planning process for the P&A for its PAIMI program? (Percent “Yes” to be shown in table format.)

	Type of Respondent
	P&A Executive Director	PAIMI Advisory Council Chair	Director of other Advocacy Organization	State Mental Health Authority Director
The P&A holds open forums to which consumers are invited.
The P&A makes a substantial effort to conduct outreach to consumers.
Individual consumers participate.
Consumer organizations participate.

	Type of Grantee
	Annual Grant Amount		P&A Organization Type
	Low	High	State-operated	Not-for-profit
The P&A holds open forums to which consumers are invited.
The P&A makes a substantial effort to conduct outreach to consumers.
Individual consumers participate.
Consumer organizations participate.

17. Display of OMB Expiration Date

The expiration date for OMB approval will be displayed.

18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submission.

B. Collection of Information Employing Statistical Methods.

The contractor prepared an extensive Evaluation Plan for CMHS which is included in the attachments. Where relevant, sections of this plan are referred to below.

1. Respondent Universe and Sampling Methods

There are only 57 grantees for the PAIMI program; HSRI plans to include 20 in the sample. The sampling plan is purposive. In order to assure adequate representation of different types of programs, we intend to include both minimum grant states and larger grantees, states from different sections of the country, and grantees whose P&A programs are State-operated and those contracted to private not-for-profit legal advocacy organizations. The final sample will include at least five grantees in each category. It will be reviewed by CMHS staff prior to initiating surveys and interviews.

Regarding available sources of data, HSRI is conducting analysis of available sources of data for all PAIMI Programs across recent annual program reports. See Attachment II, Evaluation Plan, Available Sources of Data, pages 15-26 for further discussion of source of data and analysis plan.

2. Information Collection Procedures

As described in the attached Evaluation Plan, there are six surveys for each PAIMI program included in the sample. One is to be completed via a telephone interview (P&A Executive Director), four are to be completed on-line (State Mental Health Authority Director, State Mental Health Authority Legal Counsel, Director of the state chapter of the National Alliance of Mentally Ill and Director of a Consumer Advocacy organization) and one face-to-face as a part of focus groups of PAIMI Clients. These are described in the section of the Evaluation Plan titled "Additional Data Needed", pages 26 through 32 (See Attachment II). In addition, copies of the survey instruments are included in Attachment II as Appendices A through E. As described above, the sampling plan is purposive. (Note: The survey of State Mental Health Legal Counsel was not in the original evaluation plan. It has been added in response to a comment received from the National Association of State Mental Health Directors, following publication of the Federal Register Notice.)

Each survey respondent will receive an email request to respond to the survey or to schedule an interview. The "text" of this request will mirror the information already included on the first page of each survey. (See Attachment II, Evaluation Plan Appendices to review introductions for each survey.)

As described earlier (see response in Section 16), most analysis will require only cross tabulation of data. The sample size is not sufficient to justify more sophisticated, multivariate methods.

3. Methods to Maximize Response Rates

Response rates are estimated in the attached Evaluation Plan by type of survey and intended respondents. (See Attachment II, Evaluation Plan, pages 28-32.) Our plan to maximize response rates and minimize burden on respondents utilizes three strategies depending on the target respondent: a) in person phone calls to schedule interviews with Executive Directors, b) on site visits to solicit PAIMI Client feedback and communicate that their feedback is important and will be treated with strict confidentiality; c) and use of an online survey tool to conduct four of the proposed surveys.

The Survey Monkey software provides the user with a simple interface to transform a paper & pencil survey into a web-based survey. The software allows for multiple question types (yes/no, multiple choice, write-in responses, rating scales, etc.) and for skip patterns, also known as branching. In contrast to other survey sites that HSRI tested, the Survey Monkey site loads very quickly and appears to be very user-friendly for respondents.

Respondents will receive an email requesting their participation in the survey and offering them the choice of completing the survey online or requesting a paper survey. The email will include a link to the website where the survey can be completed. For those who request a paper and pencil survey, we will send it to them through the mail. For those for whom we do not have an email address or do not respond to the initial email, we will attempt to contact them through telephone and mail. The Survey Monkey has a feature that follows up with non-respondents prompting them to participate. We hope that the online survey option will increase both initial response rates as well as to continue to prompt respondents who do not refuse to complete the surveys. As well, we believe that our tactic to offer paper surveys or in person telephone interviews will yield higher than typical response rates.

4. Tests of Procedures

All survey instruments were pilot tested on site with the New Jersey Protection and Advocacy, Inc. during October 2005. Only minor changes were suggested, primarily consisting of word choice to clarify intent or recommendations to add questions to further explore an issue. To review suggested changes from the site visit and HSRI’s responses, refer to Attachment III, Recommendations Received through FRN Process & Resulting Changes to PAIMI Evaluation Surveys.

5. Statistical Consultants

Contractors: Phone:

Virginia Mulkern, Ph.D. Human Services Research Institute 617-876-0426

John Hornik, Ph.D. Advocates for Human Potential 518-475-9146

Deborah Potter, M.A. Human Services Research Institute 617-876-0426

Faith Sabel, M.A. Human Services Research Institute 617-876-0426

List of Attachments

Attachment I Web survey example page showing OMB control number, expiration date and burden statement.

Attachment II Evaluation Plan

Appendix A P&A Executive Director Interview

Appendix B PAIMI Advisory Council Chair Survey

Appendix C PAIMI Client Survey

Appendix D State Mental Health Authority Directors Survey

Appendix E State Mental Health Advocacy Organization Directors Survey

Appendix F CMHS Project Officers Interview

Appendix G NDRN Technical Assistance Staff Interview

Appendix H PAIMI Site Monitors Interview

Appendix I PAIMI PPR Peer Reviewers Interview

Attachment III Recommendations Received through FRN Process & Resulting Changes to PAIMI Evaluation Surveys

Attachment IV Public comments received as per FRN:

• National Association of State Mental Health Program Directors

• Disability Rights Center, Inc., Arkansas Protection and Advocacy

14