Form Attachments 1-13 Attachments 1-13 Attachments 1-13

The Mental Health Transformation State Incentive Grant Cross-Site Evaluation

Attachments-1-13

The Mental Health Transformation state Incentive Grant Cross-Site Evaluation

OMB: 0930-0292

Document [doc]

Download: doc | pdf

Attachment 1

Logic Model

Consumer and Family Member Involvement

New Freedom Commission Goals and Recommendations

Goal 1 - Americans understand that mental health is essential to overall health. Goal 2 - Mental health care is consumer and family driven. Goal 3 - Disparities in mental health service are eliminated. Goal 4 - Early mental health screening, assessment, and referral to services are common practice. Goal 5 - Excellent mental health care is delivered and research is accelerated. Goal 6 - Technology is used to access mental health care and information

Narrative regarding MHT SIG Cross-site Evaluation Logic Model

This logic model offers a visual representation of the transformation process as currently viewed by the cross-site evaluation team. The logic model is intended to depict the commonalities among individual transformation processes while recognizing the diversity among the State MHT SIG projects. To these ends, the model represents the transformation process at its most general. In addition to the cross-site model, each State will create its own logic model tailored to the specific aspects of their efforts.

The placement of the NFC goals indicates that the goals will influence the entire transformation process. As a result, work associated with the elements in each of the columns will in turn contribute to the achievement of the NFC goals. At the same time, it should be noted that, while all the goals are relevant to each stage of the process, each goal does not necessarily relate to each component within a given stage. Because consumer and family member involvement is integral to each stage of the transformation process, it shares the NFC goals’ position.

Underneath the NFC goals and consumer and family member involvement are the columns representing the elements of each stage of the process. Reading the model from left to right, each stage’s elements are directed towards the accomplishment of the next stage’s elements, and the outcomes play a role in the inputs of the next round of the process. Note that final outcomes may occur after the conclusion of the grant period.

Recovery and resilience are related concepts, and are relevant to both adults and children/youth. Additionally, resilience relates to family members supporting their loved ones in the process of recovery. While resilience and recovery are listed among the primary outcomes of transformation, many consumer/survivors have noted that recovery is not an endpoint but an ongoing journey – one driven by the person in recovery rather than the service system. As such, outcomes related to these concepts might more accurately be described as “facilitation of resilience and recovery.”

Key Acronyms and Terms

MHT SIG	Mental Health Transformation State Infrastructure Grants
SMHA	State Mental Health Authority: responsible for administrating public mental health services within a given state
TWG	Transformation Working Group: guides the transformation process within each MHT SIG State
NARI	Needs Assessment and Resource Inventory: self-review document required of each MHT SIG State within the first year of the grant program
CMHP	Comprehensive Mental Health Plan: planning document required of each MHT SIG State within the first year of the grant program

Attachment 2

Discussion Guides and Associated Protocols for In-Person Site Visits

Discussion Guide for Site Visits by Type of State Agency Staff

MHT SIG Project Director

The effect of the grant on State agencies/mental health service systems. Topics to cover include:
- Use of grant funds and technical assistance from CMHS, why choices were made, and what the State decided not to do with the funds
- Things that have gone as planned
- Things that have not gone as planned
Unnecessary barriers in the grant program to implementation of transformative practices
Helpful supports in the MHT SIG program; additional ones that would be desirable
Needs Assessment, Resource Inventory and State Comprehensive Mental Health Plan processes
Identify and understand differences among the States in terms of:
- Transformation activities prior to the grant
- Decisions to pursue transformation based on grant funding
- The processes considered central to transformation
- Legislation – proposed and enacted
- Regarding policy changes, implementation and evaluation –Successes and the factors that were instrumental in achieving them. Challenges and the associated barriers.

Transformation Working Group (TWG) Chair

The process used to identify TWG and subcommittee members and how decisions were made about how the TWG and its subcommittees would be established and operate
How the TWG established priorities to work on
Procedures seen as important to facilitating broad involvement and consensus
What worked well or not for the TWG. Topics may include:
- Needs Assessment
- Resource Inventory
- State Comprehensive Mental Health Plan
- Involvement of consumer/family members
- Involvement of other stakeholders

Ways in which the MHT SIG grant has been helpful, not helpful
Biggest successes and why able to achieve
Biggest challenges and how addressing

Mental Health Agency Director

The effect of the grant on the mental health agency. Topics that might be discussed are infrastructure changes, services changes, implementation of EBPs, policy changes, contractual changes, etc. (both actual and planned)
How is the mental health agency coordinating with other agencies as a result of the grant – what is working, what is not
Consumer and family member involvement
Successes and challenges for the mental health agency

Medicaid Agency Director

Actual changes in policy, eligibility, service coverage or payment rates made as a result of the grant to date
Planned changes in eligibility, service coverage or payment rates associated with the grant
Potential changes which were considered but won’t be pursued and why

Education Agency Director

Change in programs/services offered by the public school system based on the grant to date
Future changes that are planned or being considered
How is the education agency coordinating with other agencies as a result of the grant – what is working, what is not
Successes and challenges in incorporating mental health transformation changes in the public school system

Criminal/Juvenile Justice Agency Director

Effect of the grant on programs/services offered by the justice system
Future planned changes
How is the criminal and juvenile justice system coordinating with other agencies as a result of the grant – what is working what is not
Successes and challenges in incorporating mental health transformation changes in the criminal and juvenile justice systems

Director/Senior Staff of Housing & Employment Agencies

Effect of the MHT SIG grant on housing programs (actual and planned changes)
Effect of grant to employment programs (actual and planned changes)
How are the housing and employment agencies coordinating with other agencies serving persons with SMI/SED as a result of the grant – what is working, what is not
Successes and challenges in incorporating mental health transformation changes in the housing and employment systems

The topics to be discussed will vary by State based on available information from existing State documents, reports and Web sites. The evaluators will use all available information to the extent possible to reduce the burden on State agency staff during the site visits.

Site Visit Protocols

The protocols for the in-person site visits are:

Frequency: Twice – grant years three and five

Duration: Two full days on site

Staffing: Four persons will go on each site visit: two members of the cross-site evaluation team and two consumer/family member consultants

Approach: In-person interviews for State staff and focus groups for groups of five consumer and family members

Interviewees: Persons to be interviewed are:

State staff – Project Director, TWG chair, and managers in mental health, criminal/juvenile justice, education, Medicaid, housing and employment agencies
Consumer and family members, including both persons involved in MHT SIG transformation efforts and those who are not, will be asked to participate in a focus group of up to five persons (see Attachment 4).

Burden: A total of two days will be used for in-person site visits, broken down as follows:

State agency staff – Interviews with 8 agencies directors/senior staff ranging from a half hour to 1.5 hours
Focus groups with 15 consumer and family members in groups of five persons lasting up to one and a half hours.

As needed, follow up phone interviews will be used to obtain interviews that cannot be done during the in-person site visit. This includes both State agency staff as well as consumer and family members.

For each State, the burden for State agency interviews is 9 hours (7.5 hours for the interviews plus 1.5 hours to do the scheduling), and 18 hours for both years. For both years for all States, the State agency interview burden will be 162 hours.

The burden for consumer and family members in each State is 22.5 hours (15 persons for 1.5 hours each) each year, and 45 hours for both years. The burden for all nine States for both years is 405 hours.

Proposed Schedule:

Day One

8 – 9:30 AM Interview MHT SIG Project Director

10 – 11:30 AM Interview TWG Chair

12:30 – 2:00 PM First Consumer/Family Member (CFM) Focus Group (up to 5 participants at the Mental Health agency)

2:30 – 4:00 PM Mental Health Agency Director

4:30 – 5:00 PM Medicaid Agency Director

Day Two

8:00 – 9:00 AM Education Agency Director or senior staff

9:15 – 10:45 AM Second CFM Focus Group (up to 5 participants at Education offices)

11:15 AM – Noon Criminal Justice Agency Director/staff

1:15 – 2:30 PM Housing/Employment agency senior staff

3:00 – 4:30 PM Third CFM Focus Group (up to 5 participants at Housing/ Employment agency)

Attachment 3

Sample page included with the permission of the publisher. For additional information, about this questionnaire, one can visit: http://www.mindgarden.com/products/mlqr.htm.

Protocol

Frequency: Twice – May of the third and fifth grant years

Duration: Up to 20 minutes for each interview

Approach: Phone interviews using the Multifactor Leadership Questionnaire instruments (one for leaders and one for raters)

Interviewees: Persons to be interviewed in each State are:

Leaders – MHT SIG Project Director and TWG Chair or Co-chairs (up to 3 persons total)

Raters – TWG Subcommittee Chairs or Consumer/Family Members (up to 15 persons total). The raters will be selected so that at least one-third of the raters will be consumer/family members.

Any and all reports and other documents about the interviews will only provide information for all nine States collectively, and will be written so it will not be possible to identify any individual or any State. Thus, the interviews will be confidential and the findings will only be used for analytical purposes.

Burden: Depending on the number of TWG Chairs and Subcommittees in each State, up to 15 interviews will be done at each time period.

The total burden per State is 5 hours each year (15 persons times 20 minutes per interview) and 10 hours for both years. Total burden for all nine States is 90 hours.

Attachment 4

OMB No. 0930-xxxx

Expiration Date: ______

Adult, Youth, and Family Member (AYF) Screening Questions

Note: In New Mexico, the term “behavioral health” will be used instead of “mental health.”

Please indicate the categories that best describe you.

Current Former

Adult mental health service user (age 18-64)

Older adult mental health service user (age 65+)

Youth mental health service user (age <18)

Family member/caregiver to youth mental health service user

Family member/friend of adult/older adult mental health service user

Service provider for adults

Service provider for youth/children

Representative of private, not-for-profit mental health service user/family

organization

Please identify the organization(s) you represent:

___________________________ ,__________________________

Note: If person is only a service provider, confirm response and note participation is limited to mental health service users and family members and representatives of key stakeholder organizations (if mental health service user or family member representatives are not available). Service providers who are responding are asked to provide responses from perspective of mental health service user or family member.

Are you aware of national, state or local community efforts to transform or reform mental health services? (Check all that apply)

National State Local/community No

Note: Any statewide efforts should be categorized as State. State refers to statewide activities, e.g., Transformation Working Group; local community refers to local(city, community, county) programs and activities even if funded by the State.

Are you aware of the Mental Health Transformation grant awarded to your State?

Are you willing to comment on the involvement of mental health service users and family members about this State grant?

Yes No (person will not be asked to participate)

Adult, Youth and Family Member (AYF) Semi-Structured Focus Group Facilitation/Interview Guide

Note: Participants need to be informed that this survey is designed to collect information only about the State mental health transformation grant and its associated activities, as opposed to other types of transformation activities that might be occurring in the State.

How would you describe the National/ State/ Local (Use the level of efforts that applies to the screening questions) efforts to transform mental health services? How did you first hear about it?

Are you involved in efforts to transform your state or local mental health services?

Yes No (go to question #7)

If yes: How would you describe your roles in transformation efforts? (Options shown below)

State Local

Member of State Transformation Working Group (TWG)

Work group/subcommittee member of TWG

Consultant/advisor to mental health or related program

Mental health program staff or evaluator

Consumer/family participant in meetings about mental health

transformation

Consumer/family participant in needs assessment or State mental

health plan

Consumer/family participant or contributor to grant application

Other (specify) ____________________________

Note: Substitute state or locally specific terms for Transformation Working Group (TWG), subcommittee and State mental health plan as necessary.

In your own words, please describe your involvement in transformation efforts. How long you have been involved?

Do you think State efforts are having an impact on local community efforts? Do you think local community efforts are having an impact on State efforts to transform mental health services? If yes, please provide an example?

Are you a member or staff of an organization run by mental health service users and their families, such as NAMI and the Federation of Families for Children’s Mental Health? (As needed, indicate that the word “run” equals 51% control by consumers/family members)

Yes No (go to question # 6)

If yes, please identify the organization(s). For each organization, ask: whether it is local or state, if it is active in efforts to transform mental health services, and their role in the organization (e.g., member, staff or executive director)?Informal involvement should be described. If only a member, ask: How frequently do you participate in organizational activities?

Organization Name: _______________ _______________ ________________

State or local: _______________ _______________ ________________

Active in transformation (Y/N): _______ _______________ ________________

Your role: _______________ _______________ ________________

Frequency of participation: ____________ _______________ ________________

For each organization: Are you involved with this organization to help transform mental health services? If yes, please describe how.

For participants who are members of AYF organizations involved in transformation efforts AND for participants who are representatives of key stakeholder organizations: Please describe how your organization works with local and State agencies and other groups to transform mental health services.

For participants who identify as adults or youth who receive mental health services and/or family members: Are you personally involved to promote mental health transformation? Are you involved with a mental health organization that advocates for mental health transformation? For participants indicating their involvement is related to State or local transformation efforts: What are you personally trying to accomplish by being involved in mental health transformation efforts? What is your organization trying to accomplish?

(This question needs to be handled in two parts – first ask the questions about frequency, then the ones about involvement. Questions need to be specific to the level of involvement of respondent(s), i.e., local, State or both) Thinking of both mental health service users and their families, how often are they involved in the following activities? How would you describe their type of involvement?

Participation in mental health system:	Frequency of Involvement						Type of Involvement
Participation in mental health system:	Never	Rarely	Occasion-ally	Fre-quently	Almost always	Don’t know	No Role	Minor Role	Major Role	Don’t Know
Setting local goals
Making local policy
Designing local programs
Implementing local programs
Evaluating local programs
Decisions on funding of local programs

Setting State goals
Making State policy
Designing State programs
Implementing State programs
Evaluating State programs
Decisions on funding of State programs

Do you have any clarifications or comments: ___________________________________________

________________________________________________________________________________

6. Is there adequate involvement by each of the following groups? (Answers are yes or no; capture any comments).

By diverse cultural, ethnic and religious groups:

By diverse age groups:

By persons from cities, suburbs and rural/frontier areas:

Comments: _________________________________________________________________

____________________________________________________________________________

If answered no to a question above, what should be happening that is not? ____________________________________________________________________________

____________________________________________________________________________

7. Please indicate the frequency of involvement of both mental health service users and their families (AYF) in the following activities due to the mental health transformation grant.

Participation in particular activities:	Frequency of Involvement
Participation in particular activities:	Never	Rarely	Occasion-ally	Fre- quently	Almost always	Don’t know
AYF are involved in all preliminary discussions about transformation grant efforts.
AYF can speak on their experiences during grant meetings.
AYF initiate grant meetings.
AYF set agendas for grant meetings.
AYF make assignments to others to follow through on for grant activities.
AYF hold trainings for others on the grant project.
When policy research is done for this grant, AYF have a leading role.
AYF inform the public about transformation grant efforts.

8. Has the grant lead to more involvement by mental health service users and their families? If yes, what difference has that involvement made? (Probe for differences at both local and State levels). Please examples of differences this participation has made.

Are mental health service users and their families more hopeful about the future of the mental health system as a result of the grant?

What are the major barriers getting in the way of making changes consist with grant activities at the State and local levels? Do you see any patterns of problems? (Examples are failure to involve individuals receiving mental health services and family members at all, failure to listen to individuals receiving mental health services and family members, and failure to consider a diverse set of individuals receiving mental health services and family member perspectives) What should have happened as a result of the grant, but did not? During the grant, were there ways mental health service users and families should have been involved but were not?

What are the most important things that you would most like to see change as part of mental health transformation? Are the needs of mental health service users and families being addressed by this grant?

Adult, youth and family member involvement questionnaire

Interviewer instructions: Identify the perspective from which participant will be responding (if an individual falls into more than one category, s/he should reply based on the perspective most important to them). Ask each question in reference to one state and/or local transformation project, according to participant’s involvement/familiarity. Items containing “A/Y/FY/FA” should be posed specifically for the perspective(s) for which the participant is responding. For example, if the participant identifies as a family member of a youth mental health service user, read item 2 as “There are enough family members of youth involved in grant activities.”

adult mental health service user (A)

youth mental health service user (Y)

Family member/caregiver

of youth mental health service user (FY)

Family member of adult mental health service user (FA)

Thinking about the mental health transformation grant activities that you’re familiar with, please indicate how strongly you agree or disagree with each of the following statements:	Response
	Strongly Agree	Agree	Neither	Disagree	Strongly Disagree	Don’t Know
1. A/Y/FY/FA have meaningful participation in:
Setting local goals
Making local policy
Designing local programs
Implementing local programs
Evaluating local programs
Decisions on funding of local programs

Setting State goals
Making State policy
Designing State programs
Implementing State programs
Evaluating State programs
Decisions on funding of State programs
2. There are enough A/Y/FY/FA involved in the mental health grant.
3. Mental health service users and their family members are positively affected by this grant.
4. A/Y/FY/FA are adequately compensated for their roles.
5. Involvement of A/Y/FY/FA has made a difference.
6. Grant staff are able to work collaboratively with A/Y/FY/FA.
7. The grant promotes collaboration among A/Y/FY/FA.
8. State leaders are sensitive to cultural and linguistic issues.
9. A/Y/FY/FA receive the training and support they need to participate effectively in the mental health transformation grant.
10. The grant promotes A/Y/FY/FA understanding of the process behind developing policy.
11. The grant promotes A/Y/FY/FA understanding of current policy issues.
12. The leaders of this grant make involvement by A/Y/FY/FA a priority.
13. During project meetings, the opinions of A/Y/FY/FA are discussed.
14. Action is taken as a result of A/Y/FY/FA opinions.
15. A partnership exists between A/Y/FY/FA and persons who are not consumers/family members.
16. As a result of the grant, A/Y/FY/FA have the knowledge to educate the community on important issues.
17. The grant promotes A/Y/FY/FA to take the lead in this transformation project.
18. Disagreements are handled respectfully within this project.
19. Information about the mental health transformation grant is readily available to A/Y/FY/FA.
20. Professionals use language that is easily understood by non-professional participants.
21. Efforts are made to evaluate A/Y/FY/FA involvement.
22. A/Y/FY/FA are excited about the progress of the grant.
23. Persons of all cultural and ethnic origins are respected within this grant.
24. Stigma/discrimination is not accepted at any level of this grant.

Comments and clarification: ____________________________________________________________________________________

____________________________________________________________________________________________________________

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0930-xxxx. Public reporting burden for this collection is estimated to average xx hours per client year, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of data. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, MD 20857.

Attachment 5

MHT SIG GPRA Infrastructure Indicator Definitions

GPRA Indicator 1: Increase percent of policy changes completed as a consequence of the CMHP

Percent refers to the percent of changes targeted over the life of the grant that have actually been completed at each annual measurement point.
A policy is operationalized by a written document directing some action or event. This document can be administrative or legislative in origin. Policies include formal, written documents identified as any of the following: directives, guidances, clinical practice guidelines, regulations, statutes, operations manuals, procedures, bylaws, strategic plans, mission statements, written decisions, standards, or similar documents. Financing policy changes are not included for this Indicator but, instead, should be included under GPRA Indicator 3. Policies must be significant at the State level. As such, they relate to at least one of the following: State agencies, bureaus, or departments; State legislatures; or statewide private organizations (e.g., statewide managed care association, licensing boards). To the extent that States are decentralized, targets may be set in terms of the proportion of counties, cities, or tribal agencies/bureaus/departments, or county, city, or tribal legislatures making policy changes of a particular nature (e.g., “14 of the 45 counties will adopt the State’s clinical practice guidelines”).
A change can be the creation of something that did not previously exist, the documentation of something that previously existed in an undocumented form, or the elimination or alteration of something that previously existed and had already been documented.
A policy change is completed when it exists in its final form and has been approved or passed by the party or parties with authority to do so.
A policy change is a consequence of the CMHP if it is mentioned or implied in the CMHP and, by extension, is consistent with the New Freedom Commission (NFC) goals. Note: Although all policy changes must be consistent with the CMHP, completion of the CMHP itself should not be included as a policy change for GPRA reporting purposes.
Baseline is 0.

GPRA Indicator 2: Increase number of persons in the mental health care and related workforce who have been trained in service improvements recommended by the CMHP

The mental health care workforce is composed of people who provide mental health prevention, treatment, rehabilitation, or recovery services. The related workforce is composed of people who provide ancillary support services to people who have mental health needs or are at risk for developing mental health needs. For example, employment service providers, primary care providers, school personnel, child welfare staff, peer support program staff, supported housing staff, criminal or juvenile justice personnel, and others who do not provide mental health services but do provide other services to persons with mental health needs are all members of the related workforce. Some people may be considered members of either workforce. Members of the mental health care or related workforce may or may not be self-identified consumers or family members who are providing services. Additionally, State, county, city, tribal, and organizational leaders and administrators of mental health care and related services may be considered members of the mental health care and related workforce.
Workforce members are considered to have been trained when they have engaged in a process guided by a curriculum (e.g., a syllabus, agenda, training manual, or other documents describing the content and format of the information to be covered), taking place within a structured timeframe (i.e. a specific amount of time set aside for the training within some window of time), guided by an identified trainer or training method (e.g., a specific computer-based program). States will be expected to submit a brief explanation of how the training is anticipated to impact one or more of the following: provider awareness, knowledge, attitude, skills, or behaviors; service model fidelity; or mental health service recipient satisfaction or outcomes.
Service improvements include new services as well as improvements to existing ways of providing services.
A service improvement is recommended by the CMHP if it is mentioned or implied in the CMHP and, by extension, is consistent with the NFC goals.
Baseline is 0.
Targets should be broken down according to how many workers within which sector of the workforce will receive training regarding which targeted service improvements. For example, “We will train 250 One-Stop Career Center workers on delivering the supported employment model; 100 State prison guards on the jail diversion model; and 300 CMHC psychiatrists on systematic medication management.”
Initial reporting should include an estimate of the total number of people in each of the targeted workforce sectors.

GPRA Indicator 3: Increase percent of financing policy changes completed as a consequence of the CMHP

Percent refers to the percent of changes targeted over the life of the grant that have actually been completed at each annual measurement point.
A financing policy is defined as a written document directing one or more of the following: substantial increases or decreases in appropriations for specific types of services or activities; changes in billing codes or reimbursement procedures to allow, eliminate or simplify billing for specific types of services or activities; changes to the State Medicaid Plan; innovative pooling or braiding of funding; or other changes regarding financing of specific types of services or activities or that increase efficiency. Financing policies must be significant at the State level. As such, they relate to at least one of the following: State agencies/bureaus/departments; State legislatures; or statewide private organizations (e.g., statewide managed care association, licensing boards). To the extent that States are decentralized, targets may be set in terms of the proportion of counties, cities, or tribal agencies/bureaus/departments, or county, city, or tribal legislatures making financing policy changes of a particular nature (e.g., “30 of the 40 counties will offer contracted providers financial incentives for use of evidence-based practices”).

A change can be the creation of something that did not previously exist; the documentation of something that previously existed in an undocumented form; or the elimination or alteration of something that previously existed and had already been documented.
A financing policy change is completed when it exists in its final form and has been approved or passed by the party or parties with authority to do so.
A financing policy change is a consequence of the CMHP if it is mentioned or implied in the CMHP and, by extension, is consistent with the NFC goals.
Baseline is 0.

GPRA Indicator 4: Increase percent of organizational changes completed as a consequence of the CMHP

Percentage refers to the percent of changes targeted over the life of the grant that have actually been completed at each annual measurement point.
Organizational changes must be significant at the State level.
Organizations may include State agencies, bureaus, departments, or other major subdivisions; counties, cities, or tribal agencies/bureaus/departments; or agencies providing mental health or related services to people who have or are at risk for developing mental health needs. The latter includes consumer-, youth-, or family member run organizations; private provider entities; and non-governmental organizations. To the extent that States are decentralized, targets may be set in terms of the number of counties, cities, tribal agencies/bureaus/departments, or service provider agencies making organizational changes of a particular nature (e.g., “25 of the 40 counties will hire full-time cultural competence coordinators,” or “primary care settings that serve 40% of the State’s residents will include on-site mental health specialists within their offices”).
An organizational change is something that is created, eliminated, or altered within or between organizations. Organizational changes include the following: formal, written inter- or intra-organizational agreements; creation, expansion, integration, or elimination of offices, divisions, or departments; creation or elimination of one or more position(s); creation of a new reporting structure; permanent changes to major responsibilities for existing offices, divisions, and departments; permanent changes in staff composition (e.g., substantial hiring of consumers/youth/family members, substantial increases in racial/ethnic/cultural diversity of staff); or other changes of similar import.
An organizational change is completed when it has been approved, passed, or implemented by the party or parties with authority to do so.
An organizational change is a consequence of the CMHP if it is mentioned or implied in the CMHP and, by extension, is consistent with the NFC goals.
Baseline is 0.

GPRA 5: Increase the number of organizations that regularly obtain and analyze data relevant to the goals of the CMHP

Organizations may include State agencies, bureaus, departments, or other major subdivisions; counties, cities, or tribal agencies/bureaus/departments; or agencies providing mental health or related services to people who have or are at risk for developing mental health needs. The latter includes consumer-, youth-, or family member run organizations; private provider entities; and non-governmental organizations.
Regularly is defined as occurring on a scheduled, repeated, and ongoing basis.
Organizations obtain data if they receive and/or collect data.
Analyze is defined as being systematically reviewed to facilitate program, organization, or state agency/department planning; to facilitate consumer choice or shared decision-making; or to improve the quality or efficiency of services.
Data are any quantitative or qualitative information collected through specified methods and procedures.
Data are relevant to the goals of the CMHP if they may be used to measure events that pertain to the goals of the CMHP and, by extension, pertain to the NFC goals.
Baseline is 0.
Initial reporting should include an estimate of the total number of organizations within the targeted group.

GPRA Indicator 6: Increase the number of consumers and family members that are members of Statewide consumer- and family-run networks

Consumers are defined as adults, older adults, children, or youth who currently receive mental health services, have received mental health services in the past, or are eligible to receive mental health services but choose not to. It is understood and respected that many people who meet one or more of these criteria may choose to identify with a term other than “consumer.”
Family members may be members of an adult or child/youth consumer’s immediate or extended family. Additionally, members of consumers’ extended family networks or “adopted” family members (e.g., familismo in Hispanic culture) are considered family members. Family members may also be friends, co-workers, or neighbors of an adult or child/youth consumer, or non-family caregivers of a child/youth consumer.
Members of a network are those individuals officially registered with the organization as members. If the network does not have official registration, the Indicator should instead reflect the number of individuals that the organization recognizes as members.
Statewide networks are those involved in concrete and measurable ways to network, connect, and interact with mental health consumers and/or family members throughout the State or across a significant region of the State. Statewide networks may include statewide coalitions of local or regional networks. Networks have systems for communication to flow both to and from mental health consumers and/or family members: a) the organization is able to gain input, ideas, and direction from consumers and/or family members, and b) the organization has ways of regular communication with consumers and/or family members throughout the State.
A network or organization is considered to be a consumer and/or family-run network if it is controlled and managed by mental health consumers and/or family members and if over half of the members of the board of directors identify as mental health consumers and/or family members. It is anticipated that most states will have at least three eligible networks: a Statewide consumer-run organization (possibly a member of the National Coalition of Mental Health Consumer/Survivor Organizations or NCMHCSO), a statewide affiliate of the National Alliance on Mental Illness (NAMI), and a statewide affiliate of the National Federation of Families for Children’s Mental Health (NFFCMH). Some States may have additional eligible organizations, such as specific networks for older adult consumers or young people.
Baseline is the existing membership of the targeted networks.

GPRA Indicator 7: Increase the number of programs implementing practices consistent with the CMHP

A program is a level of organization that unifies a group of direct care staff in delivering a specific service or implementing a specific practice. In the case of larger programs that have subprograms (e.g., housing programs consisting of multiple supported housing site offices/operating units), the subprograms, rather than the larger program, should be counted.
Programs are implementing practices when the practices are being actively delivered to individuals (e.g., consumers, family members, people at risk).
Practices include treatment, rehabilitation, prevention, and supportive services (e.g., evidence-based practices, consumer-operated services, culturally-specific practices, suicide prevention programs, rural telehealth programs, etc.).
Practices are consistent with the CMHP if they are mentioned or implied in the CMHP and, by extension, are consistent with the NFC goals.
Baseline is 0.
Targets should be broken down according to how many programs will implement each of the targeted practices. For example, “We will implement 20 new multi-systemic therapy programs; 15 new local anti-stigma campaigns; and 100 new school-based mental health promotion programs.”

Attachment 6

GPRA Infrastructure Indicators Data Collection Instructions and Forms

Data Collection Instructions

Specific instructions for completion of Form I (measure add/edit/change) for each GPRA follow. Note that the activities referenced in this form vary from GPRA to GPRA. Directions for specific items within this form also vary somewhat. A copy of Form I and a copy of Form II (measure completion/deletion) follow the instructions. The only Form II item that varies according to GPRA is the impact item. When combined with the information from Form I, the information collected by this item will allow for calculation of the overall GPRA Indicators (exact formulas for calculation will be determined after further discussion with the MHT SIG States).

Activities and Impact in GPRA Forms

GPRA Indicators	Activity referenced in Form I	Impact measured in Form II
1. Increase percent of policy changes completed as a consequence of CMHP	Policy change	Metric of choice
2. Increase number of persons in the workforce who have been trained in service improvements recommended by CMHP	Training	Number of unduplicated people trained as a result of this training
3. Increase percent of financing policy changes completed as a consequence of the CMHP	Financing policy change	Metric of choice
4. Increase percent of organizational changes completed as a consequence of the CMHP	Organizational change	Metric of choice
5. Increase the number of organizations that regularly obtain and analyze data relevant to the goals of the CMHP	Intervention	Number of organizations obtaining & analyzing data as a result of intervention
6. Increase the number of consumers and family members that are members of the statewide consumer- and family-run networks.	Intervention	Number of consumers/ family members in statewide organization as result of intervention
7. Increase the number of programs that are implementing practices consistent with the CMHP	Intervention	Number of programs using practices consistent with CMHP as result of intervention

Form I: Instructions for GPRA Indicator 1

Percent of policy changes completed as a consequence of the Comprehensive Mental Health Plan

Please identify this entry as one of the following: target activity add, change, or edit. Check add the first time an activity is entered, or for a completely new target activity. An entry reflecting a new development in the content or intent of the policy is considered a target activity change; an entry reflecting a correction to an earlier target activity addition or change form is considered an edit. Complete the form in entirety for target activity add. For change and edit entries, complete entry number and then only those items that have changed or need to be edited.

Please indicate the date of this entry.

Please identify the GPRA Indicator(s) linked to this entry. If this entry is not linked to any GPRA Indicators, please check the “non-GPRA” box.

1. Entry Number: Please fill-in entry number for identified policy change.

2. Working Title: Please give the policy a brief name that reflects its content. It is understood that the title may later be changed to reflect developments in this policy and the work towards its completion.

3. Relevant NFC goal(s) and appropriate recommendation(s), if specified: Please check at least one of the boxes. If the entry is related to one or more of the NFC goal(s) and/or recommendation(s), please check the corresponding box(es). If the entry is related to a goal not specifically identified by the NFC, please check the “non-NFC” box and briefly state the goal under “additional comments.”

4. Primary anticipated long-term impact on SAMHSA strategic goals: Please check at least one of the boxes, indicating which component of SAMHSA’s strategic goals the policy change is expected to have an impact on. Please check “improved accountability” if the policy change is related to the measuring and reporting of performance, i.e. the tracking of trends, measurement and reporting systems, or management practices. Please check “increased service capacity” if the policy is related to service availability, i.e. needs assessment, planning, systems improvements, outreach, assessment, referral, service expansion, or consumer choice Please check “increased service effectiveness” if the policy is related to improving service quality, i.e. outcome measurement, service improvements, dissemination of EBPs. If a target activity concerns more than one of SAMHSA’s strategic goals, please check all boxes that might apply for that activity. If the target activity doesn’t address any of SAMHSA’s strategic goals, please check “other” and describe the anticipated long-term impact of that activity.

5. Anticipated ease of completion: Please check one box indicating the anticipated ease of completion for this target activity. A “stretch” activity is defined as an activity you’d really like to accomplish because it would have a big impact on transformation but that may be a high-risk for not completing within the grant period. An “average/realistic” activity is defined as an activity that you think you can realistically accomplish in the timeframe given. A “sure bet” activity is defined as an easy activity that is meaningful and has a high likelihood of completion within the grant period. The anticipated ease of completion must be completed for each target activity.

6. Origin: Please complete sections a, b, c and d, checking either the “yes” or “no” box for each sub-item. If the response to a is “yes,” please indicate the party or parties that initiated the activity prior to the MHT SIG grant. If the response to a is “no,” please indicate whether the activity was initiated by the TWG. Note that responses to b, c and d are in no way dependent upon each other (e.g., in some cases the CMHP may describe activities not related to issues raised in the NARI, the TWG may recommend activities not described in the CMHP).

7. Groups and agencies involved

a. Lead agent: Please identify the group (e.g., TWG), agency (e.g., SMHA), or other body leading the effort to complete this policy. Please list all agents, if more than one lead agent.

b. Agency/agencies participating: Please identify the state/tribe/district/territory agency or agencies participating in this policy change. Please feel free to use the “other” boxes to indicate agencies that are not adequately represented by the options offered.

8. Population(s) affected: Please check as many boxes as apply. If a population is not represented in a box, please check “other” and write in population beside “other” box. Please use “additional comments” line if more space is needed, or to identify additional populations.

9. Referent: Please describe both the nature (e.g., written agreement, administrative regulation, law, etc.) and the intent (e.g., ensure that children/youth receive trauma-informed care; increase the number of housing slots for persons who are mentally ill and homeless, etc.) of the policy or policy change.

10. Standard for completion: Please indicate what criterion or criteria you will use to determine if and when the policy change is completed (e.g., drafted, publicly reviewed, final draft written, approved or passed, issued or implemented as measured by…, etc.)

11. Anticipated Time Frame: Please identify either the month and year or the quarter and year of the intended start and completion times for this policy.

12. Target populations to be trained/Number in need: N/A for this Indicator.

Other characteristics of item: Please identify any other characteristics of item which you feel are important and have not been previously identified on this form.

Additional comments: Please give us any additional comments you may have about the activity identified in the form, or any issues that arose in completing the form.

Form I: Instructions for GPRA Indicator 2

Number of persons in the mental health care and related workforce who have been trained in service improvements recommended by the Comprehensive Mental Health Plan

Please identify this entry as one of the following: target activity add, change, or edit. Check add the first time an activity is entered, or for a completely new target activity. An entry reflecting a new development in the content or intent of the training is considered a target activity change; an entry reflecting a correction to an earlier target activity addition or change form is considered an edit. Complete the form in entirety for target activity add. For change and edit entries, complete entry number and then only those items that have changed or need to be edited.

Please indicate the date of this entry.

Please identify the GPRA Indicator(s) linked to this entry. If this entry is not linked to any GPRA Indicators, please check the “non-GPRA” box.

1. Entry Number: Please fill-in entry number for identified training.

2. Working Title: Please give the training a brief name that describes the type and focus of the training. It is understood that the title may later be changed to reflect developments in this training and the work towards its completion.

4. Primary anticipated long-term impact on SAMHSA strategic goals: Please check at least one of the boxes, indicating which component of SAMHSA’s strategic goals the training is expected to have an impact on. Please check “improved accountability” if the training is related to the measuring and reporting of performance, i.e. the tracking of trends, measurement and reporting systems, or management practices. Please check “increased service capacity” if the training is related to service availability, i.e. needs assessment, planning, systems improvements, outreach, assessment, referral, service expansion, or consumer choice Please check “increased service effectiveness” if the training is related to improving service quality, i.e. outcome measurement, service improvements, dissemination of EBPs. If a target activity concerns more than one of SAMHSA’s strategic goals, please check all boxes that might apply for that activity. If the target activity doesn’t address any of SAMHSA’s strategic goals, please check “other” and describe the anticipated long-term impact of that activity.

7. Groups and agencies involved

a. Lead agent: Please identify the group (e.g., TWG), agency (e.g., SMHA), or other body leading the effort to complete the training. Please list all agents, if more than one lead agent.

b. Agency/agencies participating: Please identify the state/tribe/district/territory agency or agencies participating in this training. Please feel free to use the “other” boxes to indicate agencies that are not adequately represented by the options offered.

9. Referent: Please describe the nature of the training (i.e., is it a workshop, course, practicum, internship; is it one hour, one day, several days, a semester; is it in-person, on-line, blended, etc.); the topic of the training; and any other aspects of the training that you think might be important (whether the training results in CEUs, certification, etc).

10. Standard for completion: Please indicate what criterion or criteria you will use to determine whether persons have been trained (i.e., sufficient attendance; recognition that persons met training requirements; award of CEUs, certification, passing of test, etc.)

11. Anticipated Time Frame: Please identify either the month and year or the quarter and year of the intended start and completion times for this training.

12. Target populations to be trained/Number in need: Please estimate the number of persons for whom this training is intended.

Other characteristics of item: Please identify any other characteristics of item which you feel are important and have not been previously identified on this form.

Additional comments: Please give us any additional comments you may have about the activity identified in the form, or any issues that arose in completing the form.

Form I: Instructions for GPRA Indicator 3

Percent of financing policy changes completed as a consequence of the Comprehensive Mental Health Plan

Please indicate the date of this entry.

Please identify the GPRA Indicator(s) linked to this entry. If this entry is not linked to any GPRA Indicators, please check the “non-GPRA” box.

1. Entry Number: Please fill in entry number for identified financing policy change.

2. Working Title: Please give the financing policy a brief name that reflects its content. It is understood that the title may later be changed to reflect developments in this policy and the work towards its completion.

4. Primary anticipated long-term impact on SAMHSA strategic goals: Please check at least one of the boxes, indicating which component of SAMHSA’s strategic goals the financing policy change is expected to have an impact on. Please check “improved accountability” if the financing policy change is related to the measuring and reporting of performance, i.e. the tracking of trends, measurement and reporting systems, or management practices. Please check “increased service capacity” if the financing policy is related to service availability, i.e. needs assessment, planning, systems improvements, outreach, assessment, referral, service expansion, or consumer choice Please check “increased service effectiveness” if the financing policy is related to improving service quality, i.e. outcome measurement, service improvements, dissemination of EBPs. If a target activity concerns more than one of SAMHSA’s strategic goals, please check all boxes that might apply for that activity. If the target activity doesn’t address any of SAMHSA’s strategic goals, please check “other” and describe the anticipated long-term impact of that activity.

7. Groups and agencies involved

a. Lead agent: Please identify the group (e.g., TWG), agency (e.g., SMHA), or other body leading the effort to complete the policy. Please list all agents, if more than one lead agent.

9. Referent: Please describe the nature (i.e., is it a written agreement, administrative regulation, law, etc.) and intent (e.g., funding to ensure that children/youth receive trauma informed care; funding to increase the number of housing slots for persons who are mentally ill and homeless, etc.) of the financing policy or policy change

10. Standard for completion: Please indicate what criterion or criteria you will use to determine if and when the funding policy change is completed (e.g., drafted, publicly reviewed, final draft written, approved or passed, issued or implemented as measured by ….).

11. Anticipated Time Frame: Please identify either the month and year or the quarter and year of the intended start and completion times for this policy.

12. Target populations to be trained/Number in need: N/A for this Indicator

Other characteristics of item: Please identify any other characteristics of item which you feel are important and have not been previously identified on this form.

Additional comments: Please give us any additional comments you may have about the activity identified in the form, or any issues that arose in completing the form.

Form I: Instructions for GPRA Indicator 4

Percent of organizational changes completed as a consequence of the Comprehensive Mental Health Plan (includes interagency agreements)

Please identify this entry as one of the following: target activity add, change, or edit. Check add the first time an activity is entered, or for a completely new target activity. An entry reflecting a new development in the content or intent of the organizational change is considered a target activity change; an entry reflecting a correction to an earlier target activity addition or change form is considered an edit. Complete the form in entirety for target activity add. For change and edit entries, complete entry number and then only those items that have changed or need to be edited.

Please indicate the date of this entry.

Please identify the GPRA Indicator(s) linked to this entry. If this entry is not linked to any GPRA Indicators, please check the “non-GPRA” box.

1. Entry Number: Please fill-in entry number for identified organizational change.

2. Working Title: Please give the organizational change a brief name that reflects its content. It is understood that the title may later be changed to reflect developments in this change and the work towards its completion.

4. Primary anticipated long-term impact on SAMHSA strategic goals: Please check at least one of the boxes, indicating which component of SAMHSA’s strategic goals the organizational change is expected to have an impact on. Please check “improved accountability” if the organizational change is related to the measuring and reporting of performance, i.e. the tracking of trends, measurement and reporting systems, or management practices. Please check “increased service capacity” if the organizational change is related to service availability, i.e. needs assessment, planning, systems improvements, outreach, assessment, referral, service expansion, or consumer choice Please check “increased service effectiveness” if the organizational change is related to improving service quality, i.e. outcome measurement, service improvements, dissemination of EBPs. If a target activity concerns more than one of SAMHSA’s strategic goals, please check all boxes that might apply for that activity. If the target activity doesn’t address any of SAMHSA’s strategic goals, please check “other” and describe the anticipated long-term impact of that activity.

7. Groups and agencies involved

a. Lead agent: Please identify the group (e.g., TWG), agency (e.g., SMHA), or other body leading the effort to complete this change. Please list all agents, if more than one lead agent.

b. Agency/agencies participating: Please identify the state/tribe/district/territory agency or agencies participating in this organizational change. Please feel free to use the “other” boxes to indicate agencies that are not adequately represented by the options offered.

9. Referent: Please describe the nature of the organizational change (i.e., what organizations-[e.g., SMHA, child welfare] are to change at what levels [state, county, local] in what ways [memorandum of agreement to cooperate; move to umbrella agency; merger] ).

10. Standard for completion: Please indicate what criterion or criteria you will use to determine if and when the organizational change is completed (e.g., signed agreement; merger completed etc.)

11. Anticipated Time Frame: Please identify either the month and year or the quarter and year of the intended start and completion times for this organizational change.

12. Target populations to be trained/Number in need: N/A for this Indicator.

Other characteristics of item: Please identify any other characteristics of item which you feel are important and have not been previously identified on this form.

Additional comments: Please give us any additional comments you may have about the activity identified in the form, or any issues that arose in completing the form.

Form I: Instructions for GPRA Indicator 5

The number of organizations that regularly obtain and analyze data relevant to the goals of the Comprehensive Mental Health Plan

Please identify this entry as one of the following: target activity add, change, or edit. Check add the first time an activity is entered, or for a completely new target activity. An entry reflecting a new development in the content or intent of the target activity is considered a target activity change; an entry reflecting a correction to an earlier target activity addition or change form is considered an edit. Complete the form in entirety for target activity add. For change and edit entries, complete entry number and then only those items that have changed or need to be edited.

Please indicate the date of this entry.

Please identify the GPRA Indicator(s) linked to this entry. If this entry is not linked to any GPRA Indicators, please check the “non-GPRA” box.

1. Entry Number: Please fill-in entry number for the identified target activity.

2. Working Title: Please give the target activity a brief name that describes its content. It is understood that the title may later be changed to reflect developments in this activity and the work towards its completion.

4. Primary anticipated long-term impact on SAMHSA strategic goals: Please check at least one of the boxes, indicating which component of SAMHSA’s strategic goals the target activity is expected to have an impact on. Please check “improved accountability” if the target activity is related to the measuring and reporting of performance, i.e. the tracking of trends, measurement and reporting systems, or management practices. Please check “increased service capacity” if the target activity is related to service availability, i.e. needs assessment, planning, systems improvements, outreach, assessment, referral, service expansion, or consumer choice Please check “increased service effectiveness” if the target activity is related to improving service quality, i.e. outcome measurement, service improvements, dissemination of EBPs. If a target activity concerns more than one of SAMHSA’s strategic goals, please check all boxes that might apply for that activity. If the target activity doesn’t address any of SAMHSA’s strategic goals, please check “other” and describe the anticipated long-term impact of that activity.

7. Groups and agencies involved

a. Lead agent: Please identify the group (e.g., TWG), agency (e.g., SMHA), or other body leading the effort to complete this target activity. Please list all agents, if more than one lead agent.

b. Agency/agencies participating: Please identify the state/tribe/district/territory agency or agencies participating in this activity. Please feel free to use the “other” boxes to indicate agencies that are not adequately represented by the options offered.

9. Referent: Please describe the nature of the target activity to increase the number of organizations that regularly obtain and analyze data (i.e., is it an IT activity; a training activity; a quality improvement process; a policy; or legislative change. These can overlap with policies or policy changes entered for GPRA Indicator 1).

10. Standard for completion: Please indicate what criterion or criteria you will use to determine if and when an organization has increased the extent to which it regularly obtains and analyzes data (e.g., expanded data collection; entry into a data-sharing agreement; completion of training in some aspect of data collection or data analysis; or data use)

11. Anticipated Time Frame: Please identify either the month and year or the quarter and year of the intended start and completion times for this target activity.

12. Target populations to be trained/Number in need: N/A for this Indicator.

Other characteristics of item: Please identify any other characteristics of item which you feel are important and have not been previously identified on this form.

Additional comments: Please give us any additional comments you may have about the activity identified in the form, or any issues that arose in completing the form.

Form I: Instructions for GPRA Indicator 6

The number of consumers and family members who are members of Statewide consumer- and family- run networks.

Please identify this entry as one of the following: target activity add, change, or edit. Check add the first time an activity is entered, or for a completely new target activity. An entry reflecting a new development in the content or intent of the target activity is considered a target activity change; an entry reflecting a correction to an earlier target activity addition or change form is considered an edit. Complete the form in entirety for target activity add. For change and edit entries, complete entry number and then only those items that have changed or need to be edited.

Please indicate the date of this entry.

Please identify the GPRA Indicator(s) linked to this entry. If this entry is not linked to any GPRA Indicators, please check the “non-GPRA” box.

1. Entry Number: Please fill-in entry number for the identified target activity.

2. Working Title: Please give the target activity a brief name that reflects its content. It is understood that the title may later be changed to reflect developments in this activity and the work towards its completion.

7. Groups and agencies involved

a. Lead agent: Please identify the group (e.g., TWG), agency (e.g. SMHA), or other body leading this activity. Please list all agents, if more than one lead agent.

b. Agency/agencies participating: Please identify the state/tribe/district/territory agency or agencies participating in this target activity. Please feel free to use the “other” boxes to indicate agencies that are not adequately represented by the options offered.

9. Referent: Please describe the nature of the target activity (e.g., helping found or sustain CFM operated networks; publicizing CFM operated networks; policies that consumers and family members should be referred to CFM operated networks [could overlap with policies listed for GPRA 1]).

10. Standard for completion: Please indicate what criterion or criteria you will use to determine whether a person is a member of a CFM operated network.

11. Anticipated Time Frame: Please identify either the month and year or the quarter and year of the intended start and completion times for this target activity.

12. Target populations to be trained/Number in need: Please estimate number of consumers and family members expected to benefit from being members of CFM operated networks.

Other characteristics of item: Please identify any other characteristics of item which you feel are important and have not been previously identified on this form.

Additional comments: Please give us any additional comments you may have about the activity identified in the form, or any issues that arose in completing the form.

Form I: Instructions for GPRA Indicator 7

The number of programs that are implementing practices consistent with the Comprehensive Mental Health Plan

Please identify this entry as one of the following: target activity add, change, or edit. Check add the first time an activity is entered, or for a completely new target activity. An entry reflecting a new development in the content or intent of the target activity is considered a target activity change; an entry reflecting a correction to an earlier target activity addition or change form is considered an edit. Complete the form in entirety for target activity add. For change and edit entries, complete entry number and then only those items that have changed or need to be edited.

Please indicate the date of this entry.

Please identify the GPRA Indicator(s) linked to this entry. If this entry is not linked to any GPRA Indicators, please check the “non-GPRA” box.

1. Entry Number: Please fill-in entry number for the identified target activity.

7. Groups and agencies involved

a. Lead agent: Please identify the group (e.g., TWG), agency (e.g., SMHA), or other body leading the activity. Please list all agents, if more than one lead agent.

9. Referent: Please describe the nature of the target activity to increase numbers of programs implemented consistent with CMHP (e.g., policy, financing policy, plan, training, [could overlap with target activities listed for GPRA 1, 2,3]).

10. Standard for completion: Please indicate what criterion or criteria you will use to determine when a program consistent with the CMHP has been implemented.

11. Anticipated Time Frame: Please identify either the month and year or the quarter and year of the intended start and completion times for this target activity.

12. Target populations to be trained/Number in need: Please estimate the number of programs consistent with CMHP needed.

Other characteristics of item: Please identify any other characteristics of item which you feel are important and have not been previously identified on this form.

Additional comments: Please give us any additional comments you may have about the activity identified in the form, or any issues that arose in completing the form.

OMB No. 0930-xxxx

Expiration Date: _____

GPRA Form I

Target Activity: Add  Change  Edit 

Date Form Completed: _________

Linked to GPRA  Indicator(s) #: 1 2 3 4 5 6 7

1. Entry Number: _____________________ (Assigned by HSRI or by states, TBD)

2. Working Title: _______________________________________________________

Relevant NFC goal or goals (and appropriate recommendation(s), if specified):

Goal 1 	Goal 2 	Goal 3 	Goal 4 	Goal 5 	Goal 6 	Non-NFC 
1.1 	2.1 	3.1 	4.1 	5.1 	6.1 	(Please specify below)
1.2 	2.2 	3.2 	4.2 	5.2 	6.2 
	2.3 		4.3 	5.3 
	2.4 		4.4 	5.4 
	2.5 

Additional comments: ____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

4. Primary anticipated long-term impact on SAMHSA strategic goals:

 Improved accountability

 Increased service capacity

 Increased service effectiveness

 Other: please specify: _________________________________________________

5. Anticipated ease of completion:

 Stretch goal—high risk of non-completion

 Sure bet—high likelihood of completion

 Average/realistic—neither high risk nor sure bet

6. Origin

a. Did activity formally begin before MHT SIG? Yes  No 

(If “yes”) Please indicate the major initiator ___________________________

(If “no”) Was this activity begun by the TWG? Yes  No 

b. Raised in NARI? Yes  No 

c. Noted in CMHP? Yes  No 

d. Recommended by TWG? Yes  No 

Additional comments:

________________________________________________________________________________________________________________________________________________

________________________________________________________________________

7. Groups and agencies involved

a. Lead agent: ____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

b. Agency/agencies participating:

 mental health	 Medicare administration
 child welfare	 Medicaid administration
 alcohol & substance abuse	 adult criminal justice
 aging services	 juvenile criminal justice
 veterans’ affairs	 disability services
 housing	 vocational rehabilitation
 education- early childhood	 TANF administration
 education- k-12	 other:_______________________________
 education- post secondary	 other:_______________________________

7a & 7b Additional comments:

____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

8. Population(s) affected:

Ages:  children  adolescents  adults  older persons  across life span

Genders:  male  female  male and female

 All	 Asian	 Black or African American
 American Indian	 Alaska Native	 Native Hawaiian or other Pacific Islander
 White	 Latino/Hispanic (regardless of ethnicity)

Race:

 All	 Homeless	 Refugee/Immigrant
 Rural	 Urban	 Students
 Gay/Lesbian/Bisexual/Transgender	 Other (specify) _______________

Culture:

Additional Comments: ____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

9. Referent: ____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

10. Standard for completion: ____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

Start Date		Completion Date
Month:____Year:____		Month:____Year:____
or		or
 Jan-Mar	20__ __	 Jan-Mar	20__ __
 Apr-June	20__ __	 Apr-June	20__ __
 July-Sept	20__ __	 July-Sept	20__ __
 Oct-Dec	20__ __	 Oct-Dec	20__ __

11. Anticipated Time Frame:

12. Target populations to be trained/In need: ____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

Other characteristics of item: ____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

Additional Comments: __________________________________________________________________________________________________________________________________________________

_________________________________________________________________________

OMB No. 0930-xxxx

Expiration Date: _____

GPRA Form II

Activity Result: Completion  Deletion 

Date Form Completed: ________

Linked to GPRA  Indicator(s) No. 1 2 3 4 5 6 7

1. Entry Number: _____________________

2. Working Title: Please give working title appearing on Target Activity Add/Change/Edit Form

______________________________________________________________________________________________________________________________________________________

___________________________________________________________________________

If yes, fill out 3b-3d, skip 4a & 4b

If no, skip 3b-3d, fill out 4a & 4b

3 . Completion:

a. Completed? Yes  No 

b. Standard for completion applied: Please describe completion in relation to measure standard indicated on Target Activity Add/Change/Edit Form (Item 10)

________________________________________________________________________________________________________________________________________________

________________________________________________________________________

c. Date of completion: _______________________

d. Size of impact

GPRA 2: No. of unduplicated people trained as a result of this training ______

GPRA 5: No. of organizations obtaining + analyzing data due to target activity ______

GPRA 6: No. of consumers/family members in statewide org. due to target activity ______

GPRA 7: No. of programs using practices consistent with CMHP as result of target activity _________

GPRA 1, 3 & 4 (optional): Please estimate the size of impact using most appropriate metric: persons affected, dollars affected, capacity increase, etc.

__________________________________________________________________________________________________________________________________________________

_________________________________________________________________________

4. Deletion

a. Deleted? Yes  No 

b. Reason for deletion:

____________________________________________________________________________________________________________________________________________________

__________________________________________________________________________

Attachment 7

Discussion Guide and Protocols for Provider Interviews

OMB No. 0930-xxxx

Expiration Date: ______

Discussion Guide

Please confirm the following background information:

Name/Position: ___________________________________________

Organization: ___________________________________________

Address: ___________________________________________

Phone: _________________ Fax: ___________________

E-mail Address: ___________________________________________

Please indicate type of service provider: Inpatient ____ Outpatient _____

Residential _____ Emergency Services ____ Trade Association _____

1. Are you aware of that your State was awarded a Federal MHT SIG grant?

Yes ______ (go to Question 2) No ______ (go to Question 3)

2. If yes,

How did you become aware of this grant?

What changes are being expected of your organization (or your members for trade association)? Are the changes expected of your organization compatible with what it can reasonably be expected to do?

Is your organization (or your members for trade association) getting any incentives to make your program consistent with transformation grant objectives? If yes, what are they?

Is your organization (or your members for trade association) having problems adhering to the transformation grant objectives? Are there barriers to making desirable changes? Are there missing supports?

3. If no, are you aware of efforts to transform the mental health system in your State? If yes, please specify.

4. (For follow up interviews) Since last year, what are the main changes in terms of the MHT SIG grant for your program (or your members for trade association)?

Protocols

The protocols for provider interviews are:

Frequency: Three times – February of grant years 3-5

Duration: Up to 30 minutes for each interview

Sampling: Structured sample by type of provider

Approach: Phone interviews

Audiences: Providers to be interviewed are:

Inpatient service providers – adult (3) and children/youth (1)
Residential service providers – adult (4) and children/youth (2)
Outpatient service providers – adults (5) and children/youth (5)
Emergency service providers (3)
Professional associations (3)

Burden: Depending on the distribution of providers in each State, up to 28 interviews with providers will be done at each time period. The total burden per State is 14 hours each year and 42 hours for all three years. Total burden for all nine States is 378 hours.

Attachment 8

Measurement of Recovery, Resilience, and System Orientation

Introduction

This Attachment contains four different sections regarding the measurement of recovery, resilience and system orientation towards recovery, as follow:

Guidelines for selection of recovery, resilience, and system measures
List of candidate measures that States might consider using
Protocols to be followed in using these measures
Cost estimate for conducting recovery, resilience and system orientation interviews.

Guidelines for Selection of Recovery, Resilience, and System Measures

For the cross-site evaluation, States will need to select measures to use to measure 1) recovery for adults, 2) resilience for children/youth, and 3) system orientation towards recovery. States will need to select measures that meet the two CMHS criteria shown below. The cross-site team will support States with selection of measures as requested.

Required criteria for the measurement tools are:

Have established validity and reliability metrics for use in measuring recovery, resilience or system orientation towards recovery, or be existing recovery, resilience, or system orientation instruments with some psychometric testing
Be consistent with the SAMHSA National Consensus Definition of Recovery

Recovery Measures

The types of things that these measures tend to cover are:

Family and social relationships or social/community connectedness
Confidence in abilities to manage mental health needs
Hope, optimism or goal orientation
Empowerment, self-direction or responsibility for one’s life or recovery
Meaning in life or in recovery process
Well-being, wellness or self-care
(potential) Access to resources to meet basic needs

Resilience Measures

The types of things that these measures tend to cover are:

Feelings about self and ability to cope
Self management of/control over symptoms
Family and social relationships
(optional) Community/school connections
(optional) Alcohol/drug use

System Measures

These measures are intended to help identify whether:

Mental health services are person-centered
The mental health system is consumer driven
The mental health system focuses on having persons develop full and satisfying lives in the community

The types of things that these system measures tend to cover are:

Consumers have choice/control of services/providers
Consumers are given hope for recovery/improvement/positive changes
Consumers are supported in having positive relationships
Treatment planning is person-centered
Consumers are treated as full partners
(optional) Consumers get housing, employment and transportation support

Candidate Measures States May Want to Consider

Measures of Individual Recovery

Mental Health Recovery Measure (MHRM)

Young & Bullock, 2003

Ohio Mental Health Consumer Outcomes System

Ohio DMH Office of Program Evaluation and Research, 2004

Recovery Assessment Scale (RAS)

Giffort, Schmook, Woody, Vollendorf & Gervain, 1995; Corrigan, Giffort, Rashid, Leary & Okeke, 1999

Recovery Markers Questionnaire (RMQ)

Ridgway, 2004

Recovery Measurement Tool Version 4 (RMT)

Ralph, 2003

Measures of Recovery Promoting Environments (i.e., System Measures)

Elements of a Recovery Facilitating System (ERFS)

CT MHT SIG Evaluation Team, 2007

Recovery Enhancing Environment Measure (REE)

Ridgway, 2004

Recovery Oriented Systems Indicators Measure (ROSI)

Dumont, Ridgway, Onken, Dornan & Ralph, 2005

Recovery Promotion Fidelity Scale

Hawai’i Department of Health/Adult Mental Health

Recovery Promoting Relationships Scale (RPRS)²

Russinova, Rogers & Ellison, 2006

Recovery Self-Assessment Revised (RSA-R)

O’Connell, Tondora, Croog, Evans & Davidson, 2007

Optional/Complementary Measures

Competency Assessment Instrument (CAI)

Chinman, Young, Row, Forquer, Knight & Miller, 2003

Discrimination Experience subscale/Internalized Stigma of Mental Illness (ISMI) scale

Ritsher, Otilingam & Grajales, 2003

Recovery Attitudes Questionnaire (RAQ-7)

Borkin, Steffen, Ensfield et al., 1998

Recovery Attitudes Questionnaire (RAQ-16)

Steffen, Borkin, Krzton, Wishnick & Wilder, 1998

Recovery Knowledge Inventory (RKI)

Bedregal, O’Connell & Davidson, 2006

Resilience Measures

Assessing Developmental Strengths Questionnaire (ADS)

Resiliency Canada, 2003

Behavioral & Emotional Rating Scale-2 (BERS-2)

Epstein, 1998

Child & Youth Resilience Measure (CYRM)

Ungar, 2003

Connor-Davidson Resilience Scale (CD-RISC)

Connor & Davidson, 2003

Developmental Assets Profile (DAP)

The Search Institute, 2004

Healthy Kids Resilience Assessment (HKRA)

Constantine, Bernard & Diaz (WestEd), 1998

Ohio Scales

Ohio Department of Mental Health, 1999

Resilience Scale

Wagnild & Young, 1999

Protocols for Measuring Recovery, Resilience and System Orientation

Evaluation Design

The evaluation design requires States to identify two or more services for adults with severe and persistent mental illness and children/youth with severe emotional disturbance that have been or are anticipated to be impacted by the MHT SIG (the impacted group) and two or more services that have not been (the non-impacted group). The impacted group services will be ones that were not “far along” in implementing transformation prior to receipt of the MHT SIG grant. To maximize the opportunity for system transformation to have taken place, the impacted group will be enrolled after the follow-up data for the non-impacted group have been collected.

The evaluation design is a non-equivalent group design with pretest and posttest measures. This design permits causal inference although threats to validity from history and other sources need to be addressed. A number of different types of statistical analyses may be possible including meta-analysis where different measures are used, path analysis and structural equation modeling, analysis of covariance, and random regression. The choice of methods will depend on the question being asked and the characteristics of the data. Without more knowledge of the instruments States will select, it is premature to be more specific about data analytic methods.

Participant Selection

Consumers (and providers and family members, as necessary) will be randomly selected from the pool of existing and entering consumers. Replacement sampling will be employed to achieve full samples at baseline. States will need to enroll a minimum of 75 adults and 75 children/youth from the MHT SIG impacted services and the same number from non-impacted services. All participants will be interviewed at baseline and after one year.

Table G-1 shows required interviews in standard font and optional interviews in italics with parentheses. Each interview group is denoted by an “I” for individual measure and/or an “S” for system orientation measure. Thirty-eight is a very rough estimate of the number of providers who might be interviewed. This assumes consumers selected for the evaluation share primary providers at about a 2:1 ratio, and no administrators are interviewed. These assumptions may not hold true on the State and its measures.

Table G-1. Required and Optional Numbers of Interviews
Group	Baseline	12 months	Total
Adult non-impacted – first cohort
Consumers (I, S)	75	75	150
Family/SO (S)	(75)	(75)	(150)
Provider (S)*	(38)	(38)	(76)
Child/youth non-impacted – first cohort
Child/youth/family (I)	75	75	150
Family (I)	(75)	(75)	(150)
Provider (I)	(75)	(75)	(150)
Adult transformed – second cohort
Consumers (I, S)	75	75	150
Family/SO (S)	(75)	(75)	(150)
Provider (S)*	(38)	(38)	(76)
Child/youth transformed – second cohort
Child/youth/family (I)	75	75	150
Family (I)	(75)	(75)	(150)
Provider (I)	(75)	(75)	(150)
Total number expected (and optional)	300 (826)	300 (826)	600 (1,652)
Total cost expected (and optional)	$37,500 ($103,250)	$37,500 ($103,250)	$75,000 ($206,500)

Measurement of Recovery, Resilience and System Orientation

Data on system orientation to recovery and facilitation of individual recovery and resilience will be collected using the same sampling technique and data collection approach, except that system orientation information will only be collected from or for adult consumers. Depending on the system orientation measure selected and the State’s decisions about how to use this measure, providers and family members/significant others may also be asked to complete the systems measure. If such a measure is selected, a single provider’s response would likely be matched with multiple consumer responses. At a minimum, a single response per consumer per data collection point, provided by the consumer, will be collected.

Information on resilience will be collected for each child/youth respondent at each data collection point. Depending on the measure selected, the age of the child/youth and the State’s decision, this information may be collected from some combination of the child/youth, parent/caregiver or the provider. The minimum expectation is a single response per child/youth per data collection point, provided by either the child/youth or the parent/caregiver. For adolescents, responses from both the parent and the youth would be preferable. As with the systems measures for adults, multiple perspectives are desirable in measuring child/youth resilience/recovery.

Baseline measures will be taken for individual recovery for adults and resilience for children/youth. Follow-up measures will address individual recovery and resilience for adults and children/youth as well as system orientation of services for adults.

Timing of Interviews

To complete this study within the grant period, the States will need to enroll persons in the non-impacted condition between January and April of their third grant year, and persons in the impacted condition between May and August of their fourth grant year. Since admissions to any one program can be slow, States may have to enroll persons from multiple programs of the same type.

Data Collection Approaches

Data may be collected by in-person, mail, phone, or Internet interviews. States may want to employ mail, phone or internet surveys to the degree possible to reduce the overall cost and burden of this component of the evaluation. However, it is anticipated that follow-up data collection will involve interviews, as it is unlikely that the required 75% follow-up rate can be achieved through other means. Additionally, issues of accessibility are anticipated to pose a greater challenge in mail, phone or internet administration.

Selection of Instruments

It is understood that the instrument selection process will be a collaborative one within States. States will need to have selected their measures that meet CMHS criteria by the time OMB clearance is granted, which may be as early as January 1, 2008.

It should be noted that while several instruments offer limited translations, States will need to consider the degree to which these translations/adaptations alone will ensure full participation among members of the cultural communities that make up the State. States will be encouraged to consider this issue in selecting their recovery and resilience measures, and the cross-site evaluation will support this process by providing any available information about the cultural relevance of each instrument, as well as more general information about instrument translation.

Data Analysis

Data collected on recovery, resilience and system orientation will be aggregated into indicators according to the scoring plans crafted by the original developers of the measures. Then the association between MHT SIG State categories and scores on these variables will be analyzed using parametric statistics involving repeated measures.

In addition to analyzing the recovery and resilience data across States, the cross-site evaluation will draw upon the expertise of the local evaluators and stakeholders to gather information about the ways in which these components of the evaluation do and do not reflect the experiences of the diverse citizenry of each State. This information will be incorporated into the analysis, findings and recommendations of the cross-site evaluation.

Estimate of Recovery and Resilience Component Interview Costs

Cost estimates of in-person interview range from $125 - $200 per interview. This project’s estimate of $125 is supported by the following cost break down:

Interviewer time: 2 hours @ $20/hour	40.00
Travel costs	10.00
Participant compensation	25.00
Materials	2.00
Other direct (phone, etc)	5.00
Training	5.00
Overhead (35% of $87.00 total direct)	30.45
Total	$117.45

It is important to note that the cost shown reflect all interview and related costs (e.g. training, materials) but do not reflect other costs, such as those associated with allocation of a project manager and other tracking costs.

It should also be noted that the minimum/required costs only include interviews with a single participant per consumer (adult consumers and either child/youth consumers or caregivers) and do not include any additional family, significant other or provider interviews.

Finally, the costs do not include expenses associated with translating or adapting the measures for use with people who are not comfortable communicating in English. While several instruments offer limited translations, States will need to consider the degree to which these translations/adaptations will ensure full participation among members of the cultural communities that make up the State.

Attachment 9

Individual Interview Guides for Adults, Caregivers of Children and Youth,

and Children/Youth

OMB No. 0930-xxxx

Expiration Date: _____

Mental Health Transformation

State Incentive Grant

ADULT CONSUMER INTERVIEW GUIDE

BASELINE AND FOLLOW-UP

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

BASELINE DIAGNOSTIC REPORTING FORM

This form is to be completed separately from the consumer baseline interview by a staff person. Diagnostic information should be obtained from the management information system.

List the consumer’s primary DSM-IV diagnosis below.

DSM-IV DIAGNOSIS DSM-IV CODE

________________________ ___ ___ ___- ___ ___

Completed by: __________________________________________________

(Please print)

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

FOLLOW-UP DIAGNOSTIC REPORTING FORM

This form is to be completed separately from the consumer follow-up interview by a staff person. Diagnostic information should be obtained from the management information system.

List the consumer’s primary DSM-IV diagnosis below.

DSM-IV DIAGNOSIS DSM-IV CODE

________________________ ___ ___ ___- ___ ___

Completed by: __________________________________________________

(Please print)

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

ADULT CONSUMER BASELINE INTERVIEW

INTRODUCTION

My name is ______________________________, and I’m working for the ________________________. We’d like to thank you for giving up your time to help us today. This is a national study with different sites throughout the United States, and through this study we hope to learn more about what’s helpful to people receiving mental health services.

I will ask you questions about yourself such as your age, race and ethnicity. Then I will ask how things are going for you right now and about the services you are receiving. For most of the questions you will be asked to give a response to a statement. I will write down your answers for each question so we can compare your answers to those given by other people participating in this study. We will ask you to answer these questions again in 12 months.

All individuals are asked exactly the same questions. Everything you say will be confidential. Your name will not be connected with your answers. You may not want to answer a particular question. If you decide not to answer a question, please tell me and we will skip it. We can also stop the interview at any time. Your decision to participate will not affect any services you now receive or expect to receive in the future.

Remember, there is no right or wrong answers. We want to know what is true for you. Your answers are an important part of this project, and we value the information you give us.

The questions should take about ____ to ___ minutes to answer.

Do you have any questions before I start?

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

ADULT CONSUMER FOLLOW-UP INTERVIEW

INTRODUCTION

This interview is a follow-up to the interview you completed 12 months ago. As before, I will ask how things are going for you right now and about the services you are receiving and have received. For most of the questions you will be asked to give a response to a statement. I will write down your answers for each question so we can compare your answers to those given by other people participating in this study.

Remember, there is no right or wrong answers. We want to know what is true for you. Your answers are an important part of this project, and we value the information you give us.

The questions should take about ____ to ___ minutes to answer.

Do you have any questions before I start?

OMB No. 0930-0285

Expiration Date 04/30/2010

CMHS NOMs

Adult Consumer Outcome Measures

for Discretionary Service Programs

(Modified for MHT SIG Use)

A. RECORD MANAGEMENT

Consumer ID |____|____|____|____|____|____|____|____|____|____|____|

State |____|____|____|____|____|____|____|____|____|____|

Site/Program |____|____|____|____|____|____|____|____|____|____|

Interview Type [SELECT ONLY ONE]

Baseline
12 month Follow-up

Cohort

Impacted
Non-Impacted
Unknown (baseline interview only)

Interview Date |____|____| / |____|____| / |____|____|____|____|

MONTH DAY YEAR

RECORD MANAGEMENT (Continued) - DEMOGRAPHICS

[DEMOGRAPHIC DATA ARE ONLY COLLECTED AT THE BASELINE INTERVIEW]

1. What is your gender?

MALE

 FEMALE

 TRANSGENDER

OTHER (SPECIFY) _____________________________________

REFUSED

2. Are you Hispanic or Latino?

 YES

 NO

 REFUSED

[IF YES] What ethnic group do you consider yourself? Please answer yes or no for each of the following. You may say yes to more than one.

Yes No REFUSED

Central American   

Cuban   

Dominican   

Mexican   

Puerto Rican   

South American   

OTHER    [IF YES, SPECIFY BELOW]

(SPECIFY) ______________________________

3. What race do you consider yourself? Please answer yes or no for each of the following. You may say yes

to more than one.

Yes No REFUSED

Black or African American   

Asian   

Native Hawaiian or other Pacific Islander   

Alaska Native   

White   

American Indian   

What is your month and year of birth?

|____|____| / |____|____|____|____|

MONTH YEAR  REFUSED

C. STABILITY IN HOUSING

1. In the past 30 days, where have you been living most of the time?

[DO NOT READ RESPONSE OPTIONS TO THE CONSUMER. SELECT ONLY ONE.]

 OWNED OR RENTED HOUSE, APARTMENT, TRAILER, ROOM

 SOMEONE ELSE’S HOUSE, APARTMENT, TRAILER, ROOM

 HOMELESS (SHELTER, STREET/OUTDOORS, PARK)

 GROUP HOME

 ADULT FOSTER CARE

 TRANSITIONAL LIVING FACILITY

 HOSPITAL (MEDICAL)

 HOSPITAL (PSYCHIATRIC)

 CORRECTIONAL FACILITY (JAIL/PRISON)

 NURSING HOME

 VA HOSPITAL

 VETERAN’S HOME

 MILITARY BASE

 OTHER HOUSED (SPECIFY) _______________________________________________

 REFUSED

 DON’T KNOW

D. EDUCATION AND EMPLOYMENT

Are you currently enrolled in school or a job training program?

[IF ENROLLED] Is that full time or part time?

NOT ENROLLED

ENROLLED, FULL TIME

ENROLLED, PART TIME

OTHER (SPECIFY)______________

 REFUSED

 DON’T KNOW

What is the highest level of education you have finished, whether or not you received a degree?

LESS THAN 12^TH GRADE
12^TH GRADE/HIGH SCHOOL DIPLOMA/EQUIVALENT (GED)
VOC/TECH DIPLOMA
SOME COLLEGE OR UNIVERSITY
BACHELOR’S DEGREE (BA, BS)
GRADUATE WORK/GRADUATE DEGREE
REFUSED
DON’T KNOW

D. EDUCATION AND EMPLOYMENT (Continued)

3. Are you currently employed? [CLARIFY BY FOCUSING ON STATUS DURING MOST OF THE PREVIOUS WEEK, DETERMINING WHETHER CONSUMER WORKED AT ALL OR HAD A REGULAR JOB BUT WAS OFF WORK.]

EMPLOYED FULL TIME (35+ HOURS PER WEEK, OR WOULD HAVE BEEN)
EMPLOYED PART TIME

 UNEMPLOYED, LOOKING FOR WORK

 UNEMPLOYED, DISABLED

 UNEMPLOYED, VOLUNTEER WORK

 UNEMPLOYED, RETIRED

 UNEMPLOYED, NOT LOOKING FOR WORK

 OTHER (SPECIFY) ___________

 REFUSED

DON’T KNOW

3a. [IF EMPLOYED], Is your employment competitive or sheltered?

 COMPETITIVE EMPLOYMENT

 SHELTERED EMPLOYMENT

 REFUSED

 DON’T KNOW

E. CRIME AND CRIMINAL JUSTICE STATUS

1. In the past 30 days, how many times have you been arrested?

|____|____| TIMES  REFUSED  DON’T KNOW

I. REASSESSMENT STATUS

[SECTION I IS REPORTED BY PROGRAM STAFF ONLY AT FOLLOW-UP]

1. What is the reassessment status of the consumer?

 Completed interview within specified window

 Completed interview outside specified window

 Refused interview

 No contact within 90 days of last encounter

 Other (Specify) ________________________

2. Is the consumer still receiving services from your project?

 Yes

 No

J. CLINICAL DISCHARGE STATUS

[SECTION J IS REPORTED BY PROGRAM STAFF ONLY IF A CONSUMER HAS BEEN DISCHARGED]

1. On what date was the consumer discharged?

|____|____| / |____|____|____|____|

MONTH YEAR

2. What is the consumer’s discharge status?

 Mutually agreed cessation of treatment

 Death

 No contact

 Clinically referred out

 Other (Specify) __________________________________

K. SERVICES RECEIVED

[SECTION K IS REPORTED BY PROGRAM STAFF AT FOLLOW-UP]

1. On what date did the consumer last receive services?

|____|____| / |____|____|____|____|

MONTH YEAR

[IDENTIFY ALL OF THE SERVICES YOUR PROJECT PROVIDED TO THE CONSUMER SINCE HIS/HER LAST INTERVIEW; THIS INCLUDES CMHS-FUNDED AND NON-FUNDED SERVICES.]

Core Services Provided

Yes No

1. Screening  

2. Assessment  

3. Treatment Planning or Review  

4. Psychopharmacological Services  

5. Mental Health Services  

[IF YES, PLEASE SELECT THE FREQUENCY MENTAL HEALTH SERVICES WERE DELIVERED]:

Daily  Weekly  Less than Monthly  Monthly 

6. Co-Occurring Services  

7. Case Management  

8. Trauma-specific Services  

9. Was the consumer referred to another provider for any of the above core services?

Yes  No 

Support Services Provided

Yes No

1. Medical Care  

2. Employment Services  

3. Family Services  

4. Child Care  

5. Transportation  

6. Education Services  

7. Housing Support  

8. Social Recreational Activities  

9. Consumer Operated Services  

10. HIV Testing  

11. Was the consumer referred to another provider for any of the above support services?



Yes  No 

L. RECOVERY

USE INTRO FROM INSTRUMENT CHOSEN

M. CONCLUSION (FOR BASELINE)

Before we end, are there any thoughts or issues that you’d like to talk about?

___________________________________________________________________________

I want to remind you that we will meet with each participant in 12 months and ask a similar set of questions. Thanks for your participation.

We look forward to meeting with you again in 12 months.

M. CONCLUSION (FOR FOLLOW-UP)

Before we end, are there any thoughts or issues that you’d like to talk about?

___________________________________________________________________________

Thanks for your participation.

OMB No. 0930-xxxx

Expiration Date: ______

Mental Health Transformation

State Incentive Grant

CAREGIVER OF CHILD/YOUTH INTERVIEW GUIDE

BASELINE AND FOLLOW-UP

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

BASELINE DIAGNOSTIC REPORTING FORM

This form is to be completed separately from the caregiver baseline interview by a staff person. The child/youth’s diagnostic information should be obtained from the management information system.

List the child/youth’s primary DSM-IV diagnosis below.

DSM-IV DIAGNOSIS DSM-IV CODE

________________________ ___ ___ ___- ___ ___

Completed by: __________________________________________________

(Please print)

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

FOLLOW-UP DIAGNOSTIC REPORTING FORM

This form is to be completed separately from the caregiver follow-up interview by a staff person. The child/youth’s diagnostic information should be obtained from the management information system.

List the child/youth’s primary DSM-IV diagnosis below.

DSM-IV DIAGNOSIS DSM-IV CODE

________________________ ___ ___ ___- ___ ___

Completed by: __________________________________________________

(Please print)

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

CAREGIVER OF CHILD/YOUTH BASELINE INTERVIEW

INTRODUCTION

My name is ______________________, and I’m working for the ________________________. We’d like to thank you for giving up your time to help us today. This is a national study with different sites throughout the United States, and through this study we hope to learn more about what’s helpful to people receiving mental health services.

I will ask you questions about the child/youth that you care for, such as his/her age, race and ethnicity. Then I will ask how things are going for him/her right now and about the services she/he is receiving. For most of the questions you will be asked to give a response to a statement. I will write down your answers for each question so we can compare your answers to those given by other people participating in this study. We will ask you to answer these questions again in 12 months.

All individuals or caregivers are asked exactly the same questions. Everything you say will be confidential. Your name or the name of the child/youth that you care for will not be connected with your answers. You may not want to answer a particular question. If you decide not to answer a question, please tell me and we will skip it. We can also stop the interview at any time. Your decision to participate will not affect any services that the child/youth that you care for now receives or expect to receive in the future.

Remember, there is no right or wrong answers. We want to know what is true for the child/youth in your care. Your answers are an important part of this project, and we value the information you give us.

The questions should take about ____ to ___ minutes to answer.

Do you have any questions before I start?

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

CAREGIVER OF CHILD/YOUTH FOLLOW-UP INTERVIEW

INTRODUCTION

My name is __________________________, and I’m working for the _____________________. We’d like to thank you for giving up your time to help us today. This is a national study with different sites throughout the United States, and through this study we hope to learn more about what’s helpful to people receiving mental health services.

This interview is a follow-up to the interview you completed 12 months ago. As before, I will ask how things are going for the child/youth that you care for right now and about the services he or she is receiving and has received. For most of the questions you will be asked to give a response to a statement. I will write down your answers for each question so we can compare your answers to those given by other people participating in this study.

All individuals are asked exactly the same questions. Everything you say will be confidential. Your name or the name of the child/youth that you care for will not be connected with your answers. You may not want to answer a particular question. If you decide not to answer a question, please tell me and we will skip it. We can also stop the interview at any time. Your decision to participate will not affect any services you now receive or expect to receive in the future.

The questions should take about ____ to ___ minutes to answer.

Do you have any questions before I start?

OMB No. 0930-0285

Expiration Date 04/30/2010

CMHS NOMs

Child/Youth Consumer Outcome Measures

for Discretionary Programs

Caregiver Respondent Version

(Modified for MHT SIG Use)

A. RECORD MANAGEMENT

Consumer ID |____|____|____|____|____|____|____|____|____|____|____|

State |____|____|____|____|____|____|____|____|____|____|

Site/Program |____|____|____|____|____|____|____|____|____|____|

Interview Type [SELECT ONLY ONE]

Baseline
12 month Follow-up

Cohort

Impacted
Non-Impacted
Unknown (baseline interview only)

Interview Date |____|____| / |____|____| / |____|____|____|____|

MONTH DAY YEAR

RECORD MANAGEMENT (Continued) - DEMOGRAPHICS

[DEMOGRAPHIC DATA ARE ONLY COLLECTED AT THE BASELINE INTERVIEW]

1. What is your child’s gender?

MALE

 FEMALE

 TRANSGENDER

OTHER (SPECIFY) _____________________________________

REFUSED

2. Is your child Hispanic or Latino?

 YES

 NO

 REFUSED

[IF YES] What ethnic group do you consider your child? Please answer yes or no for each of the following. You may say yes to more than one.

Yes No REFUSED

Central American   

Cuban   

Dominican   

Mexican   

Puerto Rican   

South American   

OTHER    [IF YES, SPECIFY BELOW]

(SPECIFY) ______________________________

3. What race do you consider your child? Please answer yes or no for each of the following. You may

say yes to more than one.

YES NO REFUSED

Black or African American   

Asian   

Native Hawaiian or other Pacific Islander   

Alaska Native   

White   

American Indian   

What is your child’s month and year of birth?

|____|____| / |____|____|____|____|

MONTH YEAR  REFUSED

C. STABILITY IN HOUSING

1. In the past 30 days, where has your child been living most of the time?

[DO NOT READ RESPONSE OPTIONS TO THE CAREGIVER. SELECT ONLY ONE.]

 CAREGIVER’S OWNED OR RENTED HOUSE, APARTMENT, TRAILER, OR ROOM

 SOMEONE ELSE’S HOUSE, APARTMENT, TRAILER, OR ROOM

 HOMELESS (SHELTER, STREET/OUTDOORS, PARK)

 GROUP HOME

 FOSTER CARE (SPECIALIZED THERAPEUTIC TREATMENT)

 TRANSITIONAL LIVING FACILITY

 HOSPITAL (MEDICAL)

 HOSPITAL (PSYCHIATRIC)

 CORRECTIONAL FACILITY (JUVENILE DETENTION CENTER/JAIL/PRISON)

 OTHER HOUSED (SPECIFY) _______________________________________________

 REFUSED

 DON’T KNOW

Who has your child lived with during the past 30 days? [THE INTERVIEWER MAY CHOOSE MORE THAN ONE ANSWER.]

 Biological parent(s)

 Adoptive parent(s)

 Relative other than parent(s)

 Non-relative

 Independent living

 Refused

 Don’t Know

D. EDUCATION

During the last 30 days of school, how many days was your child absent for any reason?

 0 DAYS

 1 DAY

 2 DAYS

 3 TO 5 DAYS

 6 TO 10 DAYS

 MORE THAN 10 DAYS

 REFUSED

 DON’T KNOW

 NOT APPLICABLE

a. [If absent], how many days were unexcused absences?

 0 DAYS

 1 DAY

 2 DAYS

 3 TO 5 DAYS

 6 TO 10 DAYS

 MORE THAN 10 DAYS

 REFUSED

 DON’T KNOW

 NOT APPLICABLE

What is the highest level of education your child has finished, whether or not he or she received a degree?

NEVER ATTENDED
PRESCHOOL
KINDERGARTEN
1^ST GRADE
2^ND GRADE
3^RD GRADE
4^TH GRADE
5^TH GRADE
6^TH GRADE
7^TH GRADE
8^TH GRADE
9^TH GRADE
10^TH GRADE
11^TH GRADE
12^TH GRADE/HIGH SCHOOL DIPLOMA/EQUIVALENT (GED)
VOC/TECH DIPLOMA
SOME COLLEGE OR UNIVERSITY
REFUSED
DON’T KNOW

E. CRIME AND CRIMINAL JUSTICE STATUS

1. In the past 30 days, how many times has your child been arrested?

|____|____| TIMES  REFUSED  DON’T KNOW

I. REASSESSMENT STATUS

[SECTION I IS REPORTED BY PROGRAM STAFF ONLY AT FOLLOW-UP]

1. What is the reassessment status of the consumer?

 Completed interview within specified window

 Completed interview outside specified window

 Refused interview

 No contact within 90 days of last encounter

 Other (Specify) ________________________

2. Is the consumer still receiving services from your project?

 Yes

 No

J. CLINICAL DISCHARGE STATUS

[SECTION J IS REPORTED BY PROGRAM STAFF ONLY IF A CONSUMER HAS BEEN DISCHARGED]

1. On what date was the consumer discharged?

|____|____| / |____|____|____|____|

MONTH YEAR

2. What is the consumer’s discharge status?

 Mutually agreed cessation of treatment

 Death

 No contact

 Clinically referred out

 Other (Specify) __________________________________

K. SERVICES RECEIVED

[SECTION K IS REPORTED BY PROGRAM STAFF AT FOLLOW-UP]

1. On what date did the consumer last receive services?

|____|____| / |____|____|____|____|

MONTH YEAR

[IDENTIFY ALL OF THE SERVICES YOUR PROJECT PROVIDED TO THE CONSUMER SINCE HIS/HER LAST INTERVIEW; THIS INCLUDES CMHS-FUNDED AND NON-FUNDED SERVICES.]

Core Services Provided

Yes No

1. Screening  

2. Assessment  

3. Treatment Planning or Review  

4. Psychopharmacological Services  

5. Mental Health Services  

[IF YES, PLEASE SELECT THE FREQUENCY MENTAL HEALTH SERVICES WERE DELIVERED]:

Daily  Weekly  Less than Monthly  Monthly 

6. Co-Occurring Services  

7. Case Management  

8. Trauma-specific Services  

9. Was the consumer referred to another provider for any of the above core services?

Yes  No 

Support Services Provided

Yes No

1. Medical Care  

2. Employment Services  

3. Family Services  

4. Child Care  

5. Transportation  

6. Education Services  

7. Housing Support  

8. Social Recreational Activities  

9. Consumer Operated Services  

10. HIV Testing  

11. Was the consumer referred to another provider for any of the above support services?



Yes  No 

L. RESILIENCE

USE INTRO FROM INSTRUMENT CHOSEN

M. CONCLUSION (FOR BASELINE)

Before we end, are there any thoughts or issues that you’d like to talk about?

___________________________________________________________________________

I want to remind you that we will meet with each participant in 12 months and ask a similar set of questions. Thanks for your participation.

We look forward to meeting with you again in 12 months,

M. CONCLUSION (FOR FOLLOW-UP)

Before we end, are there any thoughts or issues that you’d like to talk about?

___________________________________________________________________________

Thanks for your participation.

OMB No. 0930-xxxx

Mental Health Transformation

State Incentive Grant

Expiration Date: _____

CHILD/YOUTH CONSUMER INTERVIEW GUIDE

BASELINE AND FOLLOW-UP

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

BASELINE DIAGNOSTIC REPORTING FORM

This form is to be completed separately from the child/youth baseline interview by a staff person. Diagnostic information should be obtained from the management information system.

List the child/youth’s primary DSM-IV diagnosis below.

DSM-IV DIAGNOSIS DSM-IV CODE

________________________ ___ ___ ___- ___ ___

Completed by: __________________________________________________

(Please print)

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

FOLLOW-UP DIAGNOSTIC REPORTING FORM

This form is to be completed separately from the child/youth follow-up interview by a staff person. Diagnostic information should be obtained from the management information system.

List the child/youth’s primary DSM-IV diagnosis below.

DSM-IV DIAGNOSIS DSM-IV CODE

________________________ ___ ___ ___- ___ ___

Completed by: __________________________________________________

(Please print)

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

CHILD/YOUTH CONSUMER BASELINE INTERVIEW

INTRODUCTION

My name is _______________________, and I’m working for the ___________________________. We’d like to thank you for giving up your time to help us today. This is a national study with different sites throughout the United States, and through this study we hope to learn more about what’s helpful to people receiving mental health services.

Remember, there is no right or wrong answers. We want to know what is true for you. Your answers are an important part of this project, and we value the information you give us.

The questions should take about ____ to ___ minutes to answer.

Do you have any questions before I start?

Mental Health Transformation Subject ID: ___________________

State Incentive Grant Date: _______________________

Intervention

Condition: __________________

0= Non-impacted 1=Impacted

CHILD/YOUTH CONSUMER FOLLOW-UP INTERVIEW

INTRODUCTION

Remember, there is no right or wrong answers. We want to know what is true for you. Your answers are an important part of this project, and we value the information you give us.

The questions should take about ____ to ___ minutes to answer.

Do you have any questions before I start?

OMB No. 0930-0285

Expiration Date 04/30/2010

CMHS NOMs

Child Consumer Outcome Measures

for Discretionary Programs

Child or Adolescent Respondent Version

(Modified for MHT SIG Use)

A. RECORD MANAGEMENT

Consumer ID |____|____|____|____|____|____|____|____|____|____|____|

State |____|____|____|____|____|____|____|____|____|____|

Site/Program |____|____|____|____|____|____|____|____|____|____|

Interview Type [SELECT ONLY ONE]

Baseline
12 month Follow-up

Cohort

Impacted
Non-Impacted
Unknown (baseline interview only)

Interview Date |____|____| / |____|____| / |____|____|____|____|

MONTH DAY YEAR

RECORD MANAGEMENT (Continued) - DEMOGRAPHICS

[DEMOGRAPHIC DATA ARE ONLY COLLECTED AT THE BASELINE INTERVIEW]

1. What is your gender?

MALE

 FEMALE

 TRANSGENDER

OTHER (SPECIFY) _____________________________________

REFUSED

2. Are you Hispanic or Latino?

 YES

 NO

 REFUSED

[IF YES] What ethnic group do you consider yourself? Please answer yes or no for each of the following. You may say yes to more than one.

Yes No REFUSED

Central American   

Cuban   

Dominican   

Mexican   

Puerto Rican   

South American   

OTHER    [IF YES, SPECIFY BELOW]

(SPECIFY) ______________________________

What race do you consider yourself? Please answer yes or no for each of the following. You

may say yes to more than one.

YES NO REFUSED

Black or African American   

Asian   

Native Hawaiian or other Pacific Islander   

Alaska Native   

White   

American Indian   

What is your month and year of birth?

|____|____| / |____|____|____|____|

MONTH YEAR  REFUSED

C. STABILITY IN HOUSING

1. In the past 30 days, where have you been living most of the time?

[DO NOT READ RESPONSE OPTIONS TO THE CHILD/ADOLESCENT. SELECT ONLY ONE.]

 CAREGIVER’S OWNED OR RENTED HOUSE, APARTMENT, TRAILER, OR ROOM

 SOMEONE ELSE’S HOUSE, APARTMENT, TRAILER, OR ROOM

 HOMELESS (SHELTER, STREET/OUTDOORS, PARK)

 GROUP HOME

 FOSTER CARE (SPECIALIZED THERAPEUTIC TREATMENT)

 TRANSITIONAL LIVING FACILITY

 HOSPITAL (MEDICAL)

 HOSPITAL (PSYCHIATRIC)

 CORRECTIONAL FACILITY (JUVENILE DETENTION CENTER/JAIL/PRISON)

 OTHER HOUSED (SPECIFY) _______________________________________________

 REFUSED

 DON’T KNOW

2. Who have you lived with during the past 30 days? [THE INTERVIEWER MAY CHOOSE MORE THAN ONE ANSWER.]

 Biological parent(s)

 Adoptive parent(s)

 Relative other than parent(s)

 Non-relative

 Independent living

 Refused

 Don’t Know

D. EDUCATION

During the last 30 days of school, how many days were you absent for any reason?

 0 DAYS

 1 DAY

 2 DAYS

 3 TO 5 DAYS

 6 TO 10 DAYS

 MORE THAN 10 DAYS

 REFUSED

 DON’T KNOW

 NOT APPLICABLE

a. [If absent], how many days were unexcused absences?

 0 DAYS

 1 DAY

 2 DAYS

 3 TO 5 DAYS

 6 TO 10 DAYS

 MORE THAN 10 DAYS

 REFUSED

 DON’T KNOW

 NOT APPLICABLE

What is the highest level of education you have finished, whether or not you received a degree?

NEVER ATTENDED
PRESCHOOL
KINDERGARTEN
1^ST GRADE
2^ND GRADE
3^RD GRADE
4^TH GRADE
5^TH GRADE
6^TH GRADE
7^TH GRADE
8^TH GRADE
9^TH GRADE
10^TH GRADE
11^TH GRADE
12^TH GRADE/HIGH SCHOOL DIPLOMA/EQUIVALENT (GED)
VOC/TECH DIPLOMA
SOME COLLEGE OR UNIVERSITY
REFUSED
DON’T KNOW

E. CRIME AND CRIMINAL JUSTICE STATUS

1. In the past 30 days, how many times have you been arrested?

|____|____| TIMES  REFUSED  DON’T KNOW

I. REASSESSMENT STATUS

[SECTION I IS REPORTED BY PROGRAM STAFF ONLY AT FOLLOW-UP]

1. What is the reassessment status of the consumer?

 Completed interview within specified window

 Completed interview outside specified window

 Refused interview

 No contact within 90 days of last encounter

 Other (Specify) ________________________

2. Is the consumer still receiving services from your project?

 Yes

 No

J. CLINICAL DISCHARGE STATUS

[SECTION J IS REPORTED BY PROGRAM STAFF ONLY IF A CONSUMER HAS BEEN DISCHARGED]

1. On what date was the consumer discharged?

|____|____| / |____|____|____|____|

MONTH YEAR

2. What is the consumer’s discharge status?

 Mutually agreed cessation of treatment

 Death

 No contact

 Clinically referred out

 Other (Specify) __________________________________

K. SERVICES RECEIVED

[SECTION K IS REPORTED BY PROGRAM STAFF AT FOLLOW-UP]

1. On what date did the consumer last receive services?

|____|____| / |____|____|____|____|

MONTH YEAR

[IDENTIFY ALL OF THE SERVICES YOUR PROJECT PROVIDED TO THE CONSUMER SINCE HIS/HER LAST INTERVIEW; THIS INCLUDES CMHS-FUNDED AND NON-FUNDED SERVICES.]

Core Services Provided

Yes No

1. Screening  

2. Assessment  

3. Treatment Planning or Review  

4. Psychopharmacological Services  

5. Mental Health Services  

[IF YES, PLEASE SELECT THE FREQUENCY MENTAL HEALTH SERVICES WERE DELIVERED]:

Daily  Weekly  Less than Monthly  Monthly 

6. Co-Occurring Services  

7. Case Management  

8. Trauma-specific Services  

9. Was the consumer referred to another provider for any of the above core services?

Yes  No 

Support Services Provided

Yes No

1. Medical Care  

2. Employment Services  

3. Family Services  

4. Child Care  

5. Transportation  

6. Education Services  

7. Housing Support  

8. Social Recreational Activities  

9. Consumer Operated Services  

10. HIV Testing  

11. Was the consumer referred to another provider for any of the above support services?



Yes  No 

L. RESILIENCE

USE INTRO FROM INSTRUMENT CHOSEN

M. CONCLUSION (FOR BASELINE)

Before we end, are there any thoughts or issues that you’d like to talk about?

___________________________________________________________________________

I want to remind you that we will meet with each participant in 12 months and ask a similar set of questions. Thanks for your participation.

We look forward to meeting with you again in 12 months,

M. CONCLUSION (FOR FOLLOW-UP)

Before we end, are there any thoughts or issues that you’d like to talk about?

___________________________________________________________________________

Thanks for your participation.

Attachment 10

Discussion Guide and Protocols for Interviews on Cost Impact

Discussion Guide

Please confirm the following background information:

Name/Position: ___________________________________________

Agency: ___________________________________________

Address: ___________________________________________

Phone: _________________ Fax: ___________________

E-mail Address: ___________________________________________

How were the MHT SIG funds used in your State? How much was spent on TWG, NARI, CMHP and evaluation activities? How much went for actions to address each of the GPRA infrastructure objectives? What other costs were covered by the MHT SIG grant funds?

What did your State do with the funds that would not have been done otherwise? What did your State decide not to do with the funds? Why were the selected uses chosen over other potential uses?

What do you see as the benefits of having the MHT SIG funds available for your State transformation efforts? How is your State identifying these benefits?

Are there any reasons why the benefits might have been less in your State than they would have been otherwise?

Note: Existing data sources will be used to obtain information to the maximum extent feasible. These sources will include State MHT SIG reports, financial reports, initial and continuation grant applications, etc.

Protocols

Prior to each interview, State documents will be reviewed to obtain information about how funds have been used. Phone interviews will be conducted with the Project Directors for each State’s MHT SIG project or their designee. It is anticipated that data collection may require up to one hour and the phone interview will take a half hour. Calls will be made to the States in March of grant years 3 and 5.

Thus, the burden will be 1.5 hours per State interview, 3 hours for both interviews, and 27 hours for all States for both interviews.

Attachment 11

Other National Databases that Might Be Used

U.S. Census Bureau,
- Population Data
- Survey of Income and Program Participation
U.S. Department of Health and Human Services (DHHS)
- Agency for Healthcare Research and Quality – Medical Expenditure Panel Survey (MEPS)
- Centers for Disease Control and Prevention (CDC)
  - Behavioral Risk Factors Surveillance System Data
  - Web-based Injury Statistic Query and Reporting System
  - National Center for Health Statistics
- SAMHSA/CMHS
  - Treatment Episode Data Set
  - CMHS Alternative to Seclusion and Restraint Study Data
  - State Profile System
  - Other State Agency Project Data
  - State Mental Health Revenues and Expenditures Data
  - Report on State transformation efforts by New Freedom Commission goals
American Psychological Association (APA) Directory Survey
American Psychiatric Association National Survey of Psychiatric Practice (NSPP)
Data Collection for the National Association of Social Workers (NASW)
National Alliance on Mental Illness – Grading the States: A Report on America's Health Care System for Serious Mental Illness

Attachment 12

Candidate Instruments for Recovery and Resilience Measurement and Their Psychometric Properties

Measures of Individual Recovery and Name of Developer

Mental Health Recovery Measure (MHRM), Young & Bullock, 2003

Ohio Mental Health Consumer Outcomes System, Ohio DMH Office of Program Evaluation and Research, 2004

Recovery Assessment Scale (RAS), Giffort, Schmook, Woody, Vollendorf & Gervain, 1995; Corrigan, Giffort, Rashid, Leary & Okeke, 1999

Recovery Markers Questionnaire (RMQ), Ridgway, 2004

Recovery Measurement Tool Version 4 (RMT), Ralph, 2003

Measures of Recovery Promoting Environments (i.e., System Measures) and Name of Developer

Elements of a Recovery Facilitating System (ERFS), CT MHT SIG Evaluation Team, 2007

Recovery Enhancing Environment Measure (REE), Ridgway, 2004

Recovery Oriented Systems Indicators Measure (ROSI), Dumont, Ridgway, Onken, Dornan & Ralph, 2005

Recovery Promotion Fidelity Scale, Hawai’i Department of Health/Adult Mental Health

Recovery Promoting Relationships Scale (RPRS)², Russinova, Rogers & Ellison, 2006

Recovery Self-Assessment Revised (RSA-R), O’Connell, Tondora, Croog, Evans & Davidson, 2007

Resilience Measures and Name of Developer

Assessing Developmental Strengths Questionnaire (ADS), Resiliency Canada, 2003

Behavioral & Emotional Rating Scale-2 (BERS-2), Epstein, 1998

Child & Youth Resilience Measure (CYRM), Ungar, 2003

Connor-Davidson Resilience Scale (CD-RISC), Connor & Davidson, 2003

Developmental Assets Profile (DAP), The Search Institute, 2004

Healthy Kids Resilience Assessment (HKRA), Constantine, Bernard & Diaz (WestEd), 1998

Ohio Scales, Ohio Department of Mental Health, 1999

Resilience Scale, Wagnild & Young, 1999

Recovery Measure Overview: Individual Measures

Measure

Versions

Intended for use with…

Administration

Translations

Members of cultural minority groups…

No. of items

minutes to complete

Reliability

Validity

sensitivity to change*

internal consistency ά

test-retest correlation

established measures*

other criteria*

Helped develop measure

included in test samples

MHRM Young & Bullock

Consumer

Adults with psychiatric disabilities

Self
Interview
Mail

Spanish¹

.91-.93

.91

Consumer Instrument OH DMH

Consumer
Provider

Adults receiving mental health services

Self
Interview
Mail
Phone

Japanese
Russian
Spanish
Somalian

30-40

.77-.93²

RAS Corrigan et al

Consumer

Adults with psychiatric disabilities

Self
Interview
Mail
Phone

None

.93

.88

RMQ Ridgway

Consumer

Adults with psychiatric disabilities

Self
Interview

None

5-10

RMT

Ralph

Consumer

Adults with psychiatric disabilities

Self
Interview
Mail
Phone
Internet

None

20³

* This page and next: X indicates that the measure has undergone the specified testing; more detailed results available in the recovery compendium

Recovery Measures Overview: Systems Measures

Measure

Versions

Intended for use in systems or settings serving

Administration

Translations

Members of cultural minority groups…

No. of items

minutes to complete

Reliability

Validity

sensitivity to change*

internal consistency ά

test-retest correlation

established measures*

other criteria*

Helped develop measure

included in test samples

ERFS

CT TSIG Team

Adult/youth
Child caretaker

Adults, youth and children

Self
Interview

None

REE

Ridgway

Consumer

Adults with psychiatric disabilities

Self
Interview

None

166

20-45

.72-.94²

ROSI Dumont et al

Consumer
Administrator⁴

Adults with psychiatric disabilities

Self
Interview
Mail
Phone
Internet⁵

Spanish⁵

X⁵

42⁵

30⁵

.95⁵

RPFS

HI AMHD

Evaluator

Adult mental health setting

Evaluator

None

RPRS Russinova et al

Consumer

Adults with psychiatric disabilities

Self
Interview

Spanish / culturally validated version pending

.98⁶

.72⁶

RSA

O’Connell et al

Consumer
Family/friend
Provider
Administrator

Adults with psychiatric disabilities or substance use diagnoses

Self
Mail

None

X⁷

32⁵

<10

.76-.90^2,
7

¹noted as “in progress” in 2005 ♦ ²range of alphas for subscales ♦ ³based on only a few interviews ♦ ⁴versions do not contain parallel items

⁵consumer survey only ♦ ⁶for total scale ♦ ⁷refers to first version

Representation of Diverse Cultural Communities in Pilot & Field Testing

Individual Measures	African-American/ Black	American Indian & Alaskan Native	Asian-American	Latino	Native Hawaiian & other Pacific Islander
MHRM Young & Bullock	X			X
Adult Consumer Instrument OH Department of Mental Health	X	X	X	X	X
RAS, Corrigan et al	X	X	X	X	X
RMQ, Ridgway	X¹			X¹
RMT² Ralph
System Measures
ERFS² CT TSIG Evaluation Team
REE, Ridgway	X¹			X¹
ROSI³ Dumont et al/ROSI Research Team	X	X	X	X	X
RPFS² Hawai’i AMHD
RPRS, Russinova et al	X	X¹	X^1,4	X	X^1,4
RSA, O’Connell et al	X			X

¹Limited testing ♦ ²Has not undergone formal testing ♦ ³Refers only to consumer self-report survey

⁴Asian American and Pacific Islander reported as one group

Pros and Cons: Individual Recovery Measures

Measure	Pros	Cons
MHRM Young & Bullock	Estimated administration time ≤ 10 minutes Some evidence of sensitivity to change Relatively well-established psychometric properties Available in Spanish (confirm) Developed via grounded theory analysis of consumer experience	Used exclusively with adults with psychiatric disabilities
Adult Consumer Instrument OH DMH	Currently used with all adult SMHA consumers Multiyear history of statewide use Available in multiple languages	Administration time ≥ 20 minutes
RAS Corrigan et al	Used successfully in COSP multisite Relatively well-established psychometric properties Developed via grounded theory analysis of consumer experience	Administration time ≥ 20 minutes Used exclusively with adults with psychiatric disabilities Translated versions not available
RMQ Ridgway	Estimated administration time ≤ 10 minutes Some evidence of sensitivity to change Relatively extensive history of use Contained in RMQ: individual and system measure in one Developed via grounded theory analysis of consumer experience	Used exclusively with adults with psychiatric disabilities Translated versions not available
RMT Ralph	Based on conceptual model of recovery developed by leading consumer/survivor researchers and advocates Items developed by panel of consumers	Administration time ≥ 20 minutes Relatively early stage of development Used exclusively with adults with psychiatric disabilities Translated versions not available

Pros and Cons: Systems Measures

Measure

Pros

Cons

ERFS²

CT TSIG Evaluation Team

Brief
Designed for use in both child and adult service systems
Parallel adult/youth and child/family forms
Based on model of transformed system, well-aligned with systems of care approach
Reviewed/approved by CT consumer & family workgroups

Relatively early stage of development; not yet pilot tested
Translated versions not available

REE

Ridgway

Under revision; short form forthcoming

Relatively extensive history of use
Developed via grounded theory analysis of consumer experience
Contains RMQ; individual and system measure in one
Short form forthcoming

Current format time administration time ≥ 20 minutes
Used exclusively with adults with psychiatric disabilities
Translated versions not available

ROSI

Dumont et al/ROSI Research Team

Relatively extensive history of use
Developed via grounded theory analysis of consumer experience
Collects both consumer perspective and administrative data
Consumer survey available in Spanish

Used exclusively with adults with psychiatric disabilities
Estimated administration time ≥ 20 minutes

RPFS, Hawai’i AMHD

Developed via concept mapping and expert review; both processes completed by consumers and other stakeholders

Relatively early stage of development; not yet pilot tested
No specific consumer response form

RPRS

Russinova et al

Estimated administration time ≤ 10 minutes
Relatively well-established psychometric properties
Spanish translation available; culturally validated version in progress

Used exclusively with adults with psychiatric disabilities

RSA

O’Connell et al

Recently revised

Estimated administration time ≤ 10 minutes
Relatively extensive history of use
Designed for use in settings serving people with mental health and/or substance use needs
Collective multiple perspectives using parallel forms
Includes form designed to capture family perspective

Translated versions not available

Resilience Measure Overview

Measure

Versions

Intended for use with or in systems or settings serving

Administration

Translations

Members of cultural minority groups…

No. of items

minutes to complete

Reliability

Validity

sensitivity to change*

internal consistency ά

test-retest correlation

established measures*

other criteria*

Helped develop measure

included in test samples

ADS

Resiliency Canada

Child
Youth
Adult

Children, youth and adults

Self

62¹

BERS-2

Epstein

Child/youth
Caregiver
Teacher

Children and youth ages 5 - 19

Self
Observer

None

.87-.99²

CYRM

Ungar

Youth

Transition-age youth, as defined by community or culture

Self

Multiple

.66-.84³

CD-RISC

Conner & Davidson

Self-report

Adults in general population and those receiving MH services

Self

.89

.87

* This page and next: X indicates that the measure has undergone the specified testing

Continued on next page.

Resilience Measure Overview, continued

Measure

Versions

Intended for use with or in systems or settings serving

Administration

Translations

Members of cultural minority groups…

# items

minutes to complete

Reliability

Validity

sensitivity to change*

internal consistency ά

test-retest correlation

established measures*

other criteria*

Helped develop measure

included in test samples

DAP

Search Institute

Youth self-report

Youth ages 11-18

Self
Internet

None

10-15

.81-.88⁴

.79-.84⁴

Healthy Kids

Constantine et al

Child self-report
Youth self-report

Students in grades 3-6; adolescents

Self

Spanish

56-59⁵

Ohio Scales

OH DMH

Youth self-report
Caregiver
Provider

Children and youth (5-18) receiving mental health services

Self
Interview

Chinese
Japanese
Korean
Spanish
Russian

.86-.93⁶

Resilience Scale Wagnild & Young

Self-report

Adolescents, adults and older adults

Mail
Self

Spanish
Russian
Swedish

.91

¹child and youth measures also available in 165- and 163-item versions, respectively ♦ ²range of rs for total measure scores of all three versions ♦ ³range for subscales of initial, 58-item version ♦ ⁴averages for subscales ♦ ⁵elementary school student survey has 59 items, adolescent survey has 56 items ♦ ⁶range of alphas for two subscales of caregiver and provider forms

Attachment 13

Summary of Results for the Pilot Test of

Consumer/Family Member Involvement Survey Instrument

Introduction

This summary provides a description of the persons who participated in the pilot test and the recommendations and comments coming from focus group and interviews. Six individual interviews were conducted between September 27 and October 5, 2007, and three persons participated in the focus group conducted on October 12. The notes from the focus group follow the summary below.

Characteristics of Pilot Test Participants

Nine persons participated in the pilot test of the instrument, with six persons being interviewed individually and three persons participating in the focus group. The characteristics of participants were:

Interviews Focus Group

Gender – male 4 2

female 2 1

Age – youth (under 20) 2

adult 3 3
older adult 1

Ethnicity – African American 1

Caucasian 3 3
American Indian 1
Of Hispanic Origin 1

Category – consumer 5 1

– family member 3 2

– provider (also a 1

consumer and family member)

States Represented MD, NM, OH, TX, WA (2) MA

Three of the Consumer/Family Member (CFM) Consultants participated in the interviews: Violeta Maya, Alan Rabideau (3), and Wilma Townsend (2). Ken Duckworth participated in the focus group, both in his role as a Consultant and as a reviewer.

Recommended Changes and Comments

For the comments shown below, when the number of persons making a comment is more than one, the number of persons making the comment is shown in parentheses after the comment. The comments in black are from the individual interviews while the ones in blue are from the focus group. Recommendations and comments are organized by screening questions, the general survey questions, and the Likert scale items.

Screening Questions

Question 1 - In New Mexico, they use behavioral health, not mental health.

- For last item, it might be useful to say private, not-for-profit organization.

Question 2 - Use “local community” instead of just “local”

- Use “local/county” or add county as a separate answer category

- Use “aware” instead of “heard”

- Use “transform” (or use both reform and transform). They may not mean the same thing to everyone. Changes may not be identified as “reform” or “transformation.”

- People may have heard about big national issues, e.g., Virginia, so it is important to be specific that the focus is on the State transformation grant.

- The purpose of this question is unclear – to screen out or for some other reason.

Question 3 - Question is wordy – reduce (2). Try to make clear the respondent needs to be willing to comment on consumer and family member involvement generally.

Remove question (not seen as appropriate for the focus group)
Question needs to be specific to the MHT SIG grant.

General Survey Questions

General – 1) Some persons favored the use of transformation as a better descriptive term than reform for what a person should be commenting on. One person did not like the term mental health user; behavioral health consumer is better. Use of mental health can be a stigma.

2) It is important to be sure the respondents provide answers specifically about the transformation grant versus more general transformation activities.

Questions 1A and B – No changes were suggested.

Question 1C – Change to say 1) “mental health” program staff or evaluator and 2) consumer/family participant rather than testifier/participant

Question 1D – Split question into two parts (3); one person suggested asking about length of involvement first

Question 1E – Split question into two parts (2 + CFM Consultant); one person felt it was better not to split the question. One person felt it was a technical question for a newly involved person and wondered if local efforts would have an impact on State efforts.

Question 2A – Provide an example. It was noted that NAMI is professionally oriented. CFM Consultants suggested using 51% (2).

Question 2B – Say active in transformation (not reform) and make it clear that both formal and informal involvement applies. Change from saying “it’s” to it is.

Question 2C – Rephrase to say: Are you involved with this organization to help transform mental health services?

Question 3 – One person felt reform was a better word than transform.

Questions 4A & B – The difference between these two questions was not easily identified and had to be explained (3). Perhaps, the first question could ask about current “personal” involvement. Rephrase to say: Are you involved to promote mental health transformation? Are you involved with a mental health organization that advocates for mental health transformation? Whose intention is it?

Questions 4C & D

These questions may be redundant. Rephrase to say: What are you trying to accomplish by being involved in mental health transformation efforts?
Consider not asking about “long-term” impact – any impact is an accomplishment and it may not be easy to think about.
Be specific if the question is about the State grant only.

Question 5 – Comments were:

This question needs to be handled in two parts – first frequency, then role.
The word “role” can be divisive and is not positive. Change to “involvement.”
The type of role questions are important since consumers/families can be exploited by being invited to a meeting (so an organization can meet some deliverable).
The questions need to be specific to the type of involvement a person has identified: local, State or both.
Rephrase to say: How often are mental health users and families involved? What is the type of involvement of mental health users and families?
Type of role for last three rows is not appropriate – since it is not clear what the response would indicate.
The type of role questions may be duplicative with Question 1 of the Likert scale items.
We need to be able to capture comments relative to these answers (CFM Consultant).
Add frontier to geographic areas.
Perhaps one wants to ask if there is adequate involvement by diverse cultural groups, age groups and geographic areas. If no, then what should be happening that isn’t. One might want to ask respondent to identify their cultural, age and geographic locale.
For some respondents, it may be helpful to repeat: “How often are they involved in …” before each item.
One person noted that he is on a county board which serves a number of large cities, and on another board that serves five counties, several Indian reservations, and diverse cultural groups (e.g., Chinese and Russian).
These questions can play an educational role and heighten awareness.
Type of role response categories can be confusing; definitions are needed. Type of role responses don’t flow/aren’t clear.
For frequency answers, change sometimes to occasionally, often to frequently, and use either always or almost always
Delete the NA category and replace with Don’t Know
For the type of role answers, delete regular role and replace the NA with Don’t Know
Change “cultural groups” to “cultural, ethnic and religious groups”

Question 6 – Comments on these questions were:

The phrase “non-AYF members” is awkward/confusing and needs to be changed (4). Rephrase to say: Is there an opportunity for A/Y/FY/FA to give assignments to others?
Item on training is too long. Rephrase to say: A/Y/FY/FA hold trainings for others. This may overlap with last item on informing public.
A/Y/FY/FA members lead in needs to be changed (2- based on probe of interviewee and comment from CFM Consultant).
For frequency answers, change sometimes to occasionally, often to frequently, and use either always or almost always.
Delete the NA category and replace with Don’t Know.

Question 7

Provide examples and probe by asking about differences at the local level as well as the State level.
Use lead in phrase: “Since the beginning of this grant” or “How is this different from before the grant began?” The questions need to ask whether there has there been a material change since the grant began and has the grant accelerated the existing conditions or not made a difference.

Question 8A

Do not require a particular number; instead ask: What are the major barriers …?
Also, responses may vary for younger versus older respondents, and it could be that not enough people are “lifted up” (i.e., affected positively by transformation).
If the question is broader than the grant, this needs to be made clear.
Rephrase: What are the things you as a family member should have been involved in that you were not involved in?
This is actually two questions: Should people have been involved who were not involved? For those who were involved, what did/did not happen that they wanted to happen?

Question 8B – Rephrase to say: Please provide an example of ….

Question 9

Don’t require a particular number; instead ask: What are the most important things …?
If the question is broader than the grant, this needs to be made clear.

Likert Scale Items

General – 1) It may be helpful to have a person be able to write the response categories down for reference before going through these items.

2) It may be hard for some persons to decide between agree and strongly agree.

3) What happens when a person falls into more than one category?

4) Split out process items (e.g., Q7) from content items (e.g., Q2, Q17, Q20)

Question 1 – These questions seemed duplicative with Question 5 type of role items.

Question 1 (and other questions) – Change lead in to say: A/Y/FY/FA have meaningful participation in …. In other words, delete the phrase “members of the group I am in.”

Question 2 – Delete this question since one can get from makeup of TWG.

Question 3 – Delete this question since it is vague.

Question 4 – Change “fairly” to “adequately.”

Question 5 – Rephrase to say: Has involvement of (category) made a difference? Delete question since it is hard for one person to make a statement about the influence of a category of persons.

Question 6 – The phrase “MHT SIG participants” is not clear – consider changing to say grant staff or something similar.

Question 7 – Add the word “collaboratively.”

Question 8

Suggest saying “State leaders” rather than “State mental health transformation leaders.” Linguistic had meaning for one person; not for another.
Is this question trying to get at the quality of the person, or process issues? The questions seem to have mixed purposes: questions 12, 17, and 20, are more fact oriented but all the rest are process questions.

Question 9 – One person had the following comments: Put this as the last question since it can bring up strong feelings/change the mood of the interview. It raises her blood pressure. It is not a good question to ask in New Mexico due to existing racial tensions. They are “fire words.” Toss this question out. It is not clear what one is trying to get at. One might say: Mental health problems are seen as being genetic (rather than getting into racial issues). One might break into separate questions. Rephrase in positive manner: I am respected regardless of my/my family member’s disability.

Questions 10 and 11 – No changes were suggested.

Question 12 – It is too long. Rephrase to say: A/Y/FY/FA understand current mental health policy issues. Mental health parity is not a good example (1 interview plus CFM Consultant).

Questions 13 – 15 – No changes were suggested.

Question 16 – It is unclear and too long (2). Rephrase to say: A partnership exists between A/Y/FY/FA and persons who are not consumers/family members.

Question 17 – Change to start by saying: As a result of the grant, A/Y/FY/FA ….

Questions 18 – 21 – No changes were suggested.

Question 22 – It is too long and not as clear as it could be. Rephrase to say: Efforts are made to evaluate A/Y/FY/FA involvement.

Additional Questions, Items and Suggestions

Ask: What changes are occurring in involvement by A/Y/FY/FA?
Ask: Is this something you really believe in?
Add Likert scale items:
- People who have participated in the grant activities are excited about the progress of the grant.
- People who have participated in the grant activities are hopeful about the future of the mental health system as a result of the grant.
- A/Y/FY/FA who have participated in the grant activities have learned a lot (about the policy making process) from grant activities. This might be a rework of Question 11.
Provide some general guidance prior to the interview on things to be thinking about. Note that involvement of you and others is of interest to the evaluation. Indicate a focus will be on how do you see transformation overall?

Summary

All of the interviewees felt the instrument was complete and comprehensive. The reading level was seen as being at the right level. Lower would be talking down to someone, and higher would not be appropriate. One person had suggested trying to have a third grade level before being interviewed, but was fine with the language after going through the instrument.

Notes from Focus Group on Consumer/Family Member Involvement Instrument

Conducted on October 12, 2007

Meeting Participants

Focus Group:

Ken Duckworth, Consumer/Family Member Consultant

Participant 1

Participant 2

Evaluation Team:

Clifton Chow (HSRI), Subcontract Manager

Jen Carpenter, PhD (HSRI), Content Specialist Consultant

Matt Jameson (HSRI), Research Assistant

Adrienne Elefantis (MANILA Consulting Group), Writer

To begin the focus group session, Jen provided a brief overview of the MHT SIG evaluation project, the purpose of the instruments to be discussed in the focus group, how the instruments were developed, and how they will be used during the site visits. She noted that the purpose of the focus group was to give feedback on the quality of the questions, not to try to answer the questions; the instruments will be revised after the focus group using a more traditional pilot-testing procedure. She also noted the questions do not necessarily appear in the order they will appear in the final instruments.

Suggestions and observations made by the focus group participants during the remainder of the discussion are summarized below. The number above each bullet corresponds to the number of the item in question, as given on the October 12, 2007 draft.

Adult, Youth, and Family Member Screening Questions

1. Please indicate the categories that best describe you.

There may be respondents who are not currently involved in services. Add a category for former consumers. Respondents noted, after this exchange, that there were in fact two tabs above the categories to indicate “Current” and “Former.” However, one respondent noted that these were not sufficiently salient and should be made larger.

2. Have you heard of national, State, or local efforts to reform (i.e., transform) mental health services?

“Reform” and “transformation” may not mean the same thing to everyone.
Many people are more likely to have heard about big national issues, like what is happening in Virginia; the question needs to be more specific about “what grants, where.” Otherwise, you will get too many false positives.
Some respondents may not say they are aware of mental health service “reform” because they won’t identify it under that label (i.e., it’s a word with many connotations).
Is the point of this question to screen out those who have not heard of efforts, or just find out if the respondents have heard about efforts? The purpose of the question is unclear.

3. Do you feel comfortable commenting on the involvement of individuals receiving mental health services and family members in the mental health transformation grant?

Questions like these need to be more specific in explaining WHICH grant they are talking about. Respondents may not have the MHT SIG grant in mind. One respondent suggested deleting this question.

Adult, Youth, and Family Member Semi-Structured Focus Group Facilitation/Interview Guide

4. “….What will be the long-term impact of your involvement…”

“Long-term impact” is questionable. One’s own long-term impact may be hard for individuals to think about or conceptualize. Any involvement is itself an accomplishment.
Focusing on the person is important, as people often think of themselves to be inconsequential relative to the massive mental health system.
Lots of activities consumers do are “baby steps,” so it’s not necessarily easy to think about long-term impact.
As with the prior question, be more specific if you are intending to limit the scope of the question to this particular grant or be broader.

7. “Has involvement by mental health users and their families made a difference…”

When measuring level of involvement it is important to say, “since the beginning of this grant” and how is this different from before, from baseline? This is because in a lot of States, there is already a high level of involvement. The questions need to ask, has there been a material change since the grant began? Has the grant accelerated the existing conditions, or not made a difference?

8. “What three things get in the way of making reforms…..”

This question is broader than the grant, be clear about this.
It would be good to know if respondents saw the same limitations as the people who wrote the grant.
Lack of education (or more precisely, lack of money that would enable one to get a higher education) is a barrier to involvement for many consumers and family members.
Need to ask: What are the things you as a family member should have been involved in that you were not involved in?
This is actually two questions: should people have been involved who were not involved, and for those who were involved, what did/did not happen that they wanted to happen?

9. “What are the three things you would most like to see change….”

This question is broader than the grant; be clear about this.

5. “Please indicate the overall frequency and type of involvement…”

Make it clear that this question actually has two scales, frequency and type of role.
Where “diverse cultural groups” is used, does this mean ethnic, racial, and religious? Specify.
For the response category “Always, almost always,” these are strange to put together. “Regular role” may read better as “average role.” Participants suggested we delete this category, noting that roles are either significant or not, and that the “regular” or “average” tags are not useful.

6. “Please indicate the frequency of involvement of mental health services users and their families (AYF) in the following activities…”

What does “AYF” mean? (Jen explained that the acronym would not be used in the actual focus group.)

Adult, Youth, and Family Member Questionnaire

2. “More than half of the people involved identify themselves as mental health services users or family members.”

This is not a question to ask respondents. It’s a structural thing that has to do with the grant, so don’t waste respondents’ time on that.

7. “Members of the group I am in are able to work with each other on this project.”

Add “collaboratively” to this question.

8. State mental health transformation leaders are sensitive to cultural and linguistic issues.

Is this question trying to get at the quality of the person, or process issues? The questions seem to have mixed purposes: questions 12, 17, and 20, are more fact oriented but all the rest are process questions.

Other questions the focus group would be interested in adding:

Are you more hopeful now about the future of the mental health system?
Have you learned a lot?

Suggestions for new Likert scale items were:

Consumers and family members who participated in grant-funded transformation activities are excited about the progress of the grant.
Consumers and family members who participated in grant-funded transformation activities are more hopeful about the system’s future as a result of the T-SIG grants.
Consumers and family members who participated in grant-funded transformation activities learned about the policy-making and implementation processes as a result of their participation.

File Type	application/msword
File Title	DRAFT
Last Modified By	cblyler
File Modified	2007-12-17
File Created	2007-12-17

Form Attachments 1-13 Attachments 1-13 Attachments 1-13

The Mental Health Transformation State Incentive Grant Cross-Site Evaluation

Attachments-1-13

The Mental Health Transformation state Incentive Grant Cross-Site Evaluation

OMB: 0930-0292

Measures of Individual Recovery

Resilience Measures

Assessing Developmental Strengths Questionnaire (ADS)

Resiliency Canada, 2003

Behavioral & Emotional Rating Scale-2 (BERS-2)

Epstein, 1998

Child & Youth Resilience Measure (CYRM)

Ungar, 2003

Connor-Davidson Resilience Scale (CD-RISC)

Connor & Davidson, 2003

Developmental Assets Profile (DAP)

The Search Institute, 2004

Healthy Kids Resilience Assessment (HKRA)

Constantine, Bernard & Diaz (WestEd), 1998

Ohio Scales

Ohio Department of Mental Health, 1999

Resilience Scale

Wagnild & Young, 1999

Data Analysis

Mental Health Transformation State Incentive Grant ADULT CONSUMER INTERVIEW GUIDE

Mental Health Transformation

State Incentive Grant

BASELINE AND FOLLOW-UP

BASELINE DIAGNOSTIC REPORTING FORM

FOLLOW-UP DIAGNOSTIC REPORTING FORM

Mental Health Transformation State Incentive Grant

Mental Health Transformation

State Incentive Grant

CAREGIVER OF CHILD/YOUTH INTERVIEW GUIDE

BASELINE DIAGNOSTIC REPORTING FORM

FOLLOW-UP DIAGNOSTIC REPORTING FORM

Mental Health Transformation

State Incentive Grant

CHILD/YOUTH CONSUMER INTERVIEW GUIDE

BASELINE AND FOLLOW-UP

BASELINE DIAGNOSTIC REPORTING FORM

FOLLOW-UP DIAGNOSTIC REPORTING FORM

Resilience Measures and Name of Developer

Assessing Developmental Strengths Questionnaire (ADS), Resiliency Canada, 2003

Behavioral & Emotional Rating Scale-2 (BERS-2), Epstein, 1998

Child & Youth Resilience Measure (CYRM), Ungar, 2003

Connor-Davidson Resilience Scale (CD-RISC), Connor & Davidson, 2003

Developmental Assets Profile (DAP), The Search Institute, 2004

Healthy Kids Resilience Assessment (HKRA), Constantine, Bernard & Diaz (WestEd), 1998

Ohio Scales, Ohio Department of Mental Health, 1999

Resilience Scale, Wagnild & Young, 1999

Hawai’i AMHD

Mental Health Transformation

State Incentive Grant

ADULT CONSUMER INTERVIEW GUIDE

Mental Health Transformation

State Incentive Grant