Project Outcome Assesment Survey (POAS)

OMB Control No: 0970-0379

Expiration date: XX/XX/XXXX

Project Outcome Assessment Survey

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: The purpose of this information collection is to provide feedback to the Administration for Children and Families (ACF) regarding Administration for Native Americans grant data. Public reporting burden for this collection of information is estimated to average 6 hours per grantee, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a mandatory collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0379 and the expiration date is XX/XX/XXXX. If you have any comments on this collection of information, please contact Amy.Zukowski@acf.hhs.gov.

Identifying Information (Pre-Filled by Federal Staff)

Evaluator’s Name (First), (Last):

Other ANA Staff/Contractors Present at the Site Visit:

Evaluation Region: [DROP DOWN] East, West, Pacific, Alaska

State: [DROP DOWN] AK, AL, AR, AS, AZ, CA, CO, CT, DC, DE, FL, GA, GU, HI, IA, ID, IL, IN, KS, KY, LA, MA, MD, ME, MI, MN, MO, MP, MS, MT, NC, ND, NE, NH, NJ, NM, NV, NY, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VA, VT, WA, WI, WV, WY

Date of Evaluation:

Grant History (Pre-Filled by Federal Staff)

Received TA: OTA o ETA o (Check all that apply)

Yes o No o

Comments?

Received Supplements:

Yes o No o

Comments?

Received Carryovers:

Yes o No o

Comments?

Received an NCE/LCE:

Yes o No o

Comments?

Pre-Visit Information (Pre-Filled by Federal Staff)

Background Information/General Comments

DPO Site Visit Information (if applicable)

Grantee Information (Pre-Filled by Federal Staff)

Grantee Name:

Project Title:

Grant Number:

Address:

Geographic Designation:

Grant Category: [DROP DOWN MENU] Social Development, Economic Development, Governance, Environmental Regulatory Enhancement, Language Preservation, Esther Martinez, ILEAD, NLCC

Grant Sub-Category:

Original Project Duration:

Total Federal Funding Amount:

Project Goal:

Problem Statement:

1. Objective Work Plan (Pre-Filled by Federal Staff)

1.1 Objective 1

Relevant Year(s); o Yr1 o Yr2 o Yr3 o Yr4 o Yr5

Percent Complete: 0, 25, 50, 75, 100, 100+%

Objective

Outputs Outcomes Expected:

Criteria for Evaluating Outputs and Outcomes Expected:

Deliverables (Developer note: add tags for deliverable type):

Product	Documentation Reviewed/Comments

Comments

1.1 Objective 2

Relevant Year(s); o Yr1 o Yr2 o Yr3 o Yr4 o Yr5

Percent Complete: 0, 25, 50, 75, 100, 100+%

Objective

Outputs/Outcomes Expected:

Criteria for Evaluating Outputs and Outcomes Expected:

Deliverables (Developer note: add tags for deliverable type):

Product	Documentation Reviewed/Comments

Comments

1.1 Objective 3

Relevant Year(s); o Yr1 o Yr2 o Yr3 o Yr4 o Yr5

Percent Complete: 0, 25, 50, 75, 100, 100+%

Objective

Outputs/ Outcomes Expected:

Criteria for Evaluating Outputs and Outcomes Expected:

Deliverables (Developer note: add tags for deliverable type):

Product	Documentation Reviewed/Comments

Comments

2. Outputs and Outcomes

2.1 Describe the community the project was designed to serve:

2.1.1 Number of people the project was designed to serve_____

Number of people the project actually served_____

ANA Evaluator pre-populate from OPR and then enter in actuals on-site

	Indicator(s)	Baseline	Year 1	Year 2	Year 3	Year 4	End of Project	3-Year Post Project	Means of Measurement
From OPR
Actuals

2.1.2 Is the End of Project Target still achievable and realistic?

2.1.3 Now that you are at the end of your project, is your 3-Year Post of Project Target still achievable realistic?

If not, how would you revise?

2.2 Describe how your indicator was achieved throughout the project.

2.3 Was the indicator from the application the best measurement to capture the outcomes of your project?

2.3.1 If not how would you have identified or revised the indicator?

2.4 Name up to three achievements you are most proud of and would want to share about your project?

1)

2)

3)

2.5 What project activities would you like to sustain after ANA funds end, and what is needed to sustain such outcomes?

2.6 What obstacles are foreseen to project sustainability over the next 5 years?

2.7 As a result of this project is/will it be a(n) (Please simply highlight the appropriate answer):

• New Initiative

• Continuing Program

• Merged into an existing program

• Intended to End (e.g., Manual/Regulation Development)

• Will not be sustained

2.8 How do you intend to financially continue or extend desired project outcomes, services, and activities after this ANA-funded project ends? Please check all that apply.

N/A (additional funds not required) Program income

State/Territory funds Tribal funds

Other federal grant funds Fundraising campaigns

Private foundation grants Colleges/Universities

Corporate Donations Other:      

2.9 If applicable, what level of funding do you currently have in place to continue or extend project benefits?

Desired funding levels sufficient to continue or extend project benefits are in place

Some, but not all needed funding has been secured to continue or extend project benefits

Please describe any plans or efforts being made to secure additional resources to sustain desired project outcomes, services, and activities:      

Still seeking needed funding, none currently in place

Please describe any plans or efforts being made to secure additional resources to sustain desired project outcomes, services, and activities:      

No funding secured to continue or extend project benefits and no plan in place through which to obtain funds

Not applicable because there are no current plans to continue or extend project benefits

2.10 Aside from funding, what will contribute to the sustainability of this project?

Contributions	Comments

2.11 Was any infrastructure created?

Yes o No o N/A o

Physical (utilities, IT capabilities, facilities improvements, etc.)

Organization Capacity (skills, personnel, infrastructure)

2.12 FOR NON-LANGUAGE PROJECTS: Were Native languages incorporated into the project in any way?

If so, how?

2.13 How were your community’s Indigenous knowledge (cultural, linguistic, ecological, etc.) incorporated into the project?

2.14 Did your project directly or indirectly address culture as a protective or preventative factor against suicide, substance abuse, health issues, poverty, low educational achievement, lack of self-identity, shaming of cultural identity, dysfunctional families, trauma, physical health issues, and/or violence in Native communities?

• Directly as part of the project goal and/or objectives

• Indirectly

• Not at all

2.14.1 If yes, how did your project directly or indirectly address culture as a protective or preventative factor against these or other adverse experiences?

3. Project Development

3.1 How was the project developed? Please describe how the community was involved in the development process.

3.2 Was the project’s target population involved during project planning prior to the grant award?

Yes o No o N/A o

3.3 Is there anything you would have done differently in planning your project?

3.4 What marketing or outreach methods did you use to bring attention to your project? (Check any that apply and please simply highlight the appropriate answer.)

o Brochures and Pamphlets

o Email

o Festivals and Community Events

o Networking (with Other Tribes, Agencies, Organizations, Businesses)

o Newspaper Advertisements

o Newspaper Coverage

o Onsite Community Outreach (knock on doors)

o Phone Calls

o Posting Information in Public Spaces

o Radio (includes PSAs and other programming)

o Regular Mail

o Social Media (such as Facebook, Twitter, Youtube, etc.);

o Tribal/Community Meetings

o Tribal/Organizational Publications or Newsletters

o TV (includes PSAs and other programming)

o Website

o Word of Mouth

Other

3.5 In terms of marketing/outreach, what would you have done differently?

3.6 What were the best practices in marketing to your specific population?

Target Populations	Best Practice

3.7 How was the project received by the community?

3.8 Was there opposition to the project?

Yes o No o

Comments?

3.9 How did you keep the community engaged or updated throughout the project?

3.10 How did you encourage project participants to maintain their participation in the project? Were there external factors that also helped?

3.11 How did you facilitate multigenerational activities and what were the results?

3.11.2 What are some best practices in facilitating multigenerational activities?

4. Project Challenges

4.1 Did you have any challenges that affected your ability to implement the project?

Rank	Challenge	Ability to Overcome Challenge?	Comments
	Staff turnover	
	Late start due to ANA notification	
	Late start due to hiring delays	
	Scope too ambitious	
	Geographic isolation/travel issues	
	Lack of expertise	
	Challenges with ANA processes	
	Challenges with Office of Grants Management processes	
	Underestimated project cost	
	Underestimated personnel needs	
	Partnership fell through	
	Lack of community support (planning)	
	Lack of community support (implementation)	
	Insufficient Community Infrastructure	
	Personal or Community Loss/Hardship	
	Natural Disaster	
	Lack of Organizational Capacity	
	COVID-19
	Other	

5. Project Economic Benefits

5.1 Did this project support native-owned businesses (Please simply highlight the appropriate answer)?

Yes o No o N/A o

5.1.1 If so, how many?

5.2 Of the Native-owned businesses the project supported, what were the top three types of businesses by dollar amount?

Business Name	Type (eval instruction: use a one or two word descriptor)

5.3 Were any businesses created by the project (Please simply highlight the appropriate answer)?

Yes o No o N/A o

Business Name	Type (eval instruction: use a one or two word descriptor)	Ownership Type Native, Women, Vet

5.3.1 Was income generated by businesses (developer note: only show if yes is selected in 5.3)

5.4 Were any businesses expanded?

Yes o No o N/A o

5.4.1 If yes, how were they expanded?

5.5 List income generated by the project: How was income generated?

EMPLOYMENT TABLE

Complete for any person employed as a result of the project (do not include project staff).

 

FROM THE OPR

Name of Individual	Position Title	Position Type	Industry (Drop down)	Filled by?	Date Job Filled	Total. Hours Worked	Weeks employed	Date Job Ended	Did position exist before the project?	Will position continue after the project ends?
		Full time, Part time, Intern, Stipend, Consultant/Freelance, other	Education, Construction, Government, Retail, Tourism, etc.	Native, Veteran, Female

5.6 Here is a listing of the jobs you listed in the OPR. Is this accurate, and if not, please note additional positions obtained as a result of the project activities.

5.7 Total Jobs Obtained as a Result of the Project Activities.

6. Partnerships

6.1 What role did partnerships play in your project?

[DROP DOWN] Integral, Supportive, Minor, Needed more partners

Comment:

6.2 As your project funding is ending, what role (if any) are partnerships playing in continuation of the project activities?

7. Program Specific (Grantees will only be asked questions from this section pertaining to their program).

Environmental Regulatory Enhancement (ERE)

7.1 What was the project type?

Forest Pollution Botany

Water Marine Wildlife

Air Land Other:

What was the main focus of your project? (Check all that apply)

Data collection to establish baseline Environmental Education

Data collection to monitor condition or trend Staff Training/Development

Develop/implement management systems Obtain additional federal funding

(GIS, enforcement) Obtain EPA IGAP Funding

Develop Codes/Regulations Other

Develop Tribal Environmental Protection Act

7.1.2 Can you share how your project incorporated traditional ecological knowledge and traditional knowledge?

7.2 Did you use EPA or other federal funds to meet your non-federal share for this grant?

Yes: EPA

Yes: Other

No

Was not aware this was allowable

7.3 Does your Tribe/organization receive funding from EPA, US Forest Service, or US Fish and Wildlife?

EPA

US Forest Service

US Fish and Wildlife Service

No

Unknown

7.3.1 If yes, what is it used for? How does your ANA ERE grant fit in with this funding?

Native Language Preservation (P&M) and Esther Martinez Immersion (EMI)

7.1 How did you determine the means of measurement used to measure the increase in proficiency and/or fluency? (note to evaluator: see ADR #s 6-9)

7.2 Status of language prior to the project (Note to Evaluator: Pre-fill from application)

7.2.1 Describe the current status of the language/activities (evaluator note: elaborate to note that significant changes are not going to occur within a discrete timeframe of a project, but what has improved over the course of the project)

7.3. Please provide the:

Language Family:

Language:

Dialect:

7.4 Has there been any space/place that language use has increased since the grant began? Can you rate on a scale of 1 to 5 (1 is minimal use and 5 is maximum use), language use in the following spaces:

	Prior to the Project	End of Project	How has it increased?
Community
In the Home
Schools
Tribal Government
Virtual Community

7.5 Were there any external positive effects that have been noted in the community outside of the project?

7.5 Promising practices: Are there strategies, promising practices, or resources the project would suggest that helped during implementation? (Pick the top 3 you have promising practices for from the table below)

Aspect of ILEAD Project Implementation	Effective strategies, promising practices, or helpful resources
Adapting to differing age groups or learning abilities
Discussing difficult topics or sensitive conversations with youth
Administering college readiness, workforce development, or other skill trainings
Tracking youth development over multiple years of participation
Frequency of meeting with youth (monthly, weekly, daily)
Other

7.6 What was your experience working with the ILEAD Cohort learning community such as participating in the monthly ILEAD calls, youth summit, etc.?

7.7 Over the course of the project, how many unduplicated youth participated in a project funded activity or service?

Project Year	Number of Unduplicated Youth Participants
Year 1
Year 2
Year 3
Year 4 (if applicable)
Year 5 (if applicable)
Project Total

7.8 Were there any services provided through the project that supported youth if they encountered wellness challenges or difficult situations?

Questions for Youth Beneficiaries:

7.9 How did the project change your life?

7.9.1 How did the project strengthen your sense of:

• Self-confidence:

• Cultural identity:

• Other:

7.9.2 Did being involved in the project increase interest in engaging in your community such as volunteering, taking on leadership roles, or other ways?

• Yes/No

• If yes, how?

7.9.3 Did the ILEAD grant better prepare you to enter college, the workforce, or other leadership roles?

Yes/No

If yes, how?

7.9.4 Did the project build any additional skills not mentioned above? If so, how?

7.9.5 What did you like most about your involvement with the project?

Economic Development

7.1 Did this project support community economic development by:

• The development of tribal commercial codes

• Establishing or improving courts system(s)

• Training for court personnel (note to Evaluator: see Section K of the OPR)

• Improving access to Native CDFIs

• Develop a Tribal Community or Economic Development Master plan

7.2 Tribal Court Systems

7.2.1 Did the project increase capacity of tribal courts to resolve commercial and financial disputes and to enforce commercial code?

7.2.2 Did the project promote increased independence of a tribal Judiciary from the Executive or Legislative branches?

7.2.3 Were court personnel trained on tribal commercial laws and/or regulations?

7.2.4 Was any technical assistance provided to non-native businesses, lenders, or government officials on tribal sovereignty and economic development?

• What agencies or organizations?

7.3 Community Development Financial Institutions (CDFIs)

7.3.1 Did you create a new CDFI with this funding or work towards the development of a Native CDFI?

7.3.2 Are you a Certified Native CDFI, or a part of the Native CDFIs?

7.3.3 What type of CDFI are you?

• Community Development Bank

• Community Development Credit Union

• Community Development Loan Fund

• Community Development Venture Capital Fund

7.3.4 What were the primary focus areas of the CDFI funded work?

• IDA match

• IDA match business creation

• IDA match home buying

• IDA match higher education and schooling

• Private business loans and technical assistance

• Affordable Housing/mortgage lending

• Other?

7.3.5 What barriers are there to lending in your community?

• How did the project improve access to loan funds?

• What other improvements has the CDFI contributed to in your community?

7.3.6 What barriers to accessing capital exist in your community that prevent Native businesses from starting or expanding?

• Has this project helped to overcome any of those barriers and if/so how?

7.3.7 Did the project incorporate cultural practices/health/mental wellness/education/childcare, etc. in improving access to capital?

• If so, how?

7.3.8 If part of your ANA funded objectives, how many new savings accounts were opened during the project?

7.3.9 If part of your ANA funded objectives, please address Wealth Creation:

• # of new mortgages

• $ of new mortgages

• # of new credit builder loans

• $ of new credit builder loans

7.3.10 If part of your ANA funded objectives, please describe the Increase knowledge of budgeting and saving of project participants.

7.3.11 Did the CDFI provide Technical Assistance/Financial Literacy to community members?

• Did you use a culture-based approach to financial learning?

7.3.12 Was training of CDFI employees a part of this project? (note to Evaluator: see Section K of the OPR and ensure all trainings and certifications are captured and complete table if necessary and ask grant recipient to complete Section K in the OPR)

Name of Training or Workshop	Description of Skills or Knowledge Developed; and/or Certificate Received	Total # of People Who Completed the Training	# of People who were Project staff	# of New People who Completed the Training	Total # of Hours to Complete the Training	Did this training lead to a new job or promotion?	If, yes, how many?

• How Many?

7.3.13 Were any administrative improvements made, if so, what?

7.4 Master Plans

7.4.1 Did this project work to develop a long range master plan to increase economic or community development?

• Economic

• Community

7.4.2 What changes were made to Tribal or Organizational policies or master plans to attract economic development as a result of this project?

• Increased access to Financial Institutions

• Legal/Regulatory enhancements to attract Economic Development

• Policy or capacity Improvements

  • Overall Infrastructure planning – community (roads, building improvements, etc.) or household (access to electricity, water, natural gas, etc.)

  • Natural Resources

  • IT infrastructure

  • Emergency Services

  • Community engagement supporting Economic Development

• Increased Capacity to leverage taxes

• Development of a Business Incubator

• Development of a nonprofit subsidiary or other Tribal business structures

8. Staffing

8.1 Number of Program Specialists during Project Period:

8.1.1 If more than one, how did this effect your project?

Comments:

8.2 How many Principal Investigators/Project Directors served on this project?

8.2.1 If there was turn-over, did the organization have a succession plan to transfer knowledge of the project and completed activities to the next PI/PD (Please simply highlight the appropriate answer)?

Yes o No o

Comments:

8.3 Post Award Training

8.3.1 Did the Project Director/Principal Investigator receive post-award training (Please simply highlight the appropriate answer)?

Yes o No o

If no, why not?

8.3.2 Did the financial staff member receive post-award training (Please simply highlight the appropriate answer)?

Yes o No o

If no, why not?

8.3.3 If there was turn-over, did the organization have a succession plan to transfer knowledge of the project and completed activities to the next PI/PD (Please simply highlight the appropriate answer)?

8.4 Does your organization have an IT/Data Management Department?

Yes o No o

ANA wants to support data management among its grantees.

8.5 How did your project use project information (data) to improve implementation of the project?      

8.6 How did you measure results or benefits occurring as a result of the project?      

8.7 Did you obtain feedback from community members to get their viewpoints on project progress? Yes No

8.7.a. If yes, how did you use such feedback?      

8.8 How would you characterize the availability of financial resources, including personnel availability, for data usage and management related to your ANA project:

(Scale: 1-10 from Extremely Inadequate to Extremely Adequate)

8.8.1 Did you hire a Data Manager/Analyst?

8.9 Among staff that work on this project, please indicate the highest level of knowledge in the following areas:

(Scale: (5) Extremely Knowledgeable, Moderately Knowledgeable, Slightly Knowledgeable, Use External Expertise, Not Part of Regular Project Business (1))

1. Performance Indicators

2. Descriptive statistics

3. Program Evaluation

4. Data visualization

5. Data programming

6. Data management

(what software used to manage data)

8.10 Through this project, has your organization/tribe increased their ability to collect, manage, or analyze data in any way?

8.10.1 Did staff participate in any ANA training or TA related to data?

8.11 What are the primary barriers to data management and analysis that limit the ability of your project to use data to inform key policy and programmatic issues?

8.12 How many staff are employed by the tribe or organization? 1-10, 11-20, 21-50, 50-100, 100-200, 200-300, 300-400, 400-500, 500+

8.12.1 Did you have all of the staff available to carry out your project?

Yes o No o

9. Beneficiaries

(Evaluator: use this section to capture information collected during interaction with beneficiaries/community members/partners including indirectly. If necessary ask grantee to identify beneficiaries and discuss impact.)

Beneficiary Name	Beneficiary Type: Tribal Leader; Parent; Student; Teacher; Project Staff; Elders; Community Member	Individual or Group
Beneficiary’s impact from project participation:
What made you want to participate in the project?
Were there any challenges you faced in participating in the project?
What was your favorite aspect of participating in the project?
What changes have you seen in your life or the community related to the project?

10. IMPACTS OF THE COVID-19 PANDEMIC

10.1 How has use of the ACF/ANA flexibilities been utilized to ensure project implementation? If so, what specific flexibilities?

10.2 Did COVID affect any of the following? If so, how did your project overcome any challenges or

obstacles? (See table below)

COVID Challenge	How specifically affected project	Strategies to overcome challenges
Adjusting travel logistics
Problems receiving resources pertinent to the project
Impacts to budget (COB, Budget Modification, NCE?)
Moving to remote learning or activities
Staffing
Working with or loss of partners or impact on developing MOU’s
Working with higher risk populations such as Elders
Other

10.3 Were there any innovative strategies to increase participation in virtual activities due to COVID? Please explain.

Yes o No o