Annual Report

OMB Control No.: 0970-0409 Expiration Date: TBD

U.S. Department of Health and Human Services

Administration for Children and Families

Office of Child Care

Tribal Maternal, Infant, and Early Childhood

Home Visiting Program

Guidance for Submitting an

Annual or Final Report to the Secretary

U.S. Department of Health and Human Services

Administration for Children and Families

Office of Child Care

370 L’Enfant Promenade SW, 5^th Floor East

Washington, DC 20447

Tribal Maternal, Infant, and Early Childhood Home Visiting Program

Background Information:

Guidance for Submitting an Annual or Final Report to the Secretary

Section 511(e)(8)(A) of the Social Security Act, as added by Section 2951 of the Affordable Care Act and amended by the Protecting Access to Medicare Act of 2014 and the Medicare Access and CHIP Reauthorization Act of 2015, requires that grantees under the Maternal, Infant, and Early Childhood Home Visiting (MIECHV) program for States and Jurisdictions submit an annual report to the Secretary of Health and Human Services regarding the program and activities carried out under the program, including such data and information as the Secretary shall require. Section 511 (h)(2)(A) further states that the requirements for the MIECHV grants to Tribes, Tribal Organizations, and Urban Indian Organizations are to be consistent, to the greatest extent practicable, with the requirements for grantees under the MIECHV program for States and Jurisdictions. Tribal MIECHV grantees have been notified by the Administration for Children and Families (ACF) and the Health Resources and Services Administration (HRSA) that in each year of their grant, except the first year, they must comply with the requirement for submitting an Annual Report to the Secretary that should describe activities carried out under the program during the past reporting period, or, in the case of then final report, throughout their project period.

This document provides guidance to grantees when submitting their annual or final report to the Secretary. The guidance is divided into two parts, one for submitting an annual report, the second for submitting a final report. Reports shall be submitted via GrantSolutions.gov per instructions given to each grantee by ACF as part of their cooperative agreement.

Any questions and comments regarding this guidance may be addressed to:

Anne Bergan

Senior Policy Analyst for Early Childhood Development

Office of the Deputy Assistant Secretary for Early Childhood Development

Administration for Children and Families, HHS

901 D St SW

Washington, DC 20447

anne.bergan@acf.hhs.gov

(202) 260-8515

Guidance for Submitting an Annual Report to the Secretary

Each year following the first year of the grant, Tribal Maternal, Infant, and Early Childhood Home Visiting grantees must provide a written report to the Secretary regarding the program and activities carried out under their cooperative agreement during the previous reporting period. Below are six Sections that represent the primary grant activities on which to report.

A reminder:

The audience for this report is the Secretary of Health and Human Services. As such, it is critical that you provide a comprehensive, well-written response that is responsive to each of the items below. Your report should tell the story of your program, your community, and the people you serve, providing valuable information to HHS regarding your assessment of challenges, successes, and lessons learned.

Section 1: Update on Home Visiting Program Goals and Objectives

Please discuss how you are meeting the goals and objectives you set out to accomplish when you developed your Implementation Plan.

• List each program goal and objective identified in your Implementation Plan and describe the progress made towards achieving each goal and objective during the most recent reporting period.

• If you have met any of your goals and objectives, please describe how you achieved them.

• For each goal and objective, if you encountered any challenges, describe barriers to progress that you encountered and the strategies and steps you have taken to overcome them.

Section 2: Update on the Implementation of Home Visiting Program in Targeted Community(ies)

Please discuss the planning and implementing of your home visiting program, as described in your Implementation Plan. Please address each of the items listed below. Where applicable, discuss any barriers or challenges you encountered and the steps taken to overcome these barriers and challenges.

• Please describe your progress in engaging with the broader community(ies) (e.g., partner agencies, stakeholders within your community, tribal leadership, leadership within your organization, partner agencies, the families you serve) around your home visiting program.

• Please describe your work to-date with the developer of the implemented home visiting model(s), including:

  • A description of any technical assistance and support provided during the reporting period from the model(s)

  • A description of your use or access to curriculum and other materials needed to implement the home visiting model and whether these materials were useful.

  • A description of any training and professional development supports you may have obtained from the home visiting model(s) developer and whether the training helped when implementing your program.

• Please describe your efforts to provide support and professional development to staff. Describe activities implemented over the past year and outcomes related to:

  • Staff recruitment and hiring

  • Professional development

  • Staff retention

  • Trainings to ensure well-trained, competent staff beyond model developer provided activities

  • High quality reflective, clinical, and administrative supervision

• Please describe your efforts to recruit, engage, and retain program participants, including progress on meeting objectives related to :

  • Recruitment of families

  • Engaging families

  • Retention of families

  • Participant completion of the program

• Please describe your activities related to providing high quality home visiting program services, including:

  • Ensuring the selected home visiting model(s) are being implemented with fidelity

  • Making sure all staff delivers home visits the way the model was intended

  • Developing and documenting program operations, policies, and procedures to support implementation fidelity and program quality

• Please describe steps taken toward the development of a coordinated early childhood system, including coordination and collaboration between your home visiting program and other programs and resources for pregnant women, expectant fathers, young children, and families in the community(ies).

  • Describe the actions you have taken to improve referrals and services between your program and others in the community.

  • Describe the partnerships you have developed and provide any updates on these collaborations.

• If applicable, please provide any testimonials, success stories, photos, and videos you would like to share regarding the implementation of your home visiting program.

Technical Assistance Needs and Outcomes

• Please describe any barriers or challenges you experienced related to the topics above during the reporting period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• Were your technical assistance and training needs met? If not, what additional supports would be helpful to you?

Section 3: Progress toward Meeting Legislatively Mandated Benchmark Requirements

Please provide a narrative discussing the benchmark data collection efforts, including a summary of barriers/challenges encountered during data collection efforts and steps taken to overcome them. Please address each of the items listed below:

• Please describe any approved or pending changes made to your benchmark plan, in consultation with your FPO and technical assistance providers, during the reporting period. If any changes were made, please summarize and note the reason(s) why.

• Please describe your program’s process for monitoring data collection and entry and describe your team’s progress towards collecting complete and high-quality benchmark data. These activities could include refresher trainings for home visitors on data collection, regular data quality checks, and reviewing data during staff meetings.

• Please provide an update on the number and percentage of program participants for whom you have collected high-quality benchmark data across the various benchmark constructs.

  • For any particular constructs for which there is a considerable amount (i.e., 50 percent or more) of missing data, please describe, explain, and discuss steps your program has taken to improve the completeness and accuracy of data collection.

• Please discuss how you are using benchmark data your program collects to monitor and assess the quality of program implementation and service delivery.

• Please describe any successes your team has experienced related to the collection, analysis, federal reporting, and use of benchmark data during the reporting period.

• Please describe any challenges your team has experienced with the benchmark data collection and reporting during the reporting period. How have you addressed these challenges?

Technical Assistance Needs and Outcomes

• Please describe any barriers or challenges you experienced related to the topics above during the reporting period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• Were your technical assistance and training needs met? If not, what additional supports would be helpful to you?

Section 4: Update on Rigorous Evaluation Activities

Please provide an update on rigorous evaluation activities, and report on the following:

• Please list your evaluation questions. Please include your primary PICO question and any additional evaluation questions you are posing.

• Please describe the team’s progress conducting the evaluation and any evaluation activities that took place during the reporting period. Please note the following (if applicable):

• Progress developing the evaluation plan (if not already approved)

• Progress submitting to the IRB or updating the IRB protocol

• The number of participants to be enrolled in the evaluation, including both the treatment and comparison groups if applicable, and progress towards enrolling your targeted sample

• Data collected, including the number of participants for whom data were collected, the response rate, and the type of data collected/measures used

• Progress conducting analyses of data collected

• Activities conducted to engage and inform community members of the evaluation’s progress

• Please describe any approved or pending changes made to your evaluation plan, in consultation with your FPO and technical assistance providers, during the reporting period. If any changes were made, please summarize and note the reason(s) why. For example,

• Have there been any changes in proposed evaluation design (e.g., experimental/random assignment, quasi-experimental/comparison study, pre-post, longitudinal/historical analysis, etc.).

• Have there been any changes in the participants you propose to include in the evaluation, or the number of participants you intend to include?

• Have there been any changes in your proposed data collection procedures (e.g., instruments, data collectors, timing, incentives, etc.)?

• Please list key evaluation successes or accomplishments during this reporting period. Accomplishments may be related to: 1) Design (planning and coordination with the evaluation team and program staff), and/or 2) Implementation (launch of evaluation activities, including data collection and preliminary reporting).

• Please describe any challenges to the evaluation experienced during this reporting period. How have you addressed these challenges? Did these challenges affect the program or evaluation plan?

• Please describe any interim evaluation findings.

• Please list any evaluation dissemination activities conducted during the reporting period. Evaluation dissemination activities could include reports or presentations provided to program stakeholders and community members; presentations at research, evaluation, or practitioner conferences; and work on articles for scholarly journals or other publications.

Technical Assistance Needs and Outcomes

• Please describe any barriers or challenges you experienced related to the topics above during the reporting period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• Were your technical assistance and training needs met? If not, what additional supports would be helpful to you?

Section 5: Home Visiting Program Continuous Quality Improvement (CQI) Efforts

Please provide an update on your efforts regarding planning and implementing continuous quality improvements or activities during the reporting period, including CQI opportunities, changes implemented, data collected, and results obtained. If applicable, please provide copies of CQI reports developed. Please complete the following table for any CQI projects (i.e., Plan-Do-Study-Act cycles) conducted during the reporting period. If you wish, you may present information using a CQI storyboard format instead.

Problem Statement	Aim Statement (Final)	Improvement Theory(ies)	Data Used (list exact measures)	Outcome (before and after the CQI project)

• Please discuss any CQI activities or projects that took place during the reporting period. CQI activities could include PDSA cycles, CQI team meetings, use of CQI tools, implementation of tests or solutions, collection and analysis of data, sharing of results, etc.

• Please describe the team’s progress on your CQI project(s), as outlined in the table above or in a CQI storyboard format.

• Please describe any CQI accomplishments experienced by your team during the reporting period. For example, did the tests you implemented result in any improvements?

• Please describe any challenges encountered during the CQI process. How did you address these challenges?

• What has your team learned in your CQI projects during the reporting period? Please provide lessons learned resulting from your CQI project(s).

• What are your team’s next steps in your CQI work?

Technical Assistance Needs and Outcomes

• Please describe any barriers or challenges you experienced related to the topics above during the reporting period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• Were your technical assistance and training needs met? If not, what additional supports would be helpful to you?

Section 6: Dissemination

Please describe your program’s dissemination plan and efforts over the reporting period.

• What are your dissemination purposes and goals? Which of these goals did you prioritize this year?

• Who are your target audiences for dissemination and information sharing? Which audiences did you disseminate to in the past year? Target audiences could include tribal council, the community, families, and funders (local, state, federal, foundations) or others.

• What types of information (qualitative, quantitative) did you prioritize for dissemination during the reporting period?

• What products and materials did you develop and what methods did you use to share them? Please list these and identify which audiences you targeted and how they were disseminated. For instance, did you give a presentation for tribal council? Did you create brochures or marketing materials to recruit families?

  • Relevant dissemination materials and products could include:

    • Newsletters (electronic or print)

    • Pamphlets, brochures, or fact sheets

    • Web- and social media-based products (blogs, podcasts, video clips, etc.)

    • Digital stories or videos

    • Presentations and posters

    • Press communications (TV/radio interviews, newspaper interviews, public service announcements, and editorial articles)

    • Electronic products (DVDs, audio or videos, CD-ROMs)

    • Reports and monographs (including policy briefs and best practices reports)

    • Books and book chapters

    • Peer-reviewed publications in scholarly journals – published and submitted

    • Academic course development and distance learning modules

    • Doctoral dissertations/Master’s theses

• What were the outcomes of your dissemination efforts? Did they achieve the goals and purposes you intended? Please describe any notable successes or challenges.

• Please provide copies of or links to reports or materials developed for dissemination during the reporting period.

Technical Assistance Needs and Outcomes

• Please describe any barriers or challenges you experienced related to the topics above during the reporting period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• Were your technical assistance and training needs met? If not, what additional supports would be helpful to you?

Guidance for Submitting a Final Report to the Secretary

In the final year of the grant, Tribal Maternal, Infant, and Early Childhood Home Visiting grantees must provide a written report to the Secretary regarding the program and activities carried out under their cooperative agreement during the entire project period. Below are six Sections that represent the primary grant activities on which to report.

A reminder:

The audience for this report is the Secretary of Health and Human Services. As such, it is critical that you provide a comprehensive, well-written response that is responsive to each of the items below. Your report should tell the story of your program, your community, and the people you serve, providing valuable information to HHS regarding your assessment of challenges, successes, and lessons learned.

Section 1: Report on Home Visiting Program Goals and Objectives

Please describe how you met the goals and objectives you set out to accomplish when you developed your Implementation Plan.

• List each program goal and objective identified in your Implementation Plan and provide a final summary of the progress made towards achieving each goal and objective during the grant period. Please describe how each goal and objective was accomplished. Describe barriers to achievement, or challenges you faced, and the strategies used to overcome them. If a goal or objective was revised or not accomplished, please explain why.

Section 2: Report on the Implementation of Home Visiting Program in Targeted Community(ies)

Please provide final information and reflections regarding the planning and implementation of your home visiting program, as identified in your Implementation Plan. Please address each of the items listed below. Where applicable, discuss any barriers or challenges you encountered and the steps taken to overcome these barriers and challenges.

• Thinking back to when you conducted your needs assessment for your home visiting program, and the years of implementation afterwards, please provide a summary that includes:

  • Was conducting a needs assessment useful? If so how? If the needs assessment was not useful, why do you think that is?

  • What do you see as your biggest successes related to the needs assessment and its use to support and shape program implementation?

  • What do you see as your biggest challenges related to the needs assessment and its use to support and shape program implementation?

• What lessons have you learned that may serve you in implementing similar programs moving forward?

• Please describe lessons learned around engaging with the broader community(ies) (e.g., partner agencies, stakeholders within your community, tribal leadership, leadership within your organization, partner agencies, the families you serve) around your home visiting program. How have these partners’ involvement helped or hindered the implementation of your program?

• Please describe how your community’s involvement will help sustain your program.

• Please report on the work with the developer of the home visiting model(s) during the project period, including:

  • A summary of any technical assistance and support, curriculum and materials, and professional development received from the model(s). How did the support of the model developer(s) impact the implementation of your home visiting program?

  • If the model developer(s) did not provide the support or involvement you needed, please describe the challenges you encountered and how you think that may have impacted the implementation of your program.

• Please summarize your efforts to provide support and professional development to staff. Describe activities implemented over the project period and outcomes related to:

  • Staff recruitment and hiring (if the ending of your grant has impacted retaining staff please describe)

  • Professional development

  • Staff retention (if the ending of your grant has impacted retaining staff please describe)

  • Trainings to ensure well-trained, competent staff beyond what the model developer provided and any other activities during the life of this grant

  • High quality reflective, clinical, and administrative supervision

• Please describe your efforts to recruit, engage, and retain program participants, including progress on meeting objectives related to the following. Please provide summary data about family recruitment, retention, and program completion over the project period.

  • Recruitment of families

  • Engaging families

  • Retention of families

  • Participant completion of the program

  • If you did not meet your objectives, please provide reflections on the possible reasons why this may have occurred and the lessons learned that you can use when thinking about the future.

• Please summarize your activities related to maintaining quality and fidelity of the home visiting program. How did you:

  • Ensuring the selected home visiting model(s) are being implemented with fidelity

  • Making sure all staff delivers home visits the way the model was intended

  • Developing and documenting program operations, policies, and procedures to support implementation fidelity and program quality

• Please describe the actions you took throughout the project period related to developing a coordinated early childhood system that includes home visiting programs, including coordination and collaboration between your home visiting program and other existing programs and resources for pregnant women, expectant fathers, young children, and families in the community(ies).

  • Describe the actions you took throughout the project to maximize referrals and services between your program and others in the community.

  • Describe the partnerships you developed and provide any updates on these collaborations. How you will continue to partner with them beyond this grant?

Technical Assistance Feedback

It would be very helpful to ACF if you could offer feedback on the quality of the technical assistance you received.

• Please describe any barriers or challenges you experienced related to the topics above during the project period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• What methods of technical assistance were most useful (i.e. targeted TA, webinars, site visits, meetings, etc.)? What methods were least useful?

• What additional supports would have been helpful?

Section 3: Report on Benchmark Data Development, Collection and Reporting

The MIECHV legislation for tribal grantees had strong requirements for grantees to create benchmark plans, to collect data on benchmarks and for reporting. The purpose of this section is to learn about grantees’ experiences with the benchmark process.

• Please describe your team’s experience with performance measurement before receiving the home visiting grant.

• How did your organization engage in data collection prior to receiving the home visiting grant? If applicable describe any prior data collection protocols and databases.

• Was data collection a completely new process to your home visiting staff, or had they done some in the past?

• How comfortable did you expect families to be with data collection?

• Please describe what you learned from going through the entire benchmark data collection, analysis, and reporting process.

• What did you learn in developing the benchmarks plan?

• What did you learn in collecting benchmark data?

  • How comfortable was home visiting staff with data collection?

  • How comfortable were families?

• What did you learn through the reporting process?

• If you had it to do all over again, what would you do differently in terms of your local benchmark data collection, analysis, and reporting process?

• In looking at your Home Visiting Form 3 data, what do you think were your program’s biggest achievements?

• Which of your performance measures best capture these achievements?

• What did you learn about your program achievements that surprised you?

• Were there any achievements that were not well represented in the data? If so, what were they?

• What data would your program want to collect that you did not collect? What would this new data help to inform?

• Which aspects of your Home Visiting Form 3 data indicated a need for improvement from your perspective?

• In what ways do you think that data accurately reflect your program’s implementation or data collection challenges in this area?

• Please describe any challenges that made improvement in this area particularly difficult.

• Were there challenges related to data collection itself?

  • How do you plan to overcome these challenges in the future? (data collection burden; supervision or support for data collectors; making data collection more meaningful for families; better systems)

• How will you use these data to inform your program implementation in this area?

• Now that you have reported your program’s performance data:

• What would you recommend HHS do to support effective performance measurement by grantees in the future?

• What would you change about the performance measurement requirements?

Technical Assistance Feedback

It would be very helpful to ACF if you could offer feedback on the quality of the technical assistance you received.

• Please describe any barriers or challenges you experienced related to the topics above during the project period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• What methods of technical assistance were most useful (i.e. targeted TA, webinars, site visits, meetings, etc.)? What methods were least useful?

• What additional supports would have been helpful?

Section 4: Report on Rigorous Evaluation Activities

The purpose of evaluating home visiting in your community was to learn how well it worked in your community and to add to body of evidence about effective interventions in AIAN communities. The information you provide about your evaluation experience will also help the federal program to learn how it can best support the development of knowledge about what works in home visiting in tribal communities.

Evaluation Question(s) and Community Considerations:

• Please describe your evaluation question(s) using the PICO¹ format. If you focused on subset of families for your evaluation (e.g., pregnant women, families with children of a certain age), describe this group when discussing the population you focused on.

• Population:

• Intervention:

• Comparison:

• Outcome:

• What hypothesis did you test?

• Please discuss how your evaluation question is relevant to your community. What were some of the considerations your team discussed as you were developing your evaluation questions? For example:

  • Community priorities

  • Selected model

  • Community experiences with and perceptions of evaluation

  • Resources

• Please describe if and how your community engaged in planning and conducting the evaluation study.

Evaluation Design and Methods:

• What was your planned evaluation design? For example:

• Random assignment

• Historical comparison

• Naturally-occurring comparison group

• Single case

• Time series

• Regression discontinuity

• Within person qualitative

• Cost analysis

• Other

• For your selected design, please describe the way it was implemented. For example:

• For random assignment, what was the method of randomization?

• For comparison group designs, how was the comparison group chosen? What were the inclusion and exclusion criteria?

• For single case designs, what type of single case design was used?

• For qualitative designs, what methods were used?

• Where there any changes to your evaluation design during the project period? If so, what were those changes and why were they made?

• Please briefly describe the data sources and measures used for your evaluation, including for both treatment and comparison groups, if using separate groups. (Fill in table)

Outcomes, Measures, and Data Sources for Treatment and Comparison Groups
Outcome	Measures	Sources	How often?	Treatment	Comparison
Example: Parenting quality	HOME Inventory (Bradley)	Home visitor for treatment, evaluation team for comparison	At baseline and 6 months later	X	X

• What kinds of data did you collect in order to determine content, dosage and other fidelity measures?

• What kinds of data did you collect to assess baseline equivalence (if applicable)?

Participant Recruitment and Data Collection:

• Please provide demographic characteristics of your participants in a table (number of families, age of parent/guardian, age of target child (if applicable), race/ethnicity of participants, marital status/relationship status, education level, poverty status, etc.)

• Please provide a table with response and attrition rates for participants at each data point.

• Please provide a table with response rates (N, % of total sample) for each measure at each time point.

Data Analysis Methods:

• Please describe the data analysis methods you used to test your hypothesis (e.g., equivalency analysis, outcome analysis, qualitative analysis, etc.).

Results:

• Please present your data using tables and/or graphs for each outcome measure, by treatment and comparison group (if applicable).

• If making comparisons, were there differences between treatment group and comparison?

• If data were not sufficient to answer the evaluation question(s), what alternative analyses were done?

• Were there any unexpected results?

Findings and Implications:

• What are the key findings from your evaluation? Which of these findings are most salient to your community and program stakeholders?

• How do these findings fit with your community’s understanding of the subject of your evaluation?

• What do your findings mean for your future practice? What if anything might you do differently in light of this evaluation?

• How will your evaluation inform the field of tribal home visiting?

• What are the implications for future evaluation work within your program and community and for the broader field?

Reflections on the Evaluation Process:

• What were the strengths of your evaluation?

• What challenges did you encounter when implementing your evaluation?

• Challenges recruiting study participants for treatment and comparison groups.

• Challenges retaining study participants for treatment and comparison groups.

• Challenges collecting the data.

• Challenges with measures selected and their appropriateness for AIAN populations.

• Challenges with analysis.

• Other challenges.

• What were the limitations of your evaluation?

• Limitations of the design

• Limitations encountered when conducting the evaluation and threats to validity

• What did you learn about conducting a rigorous evaluation in your community?

Technical Assistance Feedback

It would be very helpful to ACF if you could offer feedback on the quality of the technical assistance you received.

• Please describe any barriers or challenges you experienced related to the topics above during the project period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• What methods of technical assistance were most useful (i.e. targeted TA, webinars, site visits, meetings, etc.)? What methods were least useful?

• What additional supports would have been helpful?

Section 5: Report on Continuous Quality Improvement Process.

All tribal home visiting grantees participate in training on CQI and may also receive support from TA providers to conduct CQI projects. In this section, we would like to know about grantees’ experience learning about and using CQI processes with their teams.

• Before receiving this grant and/or participating in the grantee CQI meeting

  • What was your team’s experience with continuous quality improvement?

  • In particular, had you used PDSA cycles prior to the grant? What was that experience?

• Since the CQI meeting, have you engaged in PDSA cycles or other tests to inform quality improvement within your program?

Which of the following CQI tools have you used? For example:

___ Team Charter

___Creating an Aim Statement

___Process Mapping

___Root Cause Analysis (fishbone diagram)

___Affinity Diagram

___Developing Improvement Theory (If/then statement)

___Work plan for QI project

___Other (please describe)

• Please describe your team’s target area(s) for improvement.

  • What were your Improvement Theories? What did you test?

  • When did you begin to test your improvement theories?

  • What data did you collect to determine effectiveness of your tests?

  • When did your various tests begin and end?

  • How did you determine if your changes were improvements or not?

  • Did you develop alternative improvement theories in response to the tests? If so, what were the revised improvement theories?

  • How (if at all) did the results of your CQI tests change your team’s practice/policies in the target area(s)?

• Please describe how the CQI process worked within your program and tribal community.

• How (if at all) did you need to adapt the CQI process to fit your tribal community?

• How might the CQI process, such as the PDSA cycle, need to be adapted for tribal communities in general?

• What aspects of the CQI process fit within your community context, and what did not?

• What (if any) impact do you think the CQI process had on perceptions about data collection amongst your team?

Technical Assistance Feedback

It would be very helpful to ACF if you could offer feedback on the quality of the technical assistance you received.

• Please describe any barriers or challenges you experienced related to the topics above during the project period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• What methods of technical assistance were most useful (i.e. targeted TA, webinars, site visits, meetings, etc.)? What methods were least useful?

• What additional supports would have been helpful?

Section 6: Dissemination

Throughout the entire project period, there may have been many opportunities to share valuable information and engage with multiple stakeholders and target audiences through dissemination. Below, please summarize the dissemination activities of your program in each of the elements listed below.

• What were your dissemination purposes and goals?

• Who were your target audiences for dissemination and information sharing? Which audiences did you disseminate to? Target audiences could include tribal council, the community, families, and funders (local, state, federal, foundations) or others.

• What types of information (qualitative, quantitative) did you prioritize for dissemination?

• What products and materials did you develop and what methods did you use to share them? Please list these and identify which audiences you targeted and how they were disseminated. For instance, did you give a presentation for tribal council? Did you create brochures or marketing materials to recruit families?

  • Relevant dissemination materials and products could include:

    • Newsletters (electronic or print)

    • Pamphlets, brochures, or fact sheets

    • Web- and social media-based products (blogs, podcasts, video clips, etc.)

    • Digital stories or videos

    • Presentations and posters

    • Press communications (TV/radio interviews, newspaper interviews, public service announcements, and editorial articles)

    • Electronic products (DVDs, audio or videos, CD-ROMs)

    • Reports and monographs (including policy briefs and best practices reports)

    • Books and book chapters

    • Peer-reviewed publications in scholarly journals – published and submitted

    • Academic course development and distance learning modules

    • Doctoral dissertations/Master’s theses

• What were the outcomes of your dissemination efforts? Did they achieve the goals and purposes you intended? Please describe any notable successes or challenges.

• Please provide copies of or links to key reports or materials developed for dissemination during the project period.

Technical Assistance Feedback

It would be very helpful to ACF if you could offer feedback on the quality of the technical assistance you received.

• Please describe any barriers or challenges you experienced related to the topics above during the project period, for which you needed technical assistance or training support.

• Please describe any technical assistance or training you requested and/or received from Tribal MIECHV programmatic or evaluation technical assistance providers, model developers, or any other source related to the topics above.

• What methods of technical assistance were most useful (i.e. targeted TA, webinars, site visits, meetings, etc.)? What methods were least useful?

• What additional supports would have been helpful?

1 PICO stands for Population, Intervention, Comparison and Outcomes.