Form to Change - Attachment A Grant Recipient Instrucions

F2F HIC Grant Recipient Instructions 4

OMB No: 0906-0040

Expiration Date: 03/31/2022

Purpose: The U.S. Department of Health and Human Services (HHS), Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau administers the Family-to-Family Health Information Center (F2F HIC) grant program. The goal of the program is to promote optimal health for children and youth with special health care needs (CYSHCN) by facilitating their access to an effective health delivery system and by meeting the health information and support needs of families of CYSHCN and the professionals who serve them. On 03/21/2019 HRSA received Office of Management and Budget (OMB) approval for F2F HIC grant recipients to conduct feedback surveys with families and professionals served on a one-to-one basis and through training¹. The purpose of the surveys is to obtain data on how effective F2F HICs are in providing information, mentoring, and training to enable families of CYSHCN to make informed health care decisions.

Data from the surveys will support the HHS Secretary's priorities of engagement and performance and will provide mechanisms to capture consistent, performance data from F2F HIC grant recipients. The information collected will help HRSA demonstrate the reach of the F2F HIC program and respond to requests from Congress related to the F2F HIC program and program data. In addition, the data will also allow you to evaluate the effectiveness of your interventions and improve services you provide to families and the professionals who serve CYSHCN.

Reporting Requirement: HRSA requires all F2F HIC grant recipients to submit the feedback survey data annually to the HRSA-funded National Center for Family/Professional Partnerships (NCFPP). The NCFPP will clean, aggregate, and report the data to HRSA. You must submit your survey data to the NCFPP at the end of each project year. Failure to submit your survey data in the required timeframe may adversely affect your grant standing. HRSA, in collaboration with the NCFPP, will provide technical assistance to F2F HICs that request assistance with completing these instructions, including calculating number of surveys, reporting the data, etc.

Instructions: Each F2F HIC will contact randomly selected families and professionals who have received one-to-one service or training from their center and ask them to respond to one of three survey instruments. The three separate surveys (see Appendix A) are for (1) families who receive one-to-one service, (2) professionals who receive one-to-one service, and (3) individuals who attend trainings. Survey responses may be collected in-person or via telephone, e-mail, text message, social media, or other technology-based platform. The information provided in these instructions, will help determine the required number of survey responses you must report each year and the procedures you will use to collect and submit the data.

Each F2F HIC should carry out the steps below in order to collect and report the required survey data each project year:

1. Set up a data collection system. The majority of F2F HICs have a system in place to record and track services provided on a monthly basis. Data collection efforts for the F2F HIC survey should build upon this preexisting system. You are highly encouraged to use electronic systems, such as a Microsoft Access or some other contact database, which allow you to quickly determine all contacts made over time. If you currently use a paper log to record service calls from families each month, you will need to refer to this system when determining which families and professionals to contact for survey data collection.

You will also need a data system to track each person’s response. This can be as simple as an excel spreadsheet (see Appendix B for example). Responses to survey questions range from “strongly agree” to “strongly disagree,” and have a corresponding numeric value. HRSA may use this data for various federal reports, including Government Performance Results and Monitoring Act performance measure reporting. HRSA may report on overall feedback percentages for the F2F HICs or choose to report on scores related to specific survey questions. Any data reported outside of HRSA will be aggregated.

2. Submit to HRSA the number of families and professionals served one-to-one and trained. Each year, F2F HICs are required to submit an unduplicated count of families and professionals served/trained by their center. This information is reported in the HRSA Electronic Handbook as performance measures CSHCN-1 and F2F-1 (OMB Number: 0915-0298).

3. Obtain the required number of survey responses needed from HRSA. Each year, HRSA will determine and provide you the number of survey responses your F2F HIC must submit. HRSA has chosen to use a sample of families and professionals served/trained by all F2F HICs. This sample size is calculated using the total number of families and professionals served/trained for all F2F HICs in the previous reporting year (totals from data you provided in Step 2). The sample size is calculated with a 95% confidence level and 5% margin of error. Below is a sample calculation using 2016-2017 actual data.

Sample calculations based on 2016-2017 data:

Total sample Surveys

Total served by needed to achieve required for

Survey Type All F2F HICs (N) 95% confidence (n) each F2F HIC

Families (one-to-one) 100,484 383 6

Professionals (one-to-one) 35,553 381 6

Training Participants 133,540 383 6

Total Surveys Required for each F2F HIC = 18

NOTE: You will likely send more surveys than required because not every person who receives a survey will respond. For example, if you need to collect 18 total survey responses, your evaluator may need to contact 36 individuals in a year, if, on average, about half of those you contact respond. It would be a good idea to track how many responses you get so you can modify your approach for future efforts.

4. Decide on your process for carrying out the surveys. Complete this step after you know the number of survey responses needed. If you have an evaluation/feedback system in place, you can add these survey questions to it. At a minimum, you must administer 50% of the surveys to families receiving one-to-one services via telephone (versus other methods such as web-based or mail). F2F HICs are peer support projects, and it is good practice to speak with some families directly. Other surveys can be administered in the form of texts, emails, online surveys, or social media.

You are encouraged to ask families and professionals to respond to the survey within a month of their interaction with the F2F HIC to increase their recollection about the interaction. Monthly contacts may also reduce the burden of trying to gather all of your responses once a year.

5. Determine how you will randomize the families and professionals you contact. Each F2F HIC is required to randomize the individuals they ask to complete the surveys. This process will help minimize bias. You should not influence the survey findings by collecting feedback from regular, satisfied customers with whom you often interact. The method should be consistent and you should use the same system every year to identify a random sample of individuals to be surveyed. Write the process down in case you are asked to provide information on the method you use.

Do not collect responses from the same family or professional more than once a year.

It is acceptable to administer same-day surveys to training participants if this is your standard practice. Some organizations send training surveys after the training event, which is also acceptable.

6. Identify an objective “evaluator” to carry out any telephone calls. The role your evaluator has at your F2F HIC depends on the type of survey (e.g., phone versus online) and the number of survey responses needed. Volunteers or student interns can administer surveys, if practical. Some F2F HICs have part-time evaluators. It is up to your organization to select the most appropriate approach.

Objectivity is essential; therefore, any staff serving as the evaluator shall not collect responses from people they have served one-to-one directly. Your assigned evaluator can be responsible for collecting all evaluation data and tallying up the responses; this person must be an organized record keeper. You are encouraged to recruit an evaluator who is familiar with evaluation and familiar with the work of your state’s F2F HIC. If you provide multilingual services, please keep this in mind when selecting your evaluator or survey method.

7. Provide your evaluator the necessary data and be clear on the system for contacting individuals. The evaluator should have access to your paper or electronic records to identify the sample of families and professionals to be contacted. Evaluators must use the random sampling system your F2F HIC developed above in Step #4.

8. Conduct the feedback surveys. Separate surveys have been developed for each type of respondent (i.e., family, professional, or training participant). The first question in each survey is considered a screener question. The survey instruments are provided in Appendix A of these instructions. Answers from the feedback surveys can be logged in a spreadsheet or form similar to the Sample F2F HIC Survey Data Log found in Appendix B.

You are encouraged to use the most cost-effective and practical means for contacting individuals. For training participants, it is appropriate to add survey data questions to an existing post-training evaluation form. When conducting follow-up phone calls, in particular, the evaluator should follow the script provided on the “F2F HIC Feedback Survey Script” (see Appendix C). The evaluator should read the question, and follow up each question with the response options provided (i.e., 1 if you strongly disagree, 2 if you disagree, etc.). If necessary, the evaluator can repeat the question in an attempt to provide clarification to the family or professional. The evaluator should then make note of the person’s response and proceed to the next question.

Do not force anyone to remember the purpose for contacting the F2F HIC. If people do not remember who the F2F HIC is or why they contacted the center, chances are, there is no information to base their opinions on. The screener question in the family and professional surveys include a response that states, “I do not remember contacting you.” Record the fact that the family/professional did not remember on the survey data log. If “do not remember” is chosen, your log should not include any additional responses for that family or professional.

9. Contact families and professionals served/trained, preferably within one month.

You are encouraged to contact families and professionals within one month of the service date. Expanding on the example from above: If an F2F HIC must submit 25 survey responses, this data collection can be spread over the course of a year in order to lessen the burden. This would work out to approximately two (2) survey responses per month (assuming a 100% response rate). In some months, a few less or a few more surveys may be collected; this is acceptable as long as data collection is spread over the year and the required number of surveys is attained. Once the annual required number is met, survey data collection for the year can be concluded.

HRSA understands the survey schedule may vary by F2F HIC. We recommend at least a one-month interval to increase the chances the family/professional remembers contacting your center and to encourage you to avoid completing a year’s worth of surveying at one time.

Of course, there will be individuals who attend both a training and come to an F2F HIC for one-to-one assistance. A screener question is included in the training survey in an effort to avoid duplication. To the extent possible, try not to duplicate those families and professionals who are also training participants. The evaluator should guide the family or professional to answer the questions thinking about one instance of assistance or training only. It is understood that responses will sometimes correspond more to a person’s experience with the F2F HIC overall. This is acceptable.

10. For Training Surveys: In an effort to minimize getting duplicate survey responses from families and professionals who receive one-on-one services as well as participate in training, there is a screener question at the beginning of the training survey. Within your final survey count, do not count responses from those who choose ‘Yes’, but go on to report the response in your survey data log that is submitted to the NCFPP.

11. Submit your survey response logs to the HRSA-funded NCFPP. After compiling all survey responses/data, prepare the data for submission to the NCFPP. Your data must be submitted by the designated due date, typically at the end of each project year.

In conclusion, general questions and answers regarding this process are available in Appendix D of this instruction. If you have additional questions about this instruction or your data collection reporting, do not hesitate to contact your assigned HRSA Project Officer.

Thank you, for your data collection efforts, and thank you for supporting families of CYSHCN and professionals across the nation.

APPENDIX A: Survey Instruments

FAMILY SURVEY

(One-to-One Service)

What kind of information/services did you receive from [F2F name]? Select all that apply.

⃝ Answers to questions about my child’s health condition, which is ____________

⃝ Help finding other families with a child with special needs like mine

⃝ Help finding services for my child in my community

⃝ Help finding a doctor or health care provider

⃝ Help with insurance

⃝ Other (please list): ____________________________________

⃝ I do not remember contacting you. Thank the family member for his/her time, and end the survey or call.

Thinking about the services you received from [F2F name], please tell us how much you agree or disagree with the following statements:*	Strongly Disagree (1)	Disagree (2)	Neutral (3)	Agree (4)	Strongly Agree (5)
The information or services I received met my needs.	⃝	⃝	⃝	⃝	⃝
The information or services I received will help me work with those who serve my child (e.g., health care providers, service providers, and other professionals).	⃝	⃝	⃝	⃝	⃝
I will use what I learned to help my child/family.	⃝	⃝	⃝	⃝	⃝
I would recommend [F2F name] to other families.	⃝	⃝	⃝	⃝	⃝

Please share any other comments or concerns with us: ___________________________________________________

*Evaluator Note: If conducting this survey by phone, please repeat the rating scale after each question.

APPENDIX A: Survey Instruments

PROFESSIONAL SURVEY

(One-to-One Service)

What kind of information/services did you receive from [F2F name]? Select all that apply.

⃝ Help talking to a child’s family about the child’s health condition

⃝ Help connecting the family with other families with a child with disabilities/special health care needs

⃝ Help finding community services for a child

⃝ Help finding a doctor or health care provider

⃝ Help with insurance for a child’s family

⃝ Other (please list): ____________________________________

⃝ I do not remember contacting you. Thank the family member for his/her time, and end the survey or call.

Thinking about the services you received from [F2F name], please tell us how much you agree or disagree with the following statements after receiving information or services from them: *	Strongly Disagree (1)	Disagree (2)	Neutral (3)	Agree (4)	Strongly Agree (5)
The information or assistance I received met my needs.	⃝	⃝	⃝	⃝	⃝
The information or services I received will help me work better with families of children and youth with disabilities/special health care needs.	⃝	⃝	⃝	⃝	⃝
The information or services I received will help me work better with other health service professionals (e.g., health care providers, service providers, etc.).	⃝	⃝	⃝	⃝	⃝
I would recommend [F2F name] to families and other professionals.	⃝	⃝	⃝	⃝	⃝

Please share any other comments or concerns with us: _____________________________________________

*Evaluator Note: If conducting this survey by phone, please repeat the rating scale after each question.

APPENDIX A: Survey Instruments

TRAINING SURVEY

[Insert Training Name or Topic]

Have you responded to a survey about other trainings, services or assistance received from [F2F name] this year?

⃝ If Yes, STOP HERE. We do not want to burden you by asking for your feedback more than once per year, so we need no further information at this time. Thank you for your time.

⃝ If No. Please answer the next set of questions to tell us about your training experience with [F2F name].

Thinking about the training you received on [insert date], please tell us how much you agree or disagree with the statements below.*	Strongly Disagree (1)	Disagree (2)	Neutral (3)	Agree (4)	Strongly Agree (5)
The information provided met my training needs.	⃝	⃝	⃝	⃝	⃝
The information I received will help me work better (one-on-one or in workgroups) with others who serve children and youth with disabilities/special health care needs (e.g., health care providers, service providers, etc.).	⃝	⃝	⃝	⃝	⃝
I will use what I learned in the training to better support my child and/or children and families I serve.	⃝	⃝	⃝	⃝	⃝
I would recommend [F2F name] to other families or professionals.	⃝	⃝	⃝	⃝	⃝

Please share any other comments or concerns with us: _____________________________________________

*Evaluator Note: If conducting this survey by phone, please repeat the rating scale after each question.

APPENDIX B: Sample F2F HIC Survey Data Log

S = screener questions included at the beginning of each survey.

Q = corresponds to the feedback survey question.

Excel Tab 1: Families

Respondent Identifier	Response Type	Response date	S1	S1.1	S2	S3	S4	S5	S6	S6.1	S7	Q1	Q2	Q3	Q4	*Average Feedback Score
SMI245	Family	2/2/17	1	CP	1	0	0	0	0	N/A	0	4	4	4	1	3.25
MUR010	Family	2/7/17	0	N/A	1	0	0	1	1	N/A	0	4	3	3	4	3.5
Totals			1		2	0	0	1	1		0	8	7	7	5

Excel Tab 2: Professionals

Respondent Identifier	Response Type	Response date	S1	S2	S3	S4	S5	S6	S6.1	S7	Q1	Q2	Q3	Q4	*Average Feedback Score
DIA007	Professional	2/6/17	0	1	0	1	0	1	IEP	0	3	3	3	2	2.75
KRA874	Professional	2/10/17	1	1	1	1	1	0	N/A	0	4	4	4	4	4
Totals			1	2	1	2	1	1		0	7	7	7	6

Excel Tab 3: Training Participants

Respondent Identifier	Response Type	Response date	S1	S2	Q1	Q2	Q3	Q4	*Average Feedback Score
ABC321	Trainee	2/10/17	0	1	2	1	4	3	2.5
LPS041	Trainee	2/10/17	1	0	N/A	N/A	N/A	N/A
Totals			1	1	2	1	4	3

* The average feedback score for each survey = total score from all feedback survey questions (Q) / four (4) questions.

HRSA may choose to report on a specific question. For example, 90% of families who responded to the F2F HIC feedback survey reported they would recommend the F2F to other families or professionals.

APPENDIX C: F2F HIC Feedback Survey Script

Introducing the Survey

Below is language you can use to introduce yourself and the reason for the survey to the individuals you contact to complete the survey.

For Written/On-line Surveys:

Dear [Insert label or title],

In [specify month of service], you contacted the [F2F name] with a question or request for support, or you participated in a workshop or training they offered. We are interested in feedback about your experience. Please complete this survey to help with our evaluation and to help us improve our services. It will take about 5-10 minutes. The survey is only being sent to a small number of families and professionals who received information, training or support from [F2F name] so it is important that we get responses to all the surveys we send out.

Your answers are confidential, and they will be kept private to the extent allowed by law. Your answers may be used in data reports only after they are combined with answers from other families or professionals. You will not be linked to any information provided in reports. Your participation is completely voluntary. If you choose not to participate, it will not in any way impact your ability to use [F2F name] in the future.

Please complete the survey by MM/DD/YY. Thank you in advance for your help and your time.

Sincerely,

XXXX

Director of the [F2F name]

For Phone or In-Person Surveys:

Hello,

My name is [name of person contacting parent/professional] from [F2F name]. We are interested in getting feedback about your experience with us. Would you have about 5-10 minutes to answer [insert # questions depending on which survey is used] questions about your experience with [F2F name] in the month of [month of service]? It will help us improve our services. Your answers are confidential, and they will be kept private to the extent allowed by law. Your answers may be used in data reports only after they are combined with answers from other families and professionals. You will not be linked to any information provided in reports. Your participation is completely voluntary. If you choose not to participate, it will not in any way impact your ability to use [F2F name] in the future.

We appreciate your input in helping us to continue to improve our programs and services at [F2F]. Thank you for your help and your time.

APPENDIX D: Frequently Asked Questions

How will my F2F HIC reach all these individuals for survey data collection?

By keeping up-to-date contact information on individuals served and trained, you should be able to go to your files, whether they are paper or electronic. Gathering contact information from families and professionals served should be routine. As you well know, families often need to be contacted with more information, and you should have a method to reach them, which can also be used to conduct your surveys. When you collect contact information from families and professionals, it may be helpful to (1) ask if it is okay to contact them later with additional information or to gather feedback, and (2) ask if they have a contact preference.

With the exception of the 50% telephone interviews for the one-to-one family surveys, can the evaluator circulate the surveys electronically?

Yes. Many F2F HICs have the capacity to develop an on-line survey (i.e. Survey Monkey, text messaging, etc.) and the capacity to track responses. You are highly encouraged to tap into this expertise and use technology to gather feedback through electronic surveys, email, text messaging, and/or social media platforms.

How accurate will the survey data be?

If all grant recipients follow the protocol and methodology, they will be adhering to the sample size noted in Step # 3. The corresponding sample sizes reflect 95% certainty within a 5% margin of error. Sample sizes were determined using the calculators found at https://www.surveymonkey.com/mp/sample-size-calculator/ and http://www.raosoft.com/samplesize.html.

How often should I survey an individual?

Do not request a response from any one person more than once in a project year.

How does a Family-to-Family Center determine the number of people to get responses from over a year?

HRSA will determine the number of required survey responses, as described in Step #3 of these instructions.

Is the “evaluator” supposed to contact the required number of individuals provided by HRSA or more?

It is highly likely the “evaluator” may need to contact more individuals than the required number of survey responses needed. For example, if you need to collect 25 total survey responses, your evaluator may need to contact 50 individuals in a year, if, on average, about half of those you contact respond.

If time and resources permit multiple attempts to reach a family may be needed. It is recommended that a clear timeframe be set for call backs/follow-up, to avoid having a backlog. Three attempts over a two-week period is the standard for most F2F HICs. If a family has not responded after three attempts, count this as a non-respondent.

What if the evaluator at my F2F HIC calls a family in June (referencing phone services received in April), but the family member references services received six months earlier?

That is okay. It is true that many family members have multiple contacts with an F2F HIC and may not necessarily be able to put a timeframe on their requests for help. Let the family member answer the questions thinking about the impact of your center more generally. The point is you want feedback!

What is the benefit of including totals for each question, within the Survey Data Collection Log?

Comparing total scores or averages of scores for each question may help you identify areas where you can focus your improvement efforts.

Will technical assistance be available?

Yes, of course. F2F HICs vary in number of years in existence, size, scope, and technological sophistication. Technical assistance webinars will be offered through HRSA and the NCFPP. However, if you have specific questions, contact the NCFPP data team. Note: HRSA funds the NCFPP through a cooperative agreement to provide technical assistance and training for F2F HICs, among other activities.

What do I do with the feedback data I am collecting over the year?

Direct your evaluator to keep it up-to-date in a secure data collection system. Your HRSA Project Officer may ask for this information at any time. However, in most cases, feedback survey data should be submitted annually to the NCFPP. Having this data may also be useful when submitting your annual continuation (progress) report.

Does this feedback survey data collection replace other evaluation/quality improvement that our Center is already doing?

No. Other quality improvement activities currently conducted by your organization should continue. This requirement for feedback data collection and submission by no means replaces other efforts you may have to monitor and improve information and support provided to F2F HICs serving CYSHCN, their families, and health professionals.

When is the Survey Data Collection Log due? Survey data is submitted to the NCFPP each year at the time specified (typically after the May 31^st project period end date).

1 For the purposes of this survey, workshops and trainings are provided by the F2F HIC staff and include a course/topic description and learning objectives.