Supporting Statement A
Assessing
the Use of Coaching to Promote Positive Caregiver-Child Interactions
in Early Childhood Home Visiting Through Rapid Cycle Learning
This is a new Information Collection Request (ICR). The Health Resources and Services Administration (HRSA) requests Office of Management and Budget (OMB) approval to initiate information collection as part of the Assessing and Describing Practice Transitions Among Evidence-Based Home Visiting Programs in Response to the COVID-19 Public Health Emergency (ADAPT-HV) study to assess the use of coaching to promote positive caregiver-child interactions in early childhood home visiting (ECHV) through rapid cycle learning.
Through the Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program, HRSA provides grants to states, jurisdictions, and Tribal entities, who then have flexibility to develop, implement, and tailor their home visiting programs based on community needs, capacity, and resources within the parameters of statutory and programmatic requirements. State and jurisdiction awardees often contract with local implementing agencies to provide home visiting services in the communities. Home visiting is a service delivery strategy that matches expectant parents and caregivers of young children with a designated support person—typically a trained nurse, social worker, or early childhood specialist—who supports healthy pregnancy practices, encourages early language development and early learning at home, teaches positive parenting skills, connects families to other resources in their community, and provides information to support family health and well-being.1, 2 Services are voluntary and provided in the family's home or another location of the family's choice.
The COVID-19 Public Health Emergency (PHE) led to both MIECHV-funded and non-MIECHV-funded programs placing a greater focus on coaching families to promote positive caregiving skills and family-child interactions, largely due to the shift to virtual home visits.3 Home visitors suggested that this emphasis on coaching and shift away from modeling behaviors (such as in-person demonstrations by home visitors), particularly in virtual settings, enhanced the way that they worked with families. For example, some expressed that this practice led to a greater sense of family-home visitor partnership, and helped them identify new strategies to tailor services to the family.4 Although emerging research indicates that coaching practices during the PHE had positive results on child, family, and program outcomes, it also suggests some burden on home visitors. For example, existing literature on virtual home visits indicate that both coaching and modeling positive behaviors in caregiver–child interactions can be challenging for home visitors who are not in the same physical room with the child.5
Findings also indicate that although home visitors prefer the greater focus on delivering coaching over modeling behaviors, some families do not. A few resources indicated that some families noted a feeling of “missing out” on in-person demonstrations, particularly during virtual home visits.6 These families preferred modeling during home visits because it gives the child an opportunity to interact with another adult and gave them a break from daily caregiving. Given these preliminary findings and the increased use of coaching during the PHE to promote positive caregiving skills, there is a need to better understand strategies around implementing this practice change, how families perceive the practice change, as well as how to address its challenges. The goals of the ADAPT-HV study are to identify such practice changes implemented in response to the COVID-19 PHE and build evidence on how the practice changes might help advance future home visiting service quality and delivery. This study will provide new information about home visiting implementation, specifically the characteristics of coaching, that can improve programming moving forward.
There are no legal or administrative requirements that necessitate this data collection.
Limited research exists about how current ECHV programs have and continue to address the challenges of using coaching strategies during virtual and in-person visits. The purpose of this information collection is to understand how early childhood home visiting programs can implement coaching strategies during virtual and in-person visits by home visitors to promote caregiver-child interactions and positive caregiving skills and gain an understanding of how home visiting programs’ implementation of coaching strategies can be refined. The information collected in this study will help inform ECHV programs about how coaching implementation strategies can be refined to improve services. The information collection uses rapid cycle learning to identify, refine, and test various coaching strategies between home visitors and families. It will answer the following research questions:
What coaching strategies do home visitors use to improve service delivery and promote positive caregiver–child interactions?
How can the implementation of coaching strategies be improved?
How can coaching strategies influence family satisfaction with services, home visitor perceptions of families’ caregiving skills, and home visitor efficacy to meet families’ needs and preferences?
Research question 1 will be addressed through focus groups with home visiting program staff and families participating in home visiting and administrative information provided by program staff. The research question will explore the following topics:
Virtual and in-person coaching strategies home visitors use and how they vary from strategies used before the PHE.
How home visitors applied virtual coaching strategies to in-person services.
Guidance that supports the implementation of coaching strategies and entities providing the guidance.
Programmatic, community, and family contexts that serve as facilitators and barriers to the implementation of these strategies.
Research question 2 will be addressed through a brief questionnaire completed by home visitors and a focus group with program staff (including home visitors). The research question will explore the following topics:
Facilitators and barriers to implementation of coaching strategies.
Refinements to coaching strategies that have the potential to improve or scale implementation.
If refinements improve implementation and whether further refinements are needed.
Research question 3 will be addressed through questionnaires with families and home visitors, as well as focus groups with program staff and families. The research question will explore the following topics:
Program staff perspectives on improvements to service delivery and family engagement as a result of implementing the coaching strategies.
How home visiting modality (in-person, virtual, hybrid) influences how home visitors use coaching strategies.
Study design
The study will use a formative rapid cycle learning framework to identify coaching strategies, then improve and deliver them as defined strategies that can be scaled across other programs and contexts. ECHV programs (agencies that deliver home visiting services to families) will be the unit of analysis. The ADAPT-HV study will use a program eligibility protocol (Instrument 1) to purposely select four programs (referred to as program sites) that use coaching strategies.
Once the program sites are selected, the study will include four phases designed to define, test, and refine coaching strategies:
The first phase—co-definition—uses semi-structured program staff and family focus groups (Instruments 2-4) to identify, prioritize, define, and select coaching strategies that will be used at each site.
During the second phase—installation and initial pilot—sites will implement selected coaching strategies. The study team will gather implementation data through a brief weekly home visitor learning cycle form (Instrument 6) and program staff feedback on the implemented strategies through semi-structured focus groups (Instrument 5). This information will be used to identify refinements to the strategies that may improve implementation.
The third phase—refinement—involves up to two cycles of testing refinements to the strategies. In addition to gathering implementation data using semi-structured staff focus groups and home visitor learning cycle forms (Instruments 5 and 6, respectively), family post-visit questionnaires (Instrument 7) will be used to assess perceived improvements in the implementation of coaching strategies during a cycle and identify further refinements to test.
The fourth phase—summary— focuses on reviewing and assessing overall implementation and process outcomes collected across the duration of the study. Using semi-structured family and staff focus groups (Instruments 8 and 4), the study team will assess the perceived potential of coaching strategies to improve service delivery and promote family engagement and family satisfaction with home visiting programs; and summarize the practice change strategy in its most useful form based on the iterative testing and the perceived potential of the coaching strategies to improve home visiting services.
All respondents that participate in a semi-structured focus group will complete a participant characteristics form – All Phases (Instrument 9), to capture the demographic and contextual characteristics of the respondents in the study.
Universe of data collection efforts
Under this clearance, the study will use a variety of approaches. The exact data collection methods and the samples for each instrument will depend on the site. Data collection instruments (summarized in Table 1) include placeholders in sections that will refer directly to each individual site’s strategy that it is testing. Respondents will include MIECHV-funded ECHV program staff (including program directors, managers, supervisors, and home visitors), and families served by participating program sites. The samples will vary based on the content of the collection and the programs of interest. The study will collect data using well-established methodologies, including:
Semi-structured focus groups: This method involves group sessions guided by a moderator who follows a topical outline containing questions or topics focused on a particular issue, rather than adhering to a standardized questionnaire. Focus groups can be more efficient than individual interviews, since multiple individuals participate at one time. In addition, the group dynamics can yield richer responses than individual interviews for some types of topics. To encourage a high level of engagement, we estimate a maximum of six participants will participate in each focus group.
Questionnaires/Surveys: Questionnaires are common and popular tools to gather data from multiple people. Information from a questionnaire can inform research and evaluation planning as well as program support. Questionnaires may be used to gather information about specific implementation strategies or perceptions of those strategies.
Table 1 includes additional information about each data collection activity across all study phases, by instrument.
Table 1. Summary of Data Collection Activities
Instrument |
Respondent, Content, Purpose of Collection |
Mode |
Duration and Frequency |
Instrument 1. Program Eligibility Protocol |
Respondents: Program director Estimated number of respondents: 16 (16 sites x 1 respondent) Content: Program eligibility, interest, and capacity to participate in study Purpose: To prepare the program to sign an MOU and develop plans for formative evaluation |
Phone, or virtual meeting platform |
60 minutes (once) |
Instrument 2. Program Staff Focus Group Protocol 1 – Co-definition Phase |
Respondent: Program staff (e.g., director, managers, supervisors, and home visitors) Estimated number of respondents: 24 (4 sites x 6 respondents) Content: Guidance surrounding in-person and virtual coaching strategies, type of content delivered through coaching, facilitators of and challenges to implementation, family and community contexts, Purpose: Understand the practice changes that have been implemented to promote the use of coaching during virtual or in-person visits and implementation successes and challenges |
Phone or virtual meeting platform |
90 minutes (once) |
Instrument 3. Program Staff Focus Group Protocol 2 - Co-definition Phase |
Respondent: Program staff (e.g., director, managers, supervisors, and home visitors) Estimated number of respondents: 24 (4 sites x 6 respondents) Content: Review of information gathered in first co-definition staff focus group, group activity to prioritize strategies of interest, selection of strategy to refine and test Purpose: Prioritize and select a practice to test in subsequent phases |
Phone or virtual meeting platform |
90 minutes (once) |
Instrument 4. Family Focus Group Protocol - Co-definition & Summary Phases |
Respondent: Families Estimated number of respondents: 48 (4 sites x 6 respondents) Content: Awareness of, satisfaction with, and perceived utility of coaching during the visit Purpose: Understand impressions of the services they received over the study, perceptions of how they work, and suggestions for improvement |
Phone or virtual meeting platform |
60 minutes (twice) |
Instrument 5. Program Staff Focus Group Protocol – Installation and Refinement Phases |
Respondent: Program staff and home visitors Estimated number of respondents: 72 (4 sites x 6 respondents, per collection) Content: Coaching strategies used, perception of family caregiver-child interactions, perception of caregiving skills Purpose: Understand how staff implement a strategy, their perceptions of how it is working, and suggestions for improvement |
Phone or virtual meeting platform |
60 minutes (3 times) |
Instrument 6. Home Visitor Learning Cycle Questionnaire - Installation & Refinement Phases |
Respondent: Home visitors Estimated number of respondents: 40 (4 sites x 10 respondents) Content: Coaching strategies used, mode of content delivery during visits, perception of family caregiver-child interactions Purpose: Understand staff use of coaching strategies over each learning cycle |
Web-based |
10 minutes, weekly (9 times) |
Instrument 7. Family Post-Visit Questionnaire - Refinement Phase |
Respondent: Families Estimated number of respondents: 48 (4 sites x 6 respondents, per collection) Content: Coaching strategies used, mode of content delivery during visits, perception of family caregiver-child interactions, perception of coaching strategies Purpose: Understand impressions of the home visit and perceptions of caregiver-child interactions during visit |
Web-based |
5 minutes (6 times during each of 2 collection periods) |
Instrument 8. Program Staff Focus Summative Group Protocol - Summary Phase |
Respondent: Program staff and home visitors Estimated number of respondents: 24 (4 sites x 6 respondents) Content: Lessons learned and reflection surrounding coaching strategies used throughout learning cycles, home visitor self-efficacy, perceptions of family caregiver-child interactions and positive caregiving skills Purpose: Understand type, frequency, and purpose of implementation practices used, perceived potential to improve services |
Phone or virtual meeting platform |
60 minutes (once) |
Instrument 9. Focus Group Participant Characteristics Form – All Phases |
Respondent: All focus group respondents: program staff, families, home visitors Estimated number of respondents: 120 (4 sites x 6 focus groups per site x 5 respondents per group; assumes some individuals participate in more than one focus group) Content: Program staff: race/ethnicity, tenure in position, tenure with agency, enrollment capacity, actual enrollment staff capacity, race/ethnicity of families served. Home visitors: race/ethnicity, tenure in position, tenure with agency. Families: parent and child gender identity, ages, race/ethnicity of parent, participation (number of home visiting sessions attended), how long they have been receiving services. Purpose: Describe focus group sample |
Web-based |
5 minutes (once per focus group) |
Each phase of the study will collect multiple sources of information to assess implementation and the success of the tested coaching strategies during virtual home visiting that are identified by the program. This information will be shared with program staff throughout the study. The planned information collection includes the use of technological data collection techniques. Specifically, the study includes online questionnaires and web-based forms.
Online data collection allows for efficiencies and reductions in respondent burden. Web-based surveys provide efficient ways of using skip logic to quickly move to the next relevant question depending upon a respondent’s answer selection. Web-based surveys also provide ways to limit invalid responses so that they cannot be entered and can prompt the respondent to enter a valid response. Web-based surveys reduce burden related to completing and mailing (or otherwise submitting) paper forms.
All online data collection instruments will be offered through Qualtrics, a web-based survey tool. Qualtrics uses Transport Layer Security (TLS) encryption for all transmitted data. Instruments and data collected through Qualtrics can only be accessed by authorized users with appropriate login credentials. Data exported from the platform for analysis will be stored on restricted, encrypted folders on PRG and Mathematica’s networks, which are only accessible to the study team. Data will be deleted from the platform at the end of the study.
To ensure consistency in data collection, the study team will develop appropriate processes for each data collection activity. For all questionnaires (Home Visitor Questionnaire [Installation & Refinement Phases], Family Post-Visit Questionnaire [Refinement Phase]), the study team will develop web-based survey response criteria. For example, numeric range restrictions on questions about caseload, age, and program start and end dates, among others. The study team will also use skip patterns to ask respondents only the most relevant questions.
The home visitors will provide the questionnaire to families in the form of a self-administered web survey, developed by the study team, via a tablet or laptop. If they are conducting a virtual visit, the link to the questionnaire will be shared with the family member through the video platform chat or by email or by text message.
To minimize burden on program staff and participants, the study design incorporates small-scale and iterative testing with small sample sizes and short data collection periods. The study team will tailor procedures to reduce burden on program staff. The study team will be cognizant of the burden participation in research can place on programs and participants and will work closely with programs to ensure methods build on existing processes and data.
This research will not duplicate any other information collections. Based on a survey of existing literature and expert consultation, none of the instruments ask for information that can be reliably obtained through other sources and used or modified for the purposes described in section 2 above. To the maximum extent possible, we, in collaboration with HRSA staff, will make use of existing data sources before we attempt to utilize the additional fieldwork sought under this clearance and will avoid any duplication of information collection efforts. HRSA is conducting two other related studies exploring adaptations to home visiting practices during COVID-19. Each study focuses on a different set of practices and will together provide a more comprehensive picture of how home visiting programs adjusted service delivery during the PHE. While home visiting programs are permitted to participation in multiple studies, the four programs who participate in this study will not participate in the other two studies as they will be asked to prioritize one set of practice changes during recruitment.
This information collection will not have a significant economic impact on a substantial number of small businesses or other small entities. The home visiting program sites in the study may be small, non-profit organizations. The study team will only request information required for the intended use. The burden for respondents will be minimized by restricting the focus group and questionnaire length to the minimum required, by conducting focus groups at times convenient for the respondents, and by not requiring recordkeeping on the part of the programs.
A key goal of the study is to be able to test strategies for coaching and then make refinements to improve implementation with an ultimate aim of strengthening programs’ coaching practices. The iterative nature of rapid cycle evaluation necessitates repeat data collection over multiple cycles. Some of the study’s data collection will occur over two to three cycles: once during the Installation Phase, and one to two times during the Refinement Phase. Without repeat data collection, it would be difficult to assess the feasibility and effectiveness of the strategies that are implemented. The approach attempts to limit the scope of data collection to just the information needed to assess the success of the strategy being implemented. Over the rapid cycle evaluation, each home visitor respondent will complete the 10-15-minute Home Visitor Learning Cycle Questionnaire – Installation and Refinement Phases (Instrument 6) up to nine times, and each family respondent will complete the 5-minute Post-Visit Questionnaire – Refinement Phase (Instrument 7) up to six times, depending on how frequently this evaluation activity is deemed useful to the program while piloting a new strategy or process. The short time frame and multiple responses facilitate rapid adaptation and refinement when program changes do not appear to be working as intended. Administering the survey less frequently would yield less actionable data for programs to use to refine their strategies.
The program eligibility protocol (Instrument 1), program staff focus group protocol 1- Co-definition Phase (Instrument 2), program staff focus group protocol 2- Co-definition Phase (Instrument 3) program staff summative focus group protocol -Summary Phase (Instrument 8), and focus group participant characteristics form – All Phases (Instrument 9) are one-time data collection activities. The program staff focus group protocol – Installation & Refinement Phases (Instrument 5) will be administered at the end of each cycle, up to three times. Each administration will be used to assess progress in the rapid cycle evaluations and adjust the approach. The family focus group protocol – Co-definition & Summary Phases (Instrument 4) will be administered twice. The first administration will be used to select coaching strategies and define the evaluation project’s strategy. The second administration will contribute to an overall assessment of the success of the coaching strategies tested and provide information about potential improvements to those strategies.
This request fully complies with the regulation.
Section 8A:
A 60-day notice published in the Federal Register on 12-05-2023, vol. 88, No. 232; pp. 84342-43 (Appendix A). HRSA received one comment from a home visiting model developer. An abbreviated version of the comment and responses are summarized below. Copies of the public comments are included as Appendix N.
Comment: Respondent expressed concern about the ability to define and quantify the use of coaching.
Response: The study’s focus on coaching is based on 1) a large body of evidence that many providers’ desire to use coaching has not translated into practice and 2) the COVID-19 PHE leading to increased use of coaching during virtual service delivery, with parents inherently needing to take a more active role in child-directed interventions. Existing literature on coaching indicates that the concept can be defined and quantified. Building off of these findings, this study intends to help identify and test implementation strategies that enable more effective use of coaching in situations where its use can improve service delivery.
Comment: Respondent expressed concern that study instruments appear to present coaching as a universally superior method of service delivery despite the context and circumstances around home visitor/family interactions determining whether and when it is appropriate to use coaching.
Response: We have revised study instruments to emphasize tracking a home visitor’s use of strategies to conduct effective coaching when appropriate and to clarify that both modeling and coaching can be useful practices during home visits. The study intends to identify implementation strategies that enable more effective use of coaching when that technique is appropriate in order to improve service delivery. The study involves working with individual home visiting programs to develop strategies that are appropriate within the specific model(s) that they are implementing.
Comment: Respondent highlighted that the terminology of “coaching” may be misinterpreted and could have negative connotations for home visitors and/or families.
Response: The study team discussed the term “coaching” with home visiting program administrators, home visitors, and families during advisory board consultations that informed the study design. Individual advisors suggested alternate terms; however, there was no one term that emerged as a clear alternative that would make sense to all advisors. Overall, their feedback indicated that whatever term is used needs an explicit definition. Consequently, we revised focus group and interview guides to include a clear and succinct definition of coaching with the following definitions:
Coaching refers to a practice in which the home visitor supports caregiver-led, child-directed skills or activities through verbal guidance or direction.
Modeling strategies are when the home visitor demonstrates skills or activities while families observe.
Comment: Respondent shared that the protocols lacked information on obtaining informed consent.
Response: The study includes an informed consent statement for families (Appendix D) that clearly states that their participation will not affect the services they receive.
Section 8B:
Consultation with subject matter experts external to the study team was conducted in 2023 as part of the design of this study (Table 2). The subject matter experts provided feedback on the design, specifically the research questions.
Table 2. Individuals consulted with about this information collection
Subject Matter Expert Role |
Director of Early Childhood and Family Support Research School of Social Work |
Senior Research Fellow Chapin Hall |
Assistant Professor, Department of Population, Family and Reproductive Health |
Professor, Suzanne Dworak-Peck School of Social Work |
Institution |
Portland State University |
The University of Chicago |
Johns Hopkins University |
University of Southern California |
The study team also consulted with three advisory boards developed through this study, each consisting of five individuals in three different types of roles: program staff, home visitors, and families participating in home visiting. Each advisory board was asked a unique set of questions designed to draw on their specific expertise. These individuals provided feedback on language used to describe specific concepts in information collection instruments and strategies to best implement information collection procedures.
Focus group data are not intended to be representative of the experiences of all participant experiences in ECHV programs. However, it is important to recruit participants with a range of background characteristics to capture a range of possible program experiences. Without offsetting the direct costs of participating in the focus groups, such as arranging childcare, the research team increases the risk that only individuals able to overcome financial barriers to attend will participate in the study. A relatively consistent finding is that the tokens of appreciation can increase response rates,7, 8 and in some cases can double response rates.9 Providing tokens of appreciation to respondents can also be a way of acknowledging the value of the time and knowledge being shared with the study team. They also reinforce the notion that the local community knowledge being shared by respondents is valued, respected, and honored. In this collection, the study team will offer $40 gift cards to families that participate in focus groups to encourage the participants to engage in information sharing. The amount is based on estimated burden and evidence from previous studies with similar respondents and types of data collection requests. For example, results from a study conducted in 2013 on the effectiveness of tokens of appreciation for engaging participants in qualitative research found that any monetary token of appreciation was more likely to result in willingness to engage in a qualitative interview than a nonmonetary one or none.10 In the same study, among those participants who reported some willingness to engage in a 90-minute qualitative interview, those who were offered $75 were more willing to engage than those who were offered $25. As such, the study team prorated to a $40 token of appreciation for participation in 60-minute focus groups.
This data collection effort will collect personally identifiable information (PII) from program staff (names, work email addresses and telephone numbers) and families (names, phone numbers, and email addresses) for the purposes of arranging data collection (including scheduling and sending invitations to virtual data collection activities) and, for families, for sending tokens of appreciation (if digital gift cards are used). Any data collected will be de-identified, and transmitted via a password protected, secure file share system. Information will be saved on in a secure folder accessible only to the study team. The study team will destroy data at the end of the study.
Information collected will be kept private to the extent permitted by law. Issues of privacy will be discussed during training sessions with staff working on the study. All study team members receive training on data security policies and procedures consistent with their organization’s documented policies. Program staff will transfer records using a secure file transfer protocol site in case the files contain PII.
Participation in all data collection activities is voluntary. All respondents will be informed that their responses will be kept private to the extent permitted by law. At the beginning of each focus group session, the facilitator will explain and ensure all participants understand the purpose of the study, their privacy rights, and that their participation in the study is voluntary. The focus group facilitators will ask that respondents do not share any information or personal experiences that they hear from others during the group. However, given the nature of a focus group (i.e., multiple respondents sharing information together), all respondents will hear responses from the group and confidentiality cannot be fully guaranteed. Focus groups will be recorded. Recordings, notes, and transcriptions will be saved to a secure drive and only the study team will have access.
For questionnaires, privacy means that only the study team will have access to their responses and that individual responses will not be shared with any home visiting program or local or state agencies or identified in any report. To ensure privacy for questionnaires administered after an in-person visit, home visitors will instruct families to return the tablet or laptop only after they have pressed the “submit” button on the web survey. The web questionnaire will include a consent statement at the beginning of the form explaining that home visitors will not have access to any families’ responses.
No information will be given to anyone outside of the study team and HRSA. All PII, typed notes, and audio recordings will be stored on restricted, encrypted folders on PRG and Mathematica’s networks, which is only accessible to the study team. The data will be stored for the duration of the study and destroyed at the end of the study in accordance with the National Archives and Records Administration (NARA) requirements.
The Mathematica Institutional Review Board (IRB) will review and approve all final data collection protocols prior to the commencement of data collection.
The study will collect information about respondents’ race and ethnicity in Instrument 9. Focus Group Participant Characteristics Form - All Phases which will be used to describe the study sample. Race and ethnicity questions will align with Figure 2. Race and Ethnicity Questions with Minimum Categories Only and Examples from the revised SPD-15 standards (effective 3/28/24; see 89 FR 22182). The study will use minimum categories and examples, rather than the more detailed categories, to protect participant confidentiality. Given the small respondent sample sizes (N=120), the potential risks to confidentiality in situations where there are a very small number of individuals from a given racial or ethnic group outweigh the potential benefits of having more detailed demographic information. These demographic questions are essential to understand the contexts that strategies are being implemented in, and enable strategies to be tailored in response to the needs of respondents. Additionally, to address variability, the study team will explicitly define the adaptations and contextual factors that the study responded to when framing our findings so that future programs can examine their own contexts and pursue appropriate strategies.
No other sensitive questions are included in the instruments.
The estimated burden per respondent varies (as shown in Table 3). The total burden for this information collection is 325.8 hours. There may be variation in the number of respondents in each program sites (e.g., some selected program sites may have fewer home visitors). The total burden hours presented here assumes the maximum number of respondents across all four program sites. This burden estimate includes the time expended by persons to generate, maintain, retain, disclose, or provide the information requested. The general equation used to calculate burden for each form is: # participants * # responses per participant * number of program sites * average estimated burden per response = total burden hours for each instrument. Below, we describe how the total number of responses were calculated for forms with multiple responses per participant:
Program Staff Focus Group Protocol – Installation & Refinement Phases (Instrument 5): Program participants will participate in semi-structured focus groups. There will be up to three focus groups in each of the 4 sites, each with up to 6 participants, for a total of 72 observations (3 focus groups * 4 sites * 6 participants).
Home Visitor Learning Cycle Questionnaire – Installation & Refinement Phases (Instrument 6): The study team will administer a 10-minute web survey to up to 10 home visitors in each of the 4 sites, for a total of 40 respondents. The team will administer these to home visitors multiple times over the study. The study team assumes they will administer these weekly for 3 weeks during each of the 3 cycles within the Installation & Refinement Phases for a total of 9 surveys per home visitor (3 cycles * 3 weeks/cycle), resulting in a total of 360 observations (9 responses per respondent * 40 respondents).
Family Post-Visit Questionnaire – Refinement Phase (Instrument 7): The study team will administer a 5-minute web survey to up to 12 family members in each of the 4 sites, for a total of 48 respondents. The study team assumes they will administer questionnaires surveys after each visit during a 6–12-week period, up to a maximum of 6 times per family, resulting in a total of 288 observations (6 responses per respondent * 48 respondents).
The instruments as written and submitted deliberately include more topics/questions than there is time for in the length allocated to the data collection activities in the burden table. Individual study team members working with the sites will select the questions most relevant to what their sites are working on and drop irrelevant questions.
12A. Estimated Annualized Burden Hours
Type of Respondent |
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Total Responses |
Average Burden per Response (in hours) |
Total Burden Hours |
Program staff |
Program Eligibility Protocol |
16 |
1 |
16 |
1.00 |
16 |
Program staff |
Program Staff Focus Group Protocol 1 (Co-definition Phase) |
24 |
1 |
24 |
1.50 |
|
Program staff |
Program Staff Focus Group Protocol 2 (Co-definition Phase) |
24 |
1 |
24 |
1.50 |
36.0 |
Program staff |
Program Staff Focus Group Protocol (Installation & Refinement Phases) |
24 |
3 |
72 |
1.00 |
72.0 |
Program staff |
Program Staff Focus Group Protocol (Summary Phase) |
24 |
1 |
24 |
1.00 |
24.0 |
|
|
|
|
|
|
|
Families who participate in home visiting |
Family Focus Group Protocol (Co-definition & Summary Phases) |
48 |
1 |
48 |
1.00 |
48.0 |
Home Visitor |
Home Visitor Learning Cycle Questionnaire (Installation & Refinement Phases) |
40 |
9 |
360 |
0.17 |
61.2 |
Families who participate in home visiting |
Family Post-Visit Questionnaire (Refinement Phase) |
48 |
6 |
288 |
0.08 |
23.0 |
Program staff and families who participate in home visiting |
Focus Group Participant Characteristics Form (All Phases) |
120 |
1 |
120 |
0.08 |
9.6
|
Total |
|
368.0 |
|
976.0 |
|
325.8
|
|
|
|||||
The estimated total cost to respondents is approximately $19,314.70 (as shown in Table 4). These estimates are based on our experience with collecting information, interviewing professional staff, and conducting focus groups. The study team based the average hourly wage estimates for deriving total annual costs on data from the Bureau of Labor Statistics, Usual Weekly Earnings of Wage and Salary Workers (2024 first quarter). For each instrument in Table 4, the team calculated the total annual cost by multiplying the annual burden hours by the average hourly wage. The study team used the mean hourly wage of $32.6811 for women in professional and related occupations for program staff (including home visitors), as the team expects many of the staff working in these positions to be women.12 The team used the mean hourly wage of $20.1013 for women high school graduates with no college education for families participating in the focus groups.14
For the focus group participant characteristics form, the cost to respondents is based on two types of respondents: families and program staff (including home visitors). Based on the planned data collection, a maximum of 96 respondents will be families and 24 will be program staff. The study team used the mean hourly wages for families and program staff for this instrument as outlined above.
The total respondent cost presented here assumes the maximum number of respondents in each program site. Since award recipients are spread across the country, the median hourly rate is used, as opposed to adjusting for locality. Additionally, wage has been doubled to account for overhead costs.
Table 4. Estimated Annualized Cost to Respondents
Type of Respondent |
Total Burden (hours) |
Average Hourly Wage |
Total Respondent Cost ($) |
Families who participate in home visiting |
78.68 |
$40.20 |
$3,162.94 |
Program staff |
247.12 |
$65.36 |
$16,151.76 |
Total |
325.8 |
|
$19,314.70 |
The study team will provide an honorarium to participating programs in recognition of the administrative burden associated with their participation in the study. The study team will provide an honorarium of $100 to any program that participates in the initial eligibility discussion, regardless of whether they are eligible and/or are selected as a study site. For programs sites that participate in the study, the honoraria will be $1,500 per program site. As demonstrated in Table 3, which provides burden estimates for just the data collection activities associated with the study, this amount is less than the anticipated cost of staff time needed to complete individual-level recruitment, coordinating focus groups with program staff and families, and data collection throughout the cycles. The study team will ask each study site to designate a liaison to coordinate data collection activities as described in A.9. There are no other total annual cost burdens to respondents or record keepers resulting from the collection of information.
The total cost of this information collection to the Federal Government is $243,895, which includes the cost of the contract to PRG to perform the study as well as the cost of federal employees supporting the study.
The annualized contractual cost for this information collection is $204,375, which includes designing data collection instruments, collecting all data and analyzing data. Estimates are based on the study team’s budget for each task (design, implementation, and analysis) and include labor hours, other direct costs, and subcontractor costs. The cost of the incentives is included in the cost of supplies paid to PRG.
In addition, costs to the federal government include the cost of federal staff time for project oversight and development. This includes approximately 10% of a federal public health analyst at Grade 13, Step 4 ($95 per hour for 416 hours) for a total cost of $39,520. Wage has been multiplied by 1.5 to account for overhead costs.
This is a new information collection request.
The information collected under this request will be used to test the success of various strategies implemented at selected ECHV programs and inform HRSA activities.
Project Time Schedule
Beginning 1-2 months after OMB approval, the study team will conduct study recruitment activities, including collecting data through the program eligibility protocol. 2-4 months after OMB approval, the study team will work with selected programs to implement a collaboratively identified strategy and begin information collection activities. Timeline of collection may vary for individual program sites.
Plans for Tabulation.
For this information collection, quantitative questionnaire data, such as from a Home Visitor Learning Cycle Questionnaire, will include descriptive statistics and cross tabulations to assess sample size, characteristics, response rates, and data quality. To analyze qualitative data, such as focus groups, the study team will develop and apply a coding scheme to identify and summarize common themes across topics or respondent types. Following each learning cycle, the study team will share findings with program staff and HRSA, such as summary statistics from a learning cycle form, or initial themes identified during focus groups.
Plans for Publication
At the end of the study, the study team will develop a summative report that shares information with HRSA describing work with the sites and lessons learned about how implementation strategies address challenges ECHV programs face around coaching. These findings, such as themes and recommendations identified through thematic coding of focus groups, will be disseminated in publications (briefs, manuscripts) and other public facing products. The main audience for these publications will vary, but may include HRSA leadership and staff, ECHV practitioners, researchers, advocates, and other interested parties in the broader field of ECHV. In sharing findings, we will describe the study methods and limitations with regard to generalizability and as a basis for policy.
The OMB number and expiration date will be displayed on every page of every form/instrument.
There are no exceptions to the certification.
1 National Home Visiting Resource Center. (2022). What is Home Visiting? https://nhvrc.org/what-is-home-visiting/
2 Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program. (2022). https://mchb.hrsa.gov/programs-impact/programs/home-visiting/maternal-infant-early-childhood-home-visiting-miechv-program
3 Rapid Response – Virtual Home Visiting Collaborative. (2020). Engaging families in virtual visits: A protective factors’ approach [Webinar]. Retrieved April 21, 2023, from https://rrvhv.earlyimpactva.org/webinar/engaging-families-in-virtual-visits-a-protective-factors-approach; Roben, C. K., Kipp, E., Schein, S. S., Costello, A. H., & Dozier, M. (2022). Transitioning to telehealth due to COVID-19: Maintaining model fidelity in a home visiting program for parents of vulnerable infants. Infant Mental Health Journal, 43(1), 173-184. DOI: 10.1002/imhj.21963
4 Chazan-Cohen, R., Fisk, E., Ginsberg, I., Gordon, A., Green, B. L., Kappesser, K., Lau, S., Ordonez-Rojas, D., Perry, D. F., Reid, D., Rodriguez, L., & Tomkunas, A. (2021). Parents’ experiences with remote home visiting and infant mental health programs during COVID-19: Important lessons for future service delivery. Perigee Fund. https://perigeefund.org/wp-content/uploads/2021/10/ParentVoices-FullReport-English.pdf.
5 Behl, D. D., Blaiser, K., Cook, G., Barrett, T., Callow-Heusser, C., Brooks, B. M., & White, K. R. (2017). A multisite study evaluating the benefits of early intervention via telepractice. Infants & Young Children, 30(2), 147-161. DOI: 10.1097/IYC.0000000000000090; Cole, B., Pickard, K., & Stredler-Brown, A. (2019). Report on the use of telehealth in early intervention in Colorado: Strengths and challenges with telehealth as a service delivery method. International Journal of Telerehabilitation, 11(1), 33. DOI: 10.5195/ijt.2019.6273; Olsen, S., Fiechtl, B., & Rule, S. (2012). An evaluation of virtual home visits in early intervention: Feasibility of “Virtual Intervention”. The Volta Review, 112(3), 267-281. Retrieved April 21, 2023, from https://www.proquest.com/scholarly-journals/evaluation-virtual-home-visits-early-intervention/docview/1266509694/se-2
6 Hadley, A., Hayes, J., Pai-Samant, S., & Stern, F. (2023). Virtual home visiting during the COVID-19 pandemic: Lessons learned for research, practice, and policy. Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Retrieved April 21, 2023, from https://www.acf.hhs.gov/opre/report/virtual-home-visiting-during-covid-19-pandemic-lessons-learned-research-practice-policy.
7 Singer, E., and C. Ye. “The Use and Effects of Incentives in Surveys.” The ANNALS of the American Academy of Political and Social Science, vol. 645, no. 1, 2013, pp. 112–141. https://doi.org/10.1177/0002716212458082
8 Sundstrom, E. D., Hardin, E. E., & Shaffer, M. J. (2016). Extra credit micro-incentives and response rates for online course evaluations: Two quasi-experiments. Teaching of Psychology, 43(4), 276-284.
9 Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002).
Increasing response rates to postal questionnaires: Systematic review. BMJ, 324(7347),
1183.
10 Kelly, Bridget, Marjorie Margolis, Lauren McCormack, Patricia A. LeBaron, and Dhuly Chowdhury. “What Affects People’s Willingness to Participate in Qualitative Research? An Experimental Comparison of Five Incentives.” Field Methods, vol. 29, no. 4, November 2017, pp. 333–350. https://doi.org/10.1177/1525822X17698958.
12 Sandstrom, Heather, Sarah Benatar, Rebecca Peters, Devon Genua, Amelia Coffey, Cary Lou, Shirley Adelstein, and Erica Greenberg. 2020. Home Visiting Career Trajectories: Final Report. OPRE Report #2020-11, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. https://www.acf.hhs.gov/sites/default/files/documents/opre/hvct_final_report_feb_2020.pdf
14 https://mchb.hrsa.gov/sites/default/files/mchb/about-us/home-visiting-infographic.pdf
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Annalee Kelly |
| File Modified | 0000-00-00 |
| File Created | 2024-08-01 |