NonSub Change Request Memo - CWCC - COVID-19 Changes

To: Josh Brammer

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)

From: Mary Mueggenborg and Laura Hoard

Office of Planning, Research, and Evaluation (OPRE)

Administration for Children and Families (ACF)

Date: May 20, 2020

Subject: NonSubstantive Change Request – Building Capacity to Evaluate Child Welfare Community Collaborations to Strengthen and Preserve Families (CWCC) Cross-Site Process Evaluation (OMB #0970-0541)

This memo requests approval of nonsubstantive changes to the approved information collection, Building Capacity to Evaluate Child Welfare Community Collaborations to Strengthen and Preserve Families (CWCC) Cross-Site Process Evaluation (OMB #0970-0541).

Background

In February 2020, OMB approved the new information collection request (ICR) for the Building Capacity to Evaluate Child Welfare Community Collaborations to Strengthen and Preserve Families (CWCC) Cross-Site Process Evaluation. The Cross-Site Process Evaluation is designed to help OPRE and the Children’s Bureau (CB) understand how communities come together to develop and implement integrated approaches to preventing child maltreatment. The cross-site evaluation includes documenting project and organizational leadership approaches, service integration and alignment strategies, and recruitment and assessment methods to identify and serve families most in need of prevention services. The study will provide a rich, detailed description of how grantees work to achieve their projects’ goals and illuminate challenges and strategies for doing this work.

This study uses three modes of data collection: Document review, Survey, and In-person Site Interviews. The ICR that OMB approved in February included instruments for:

• Recruiting survey sample and fielding a survey, annually; and

• Conducting in-person semi-structured interviews, annually.

In response to the COVID-19 pandemic and associated disruptions and travel constraints, and in order to learn more about the COVID-19-related challenges grantees are facing and how they are responding to those challenges, we designed an alternative approach to conducting in-person site visits and modified some of the instruments for this first round of data collection.

We anticipate the need for the changes for at least as long as the pandemic conditions and constraints persist (that is, continued restrictions on travel, continued disruptions of life and work), but may continue with virtual data collection for future data collection waves. It is possible that we will want to incorporate similar changes to future data collection materials and will submit an additional change request in the future, if necessary.

Overview of Requested Changes

This memo requests the following nonsubstantive changes:

• The addition of language acknowledging the unforeseen and challenging effects of the COVID-19 pandemic;

• A shift from in-person, on-site interviews to virtual, telephone interviews; and

• The addition of COVID-19-related probes to interview questions about both challenges and strategies.

COVID-19 Related Language

We propose the following revisions to the emails to be sent to prospective survey respondents (Appendices C and D):

• Appendix C: Email from Project Directors to Survey Invitees Introducing the Data Collection Effort is an email template that the study team will provide to grantee project directors to inform their grant project participants about the survey. The revisions consist of a single sentence acknowledging the challenges and uncertainties associated with the COVID-19 pandemic (please see Appendix C-2: COVID Email from Project Directors to Survey Invitees Introducing the Data Collection Effort).

• Appendix D: Email from Project to Survey Invitees including the Survey Link is the email that the cross-site study will send to survey invitees that includes the link to the online survey. The revisions consist of a single sentence acknowledging the challenges and uncertainties associated with the COVID-19 pandemic (please see Appendix D-2: COVID Email from Project to Survey Invitees including the Survey Link).

The justification for these revisions is compassion; we wish to demonstrate that we are aware of the difficult circumstances that the respondents and their communities may be facing, and we seek to minimize further disruption.

Change to Virtual Interviews

We propose revisions to the mode of data collection for the site visits, shifting from in-person visits with interviews, to virtual visits, with interviews conducted by telephone. This request entails the development of two emails (Appendix F: COVID Email from Project Directors Inviting Interviewees and Appendix G: Virtual Site Visit Planning Email).

• Instrument 3: Site Visit Planning Template is an instrument that the study team will provide to grantee project directors to guide the site visit planning process. If we must conduct visits virtually, we request to instead plan interviews using an email to be sent from the study team to the project director prepopulated with proposed interview candidates (please see Appendix F: Virtual Site Visit Planning Email). If we are conducting interviews via telephone, we will not require the project director to schedule them; rather, the project director will review the list we email to her and will confirm or revise our list of proposed interviewees through a phone conversation.

• Appendix G: COVID Email from Project Directors Inviting Interviewees is a new email template that the project directors can use to present the interviews to the prospective interviewees. In the approved protocol, the project director would contact the interviewees herself to schedule interviews, but in the requested modification, the study team will do that.

The justification for these changes is that as we have temporarily shifted from in-person to virtual mode, it doesn’t make sense for the grantee project director to schedule interviews for our team, so the communication instruments need to be revised accordingly.

COVID-19-Related Probes to Interview Questions

We propose changes to two of the interview protocols to add probes related to challenges and strategies associated with the COVID-19 pandemic.

• Instrument 4: Site Visit Discussion Guide for Project Directors and Leaders from Partner Organizations – Interview #1 is the initial interview with grantee project directors and leaders of organizations partnering with them on the grant. In the interview, we ask about plans, changes to plans, challenges, and strategies; we request to add probes to these questions that will allow respondents to describe the COVID-19-related challenges they have encountered and strategies that they have employed in anticipation or in response to those challenges (please see Instrument 4a: COVID Site Visit Discussion Guide for Project Directors and Leaders from Partner Organizations – Interview #1).

• Similarly to Instrument 4, Instrument 5: Site Visit Discussion Guide for Staff from Lead and Partner Organizations – Interview #1 is designed to elicit staff perspectives on the grant processes, challenges, and effective strategies, and the proposed revisions are aimed at eliciting COVID-19-related changes, challenges, and strategies (Instrument 5a: COVID Site Visit Discussion Guide for Staff from Lead and Partner Organizations – Interview #1).

The justification for these changes is both to acknowledge the effects of the pandemic and an evaluation imperative to understand the full context within which grantees and their partners are working by asking about this prominent change to the way their communities and organizations are operating.

Time Sensitivities

Although we received OMB approval in February and had planned to begin this data collection in March, we have paused data collection in order to allow grantees to gain some measure of equilibrium during the COVID-19 pandemic and its resulting changes and daily challenges. Our project timeline includes annual surveys and annual visits, so we would like to begin data collection as soon as possible (not later than July 1^st) in order to allow some time between rounds of surveys and visits.