2

National Institute of Health

National Institute of Child Health and Human Development

Testing of automated self-response methods for collecting the reduced content version of NCS pregnancy status follow-up contacts

Request for OMB Clearance of Data Collection Instruments

under NIH/NICHD Generic Clearance # 0925-0590

NICHD Division: Program Office, National Children’s Study

Contact Name: Dr. Ken Schoendorf

Contact Phone: 301-594-1302

Contact Email: Schoendk@mail.nih.gov

Title: Testing of automated self-response methods for collecting the reduced content version of NCS pregnancy status follow-up contacts

Plan Control Number: 0925-0590

Program Title: Formative Research and Pilot Methodology Studies for the National Children’s Study

Program Goals: Studies performed under this program are designed to improve data collection within the National Children’s Study (NCS). Projects under this Generic Clearance will include projects designed to:

- improve Community Engagement at Study Locations, thus improving participant recruitment and retention

- inform methods of engaging Community-level health and medical providers, resulting in improved participant recruitment and retention

- test technologic methods of interaction with participants (e.g., Interactive Voice Recognition) to decrease respondent burden, improve participant satisfaction, and decrease cost.

Purpose of the Survey: This plan describes a proposed test for evaluating response rates using different self-response methods (Interactive Voice Response, web/e-mail, and text messaging) for collecting a reduced-content version of the NCS pregnancy status follow-up contacts. The current protocol uses a longer questionnaire administered by Computer Assisted Telephone Interview (CATI) that asks detailed questions about factors related to the probability of pregnancy (e.g., use of birth control) and is not suitable for all data collection methods (e.g., text messaging). Compared to the existing phone calls, the automated self-response methods, if successful, will result in lower costs to the Study, less burden to the participant, and the collection of data earlier in pregnancy.

The primary goal of the follow-up contacts is to accurately identify women in the study locations as early in the first trimester of pregnancy as possible. With this in mind, the Program Office is considering an approach for the follow-up contacts that differs from the current approach on the following parameters:

• Reduces the content of follow-up contacts to focus only on current pregnancy status and the intention to get pregnant. Detailed information allowing for movement between moderate and low probability of pregnancy groups will not be collected

• Increases the frequency of contact for women in the moderate and low probability of pregnancy groups, allowing for earlier identification of pregnancy and the collection of data earlier in pregnancy

• Provides several different self-response methods for responding to the follow-up contacts, potentially increasing participant response and follow-up.

Evaluating the success of this alternative approach at identifying women early in their first trimester of pregnancy will require testing within the NCS population. However, before implementing an alternative data collection protocol using multiple self-response methods with the NCS population, we propose starting with a smaller test with a recruited panel of women. Specifically, this first stage of testing will evaluate using an Interactive Voice Response (IVR) system, a web/e-mail application and a text messaging application, all self-response approaches, and focus on usability issues with those applications as well as any major issues with the design of those applications or contact protocols that might impact cooperation.

Once this first usability-type test is completed, the next stage of testing will occur as part of the NCS Vanguard Pilot phase of the Study, with a subsample of the women in the low and moderate Pregnancy Probability Group. In this second stage of testing, the alternative automated self-response approach for the follow-up contacts will be implemented to determine if this results in differences compared to the current CATI protocol in terms of:

• cooperation with the follow-up contacts, both in terms of response rates and the level of effort needed to get a complete response, and;

• number of women identified in their first trimester of pregnancy as opposed to later stages of pregnancy.

This request is only for the first stage of the project – the usability portion that does not use the NCS Pilot participants. However, information on the planned second stage is also included in this request to provide the overall context for the proposal

Target Respondents: The first stage of testing will target women outside of the NCS Vanguard Pilot in the DC-metro area who meet the NCS age eligibility requirements for pregnancy screening and represent a wide range of race/ethnic groups, and education levels. The second stage of testing will use a subsample of women identified for follow-up in the moderate and low probability of pregnancy groups of the NCS Vanguard Pilot cohort.

Survey Design and Administration:

Recruitment: For the first stage of testing, the contractor (Westat) will use the same professional recruiting company who has successfully recruited all the respondents for the pretesting of the NCS Enumeration and Pregnancy Screening interview instruments, advance materials, and informed consent booklets. The recruiting screener (Appendix A) will collect demographic information (age, education, ethnicity and race, household structure, marital status, presence of children), as well as the following:

• Level of internet/email use

• Level of texting use

• Prior experience with IVR.

Lastly, the recruiting screener will include a final question at the end of the screener that asks something like “if we have other similar surveys in the future that also might be of interest to you, how would you prefer we contact you—by E-mail, phone, or by text?” This will give us a measure of the respondents’ preferred method for contact for a survey type application, without linking that preference to this study. Taking this approach will provide a mechanism for looking at how closely preference matches usage. To the extent possible, given the small number of cases, we’ll also examine whether cooperation differs between those whose assigned mode and preferred mode are the same versus those who were assigned to a mode other than the one they reported as preferring.

In recruiting respondents for the first stage of testing, the recruiter will tell potential respondents that their participation is requested in one in-person visit, at which the pregnancy screener is administered and assignments to a response category are made, and up to three short follow-up interviews which they will complete at home using the mode to which they are assigned.

The second stage of testing does not require respondent recruitment since the testing will occur with a subsample of respondents in the NCS Vanguard Pilot locations.

Data Collection. For the first stage of testing, information gathered during the recruiting and pregnancy screening phases will determine assignments to the mode of contact for the first follow-up interview. Recruiters will ask all potential participants about their usage of both the web and texting for non-work related purposes. Women who indicate that they use either the web or text at least once a week for non-work related purposes are eligible to participate in the experimental assignment of mode of first contact:

• IVR; or

• mode for which they report the highest frequency of use for non-work purposes.

In the first stage, data collection will follow the protocols shown in Attachments B & C, but in order to have the automated methods available to include in the NCS Vanguard Pilot for stage 2 testing, the analysis from the usability testing will only reflect two contacts in the 6-week cycle and one contact in the 12-week cycle. Limiting the contacts in this first phase of testing will allow introduction of these methods into a subsample of the NCS Vanguard Pilot. This first stage will, however, continue with the remaining contacts in order to complete a full assessment that reflects respondent behaviors across multiple contacts.

The second stage of testing will use the full protocol for both the 6- and 12-week cycles and will continue until the NCS Vanguard Pilot ends these contacts, or if a point is reached at which a notable drop off in cooperation relative to the CATI cooperation rates to date is observed. The second stage will apply the same criteria in assigning women to mode.

After granting consent (Appendix D), participants will be asked to complete the pregnancy screener (Appendix E) and assigned to a response mode, as described above. Those assigned to the text messaging group will be asked to sign an additional consent (Appendix F). The follow-up scripts for each mode are shown in Appendices G – I.

Use of Results: The first and second stages of testing will use the same high-level design and contact protocol. A high-level design, showing each of the manipulations is attached (Appendices B & C). However, despite sharing the same high-level design across the two stages, each stage serves different research objectives and will provide different types of information about the follow-up contacts.

High-level research questions. The objective of the first stage of testing is primarily to identify major problems with the applications or the contact protocol. The focus is mainly usability of the applications, but also acceptance of the contact protocol as measured by:

• timeliness of response

• retention across contacts (6-week protocol only)

• responses to debriefing questions covering things such as their recognition of the first follow-up survey request, any perceived privacy concerns with the mode of contact, reasons for not responding (as applicable), and so on.

The high-level research questions for the second stage of testing are shown below to provide context for the ultimate use of the results of this study.

High-level research questions

1.	What results in a higher response rate – asking women to respond in the mode for which they indicate high frequency of use (text or web), or in an IVR?
2.	Does switching modes due to non-response improve response/retention at subsequent follow-up contacts?
3.	Does the frequency of contact affect response rate, by mode and round of follow-up (e.g., every six weeks versus every 12 weeks)? Does the effectiveness of switching mode for non-response differ by frequency of contact?
4.	What is the distribution of types of non-response by mode and round of follow-up?
5.	For non-response, what are the reasons for not starting or not completing the survey (follow-up debriefing with an interviewer)?
6.	(In response to debriefing questions) Do women report: Any concerns about security of information? Any concerns about or actual occurrence of other household members getting their intended survey? Any issues with the survey content?

Data Collection Burden:

Type of Respondent	Number of Respondents	Frequency of Response	Average Time per Response	Annual Hour Burden
In-person Interview Participants	200	1	0.50	100 hours
First Follow-up Participants	180	1	0.03	6 hours
Second Follow-up Participants	160	1	0.03	5 hours
Third Follow-up Participants	80	1	0.03	3 hours
Respondent Debrief Participants	30	1	0.08	3 hours
Non-Respondent Debrief Participants	50	1	0.08	4 hours
Total	700*			121 hours

* The total number of individuals is 200. There will be a total of approximately 700 contacts with those individuals.

IRB Approval: The study has received IRB approved via expedited review from the local Westat IRB, included as Appendix J – Westat IRB Expedited Approval.

Sensitive Questions: With the exception of questions regarding race/ethnicity and gender, the survey instruments do not contain sensitive questions. The OMB format for asking race and ethnicity questions will be applied.

Remuneration: Participants will receive a $50 cash incentive at the time of the in-person interview.

Appendices:

• Appendix A – Westat NCS Formative Research Recruiting Questionnaire

• Appendix B – Flowchart for 6-week Contact

• Appendix C – Flowchart for 12-week Contact

• Appendix D – Consent Form for Westat Formative Research Study

• Appendix E – Pregnancy screener

• Appendix F – Consent Form to Receive Text Messages

• Appendix G – Interactive Voice Response Phone Follow-up Script

• Appendix H – Text Message Follow-up Script

• Appendix I – Web E-mail Follow-up Script

• Appendix J – Westat IRB Expedited Approval

5