A.5 LOI2-QUEX-5 Bayley 3 Approved Project Protocol

Attach 5: Project Protocol

Bayley-3 Short Form

PROJECT PROTOCOL

OVERVIEW

Background

This study continues work begun under contract #8450.34.01, in which Westat was a subcontractor to the University of Texas Health Science Center, San Antonio. Under this contract, Westat used Item Response theory (IRT) analysis to identify the item content of each age-specific Bayley-3 Short Form. This contract is now closed and the data collection and psychometric analyses will be conducted under contract # 8868 under which Westat provides support to the National Children’s Study.

Purposes and Objectives

This study has two purposes. The first purpose is to examine the feasibility, acceptability, and cost of implementing a short form of the Bayley Scales of Infant and Toddler Development, Third Edition (i.e., the Bayley-3 Short Form) at 6-, 12-, 18-, 24- and 36-months in the National Children’s Study (NCS). The second purpose is to collect data in order to develop standardized norms for the short form using Item Response Theory (IRT) analysis so that Bayley-3 Short Form IRT scores can be generated for each child as if the full Bayley-3 had been administered. The Bayley-3 Short Forms will serve as strong baseline measures to anchor the NCS, enable future comparisons with various outcome measures, and to parse the effects of various exposures on children’s development. In examining developmental growth it is strongly advisable to have multiple data points using the same measure, in order to conduct growth curve modeling and to examine children’s developmental trajectories. Having multiple age versions will also provide the NCS with the flexibility to obtain strong baseline measures at any of the data collection points, for example perhaps a Bayley-3 Short Form cannot be obtained at 6- or 12 months for a particular child (or group of children) but can be obtained at 24 months. This instrument is also intended to be feasible for reliable administration in the field by data collectors with no background in neurodevelopmental assessment or child development.

METHODS

Research Plan

This study will take place in a series of steps, with each subsequent step dependent on the demonstration of feasibility, acceptability, and cost effectiveness of the administration of the Bayley-3 Short Form. At each step we will obtain timings on each of the subtests (e.g., cognitive, language, and motor), review descriptive statistics to ensure that there is sufficient variability to uncover potential systematic group differences, compute estimates of reliability (using classical test theory, i.e., alpha), and collect information on cost per administration.

The first step is to design and implement a small study to demonstrate the feasibility, acceptability, and cost effectiveness of developing a short form of the Bayley-3 at 6- and 12-months. These ages were selected for two major reasons. First, a short Bayley-3 should be obtained as a baseline measure as early as possible in the study. Second, these ages were chosen because of chronological proximity of NCS data collection events. As part of this step we will obtain timings on each of the subtests (e.g., cognitive, language, and motor), compute estimates of item reliability (using classical test theory, i.e., alpha), and collect information on cost per administration. Data will be collected from 125 participants at each age group, 6- and 12-months.

If step 1 demonstrates appropriate feasibility, acceptability, and cost effectiveness in the early months of age, then we will proceed to step 2 which has two parts. The first part will involve a sample of 125 24-month olds and 125 36-month-olds. After feasibility, acceptability, and cost effectiveness have been demonstrated at 24- and 36-months of age, we will add a sample of 125 18-month olds.

Step three depends on the success of steps 1 and 2. If these steps demonstrate feasibility, acceptability, and cost effectiveness, then we will augment the data collection in order to add 75 cases at each age, bringing the total sample size to 1000, with 200 participants at each age. Two hundred participants at each age provides the minimum number of cases per group required for Item Response Theory analysis in order to achieve the goal of generating IRT standardized norms and scores. As a rule of thumb, IRT requires 250 data point per item in order to obtain item parameters and thereby generate IRT norms and IRT standard scores. Because many Bayley items overlap across the ages, only 200 cases per age are required. Collecting enough cases to conduct IRT norms is a necessary final step in order to equate the Bayley-3 Short Form to the long Bayley-3. Equating the Bayley-3 Short Form to the full form of the Bayley-3 is a necessary step because it will enable the NCS to claim that a score on the Bayley-3 Short Form is the same “as if” the full Bayley-3 had been administered. The equating of the short forms with the full Bayley-3 will enable the comparison of Bayley-3 Short Form scores with the publisher’s full Bayley-3 standardized norms.

In sum, we will recruit a total of up to 1050 non-NCS participants (the goal being 1000 cases plus up to 50 additional cases to allow for “unusable data”), for n = 200 at five age ranges, 5 month to 7 months 30 days, 11 months 0 days to 13 months 30 days, 16 months 0 days to 20 months 30 days, 22 months 0 days to 27 months 30 days, and 33 months 0 days to 43 months 30 days.

Study Design

The study design involves a single one-time-visit (up to 65 minutes maximum at the older ages because the test requires more items to cover children’s expanding skills). The age-appropriate Bayley-3 Short Form will be administered to children (with parents present) by trained staff. The estimated time for the child's participation on the Bayley-3 Short Form is 30 to 50 minutes, depending on the child's age, plus up to an additional 15 minutes in order to establish rapport with the child and allow time for the parent or guardian to review the consent form and ask any questions. The entire session should take less than 45 to 65 minutes, on average

Study Population

Our population is normal-healthy infants and toddlers who were born full-term.

Inclusion Criteria: Infants and toddlers who match the age ranges of each age-specific Bayley-3 Short Form will be included: infants and toddlers from 5 months 0 days to 7 months 30 days, 11 months 0 days to 13 months 30 days, 16 months 0 days to 20 months 30 days, 22 months 0 days to 27 months 30 days and 33 months 0 days to 43 months 30 days. Infants and toddlers included will also meet the following criteria: no medical complications at birth or currently, and not diagnosed or receiving treatment for mental, physical or behavioral difficulties.

Exclusion Criteria: Infants and toddlers who are outside the age windows of our 6-, 12, 18-, 24-, and 36-month Bayley-3 Short Forms are not eligible for participation. Also excluded are infants and toddlers who were born earlier than 36 weeks of gestation or later than 42 weeks, even if their chronological age falls within our age windows. Non-English speaking families will not be eligible because the publisher norms are standardized on an English-speaking sample. Non-English speaking will be determined on the basis of primary language spoken in the home. Current NCS participants will not be eligible.

Recruitment Procedures:

To obtain this sample, Westat will purchase names and telephone numbers of the families of age-appropriate children from a marketing firm (e.g., InfoUSA). Parents will be contacted by mail initially with telephone follow-up by a research assistant within 7 days. Because this telephone call is for the purpose of government-sponsored research, it will be exempt from “Do Not Call” limitations. The cover letter and accompanying flyer with study contact information are in Appendix 3, Exemplar Cover Letter and Appendix 2, Exemplar Recruitment Flyer.

If necessary, we will also recruit children through announcements on the Westat web page, on WesInfo (Westat’s intranet) by posting the flyer and, possibly, the NCS Facebook page, at the discretion of the Project Office. If necessary, flyers will also be placed in local child care centers, libraries, and other locations that focus on children (e.g., community recreation centers), as well as community newsletters and newspapers.

It is anticipated that parents or guardians of potential participants who are recruited from flyers, or local media will contact the research assistant (TBD) by telephone. If convenient for the parent/guardian at that time, a screening questionnaire (See Appendix 1, Exemplar Screener.) will be completed by telephone to see if inclusion criteria for study participation are met. In addition, the parent will be asked to provide demographic information to ensure that our sample will match the publisher’s standardization dataset demographics. The demographic information provided by parents in the phone screener will also be used in the psychometric analysis to rule out item bias and to help ensure that the Bayley-3 Short form is the psychometric equivalent of the full Bayley-3 scales. If it is not convenient to complete the screener at that time, a follow-up telephone call with the parent will be scheduled. At the end of the screener, the research assistant will say (per the instructions at the end of the screener): “I will forward this information to our study staff and they will see if your child is eligible to participate.” The results of the screener will be discussed with the PI and if study-eligible, then the research assistant will call the parent or guardian to report that the child is eligible and a testing session can be scheduled. The wording for this is provided at the end of the screener, as well as for ineligibility.

Protected Health Information (PHI) will not be used to identify or recruit potential participants.

Risk-Benefit Analysis

The risks to children from these activities are minimal and no pain is involved. Children may become fussy or may cry, but this is normal for children this age. However, if a child becomes fussy and does not want to continue, we will stop the session. Although there are no direct benefits for being in this study for the parent/caregiver or child, there are indirect benefits. Participants will help us create a new tool that shows what and how children learn that can be used to study the well-being of the nation’s children in future national studies. Parents and caregivers also report that they enjoy watching their children learn and use new skills.

Consent Process

At the time of the scheduled appointment and before administering the Bayley-3 Short Form is begun, we will obtain informed consent from parents or legally authorized representatives for their child’s participation. (Please see Appendix 4, Exemplar Consent Form.) Parents or guardians will be provided with opportunity to ask questions about their child’s participation before proceeding. Sufficient time has been built into the study session so that parents/guardians have time to do this. It will be emphasized that testing will stop if the child becomes distressed or uncooperative and that there will be no negative consequences to withdrawing the child’s participation.

Confidentiality

The screening questionnaire information and the completed Bayley-3 Short Form scoring sheets, which contain the de-identified demographic information, will be kept in a locked file cabinet in a locked office. Each subject will be given a unique subject ID number and demographic information from the screening questions will be translated into numerical information. Item-level test data is scored as 1 or 0. The contact information (parent/caregiver address and phone number) will be kept in a separate spreadsheet which only the PI and a research assistant will have access to. The screening information and Bayley-3 Short Form item level data will be in a separate spreadsheet with will include no identifiable information and all cases will be identified by the study ID number.

ANALYSES

Data Entry, Clean-Up and Data Analysis

The Bayley-3 Short Form data collection will comply with the NCS data use and data security plan. The NCS data security plan was last approved on 1/25/2013 by the NCS Mission Assurance Team (per Donna Farrantello, Information Security Coordinator, Westat NCS-Related Contracts).

The Bayley-3 Short Form data will be stored in two separate Excel spreadsheets. One spreadsheet will include identifiable data that include the name of the parent (or legally authorized representative), name of the child, and contact information (address, telephone number), and a crosswalk to a case number used to identify cases in the other spreadsheet. The de-identified spreadsheet will not include any names or identifiable information that could reveal a child’s identity. The de-identified spreadsheet will include typical demographic information (income level, ethnicity, parent education), translated into numeric form, that is necessary for subsequent analyses to rule out test bias. The de-identified spreadsheet will also include the child’s scores on the Bayley-3 Short Form items: 1 = credit, 0 = no credit.

These two spreadsheets will be stored on the Secure NCS Analyst Zone and will be accessible only to the psychometrician conducting the IRT analysis, the PI and one research assistant who will be entering the data.

Data Cleaning

Data will be reviewed to ensure that basal and ceiling rules were followed, and a rule will be set to determine that a complete Bayley-3 Short Form was obtained for each subscale (typically 66% of all the items that should have been administered). IRT analyses will be conducted to calibrate and equate the Bayley-3 Short Form items to the publisher full form standardization item data set and to obtain IRT reliabilities. The goal is to establish that the Bayley-3 Short Form administration is the equivalent of (i.e., equate to) the publisher dataset from the standardization of the full Bayley-3.

Psychometric Analyses and Report

These cleaned and de-identified data will be analyzed by an Item Response Theory analysis (IRT) expert who will be able to generate IRT scores for this shortened Bayley that are the equivalent of scores that would have been obtained on the full Bayley. At that time, a psychometric report will be written and de-identified data files will be delivered to the PO, along with an IRT program that the NCS will use to convert shortened Bayley scores into the IRT equivalents of the full Bayley-3.

Incentives

To thank them for their time, study participants will receive a $25 monetary incentive upon completion of the assessment session. The study participants also will receive an age-appropriate toy.

Licensure Agreement

Westat applied for a license agreement with Pearson, the publisher of the Bayley Scales of Infant and Toddler Development, Third Edition, in order to develop IRT standard scores and IRT standardization norms for the Bayley-3 Short Form. This application has been approved.

1