Supporting Statement B for
NIH Toolbox for Assessment of
Neurological and Behavioral Function (NIA)
April 11, 2011
Molly Wagster, Ph.D.,
Contracting Officer Technical Representative
NIH Toolbox
National Institute on Aging
Gateway Building, 350
7201 Wisconsin Ave
Bethesda, MD 20892
Telephone: 301-496-9350
Fax: 301-496-1494
E-mail: wagsterm@nia.nih.gov
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 6
B.1 Respondent Universe and Sampling Methods 6
B.2 Procedures for the Collection of Information 8
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 10
B.4 Test of Procedures or Methods to be Undertaken 11
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 12
List of Attachments
BATTERY OF TOOLBOX MEASURES
Sensation domain
Automated Audiometry from the NIH Toolbox (AANT)
Hearing Handicap Inventory
Words-in-Noise (WIN)
Odor Identification Ages 10-85
Odor Identification Ages 3-9
University of Pennsylvania Smell Identification Test (UPSIT)
Pain Assessment Proxy Ages 3-7
Pain Assessment Ages 8-12
Pain Assessment Ages 13-17
Pain Assessment Ages 18-85
Sucrose Preference Test
Toolbox Taste Test
Dynamic Visual Acuity (DVA)
Vision Health-Related Quality of Life Survey
Brief Kinesthesia
Pediatric Pain Scale
Tactile Discrimination Test
Cognition domain
Dimensional Change Card Sort
Bateria III Picture Vocabulary
Flanker
Imitation Based Assessment of Memory (IBAM)
List Sorting
Oral Symbol Digit Modalities Test
Pattern Comparison
Rey Auditory Verbal Learning Test (RAVLT)
Reading Recognition
Determining Handedness
Vocabulary Comprehension
Spanish Word Accentuation Test
Emotion domain
Instructions for Emotional Health Questionnaire
Emotional Health Questionnaires for Toolbox
Supplemental Emotional Health Questionnaires
Motor functioning domain
9 Hole Pegboard
Endurance 2 Minute Walk Test (2MWT)
Locomotion 4 Meter Walk Test
Grip Strength Dynamometry
Strength HHD Knee Extension
Balance Accelerometer Measure (BAM)
ADDITIONAL MEASURES
Initial Questionnaire - ADULT
Initial Questionnaire - PROXY and CHILD
Additional Somatosensation Questions
Additional Audition Olfaction Taste Questions
Cognition Information Form
Health Care Access and Utilization
Instrumental Activities of Daily Living (IADL) - Adults
Applied Cognition - Adult
Pediatric Perceived Cognitive Function (PCF) - Child
Pediatric Perceived Cognitive Function (PCF) - Proxy
PROMIS Pediatric Physical Function Upper Extremity
PROMIS Pediatric Physical Function Mobility
Pediatric Functional Assessment of Chronic Illness Therapy – Fatigue (FACIT-F)
PROMIS 29 - Participant
School Performance and Activities
Sociodemographic Form – Parent
Test Anxiety Scale (TAS)
Falls Efficacy Scale - International (Short FES-I)
EtOH Questionnaire
Education Adult Self-Report
Education Proxy Report for Children
RECRUITMENT MATERIALS
Recruitment Instructions, Scripts and Letters
REGULATORY DOCUMENTS
Certificate of Confidentiality
Northwestern University IRB Approval Letter
Consent Documents
OTHER DOCUMENTS
Team Member Contact List
NIH Privacy Act Review Memo
NIH Privacy Act Systems of Record 09-25-0200
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
The instruments in the NIH Toolbox are designed primarily for use in a Target Population with the following characteristics:
General non-institutionalized population dwelling in the community
Ages 3-85 years
Availability of a proxy respondent for pediatric population
Capable of following instructions given in English or Spanish and understanding the informed consent process
Adequate visual, auditory, vestibular, or motor functioning or availability of assistance or assist devices is required to complete some of the items in the test battery
This target population is estimated to include approximately all 283,764,466 civilian noninstitutionalized individuals ages 3-85 years in the United States. A total of 8,400 individuals and 4,500 parent-proxies will be sampled from this population. Distribution of the sample and the respondent universe across the strata are presented in Table B.1.1. The accrual vendor’s existing recruitment database of 492,008 subjects will be randomly sampled and screened for household members’ age, gender, race/ethnicity, education and primary language. We expect that approximately 15% of the eligible individuals will agree to participate and 85% of the selected participants will actually complete the tests successfully.
Table B.1.1. Distribution of Respondent Universe And Proposed Sample Across Strata
|
Universe1 |
Sample |
Total |
283,764,466 |
8400 |
|
||
Males |
139,361,273 |
4200 |
Females |
144,403,193 |
4200 |
|
||
English-speaking2 |
232,686,862 |
5850 |
Spanish-speaking2 |
31,214091 |
2550 |
|
||
Age 3 |
4,168,945 |
300 |
Age 4 |
4,122,392 |
300 |
Age 5 |
4,133,901 |
300 |
Age 6 |
4,072,774 |
300 |
Age 7 |
4,060,824 |
300 |
Age 8 |
4,104,423 |
300 |
Age 9 |
3,921,722 |
300 |
Age 10 |
3,922,622 |
300 |
Age 11 |
3,908,905 |
300 |
Age 12 |
3,946,112 |
300 |
Age 13 |
4,032,341 |
300 |
Age 14 |
4,071,443 |
300 |
Age 15 |
4,106,574 |
300 |
Age 16 |
4,191,736 |
300 |
Age 17 |
4,250,817 |
300 |
Age 18-29 |
49,965,595 |
600 |
Age 30-39 |
39,581,409 |
600 |
Age 40-49 |
43,459,875 |
600 |
Age 50-59 |
40,494,881 |
600 |
Age 60-69 |
27,275,851 |
600 |
Age 70-85 |
21,971,324 |
900 |
The NIH Toolbox sample frame (see tables below) will be draw from the Delve sample.The sample frame was derived from the 2010 Census (respective to each age group). The Delve database is updated in a process involving continuous recruitment. The Delve sample has been evaluated over the last two weeks and has been confirmed to meet the needs of our sample frame (including oversampling required to insure final completes) without further revision.. No revisions to the sample or frame are predicted given the very short accrual period to follow (<3 months).
Adults:
Age |
|
Race / Ethnicity* |
Educational Attainment |
18 - 29 |
19% H, 15% B, 4% A, 3% O |
16% Less, 29% HS, 55% More |
|
30 - 39 |
19% H, 14% B, 6% A, 3% O |
12% Less, 27% HS, 61% More |
|
40 - 49 |
14% H, 13% B, 5% A, 2% O |
11% Less, 31% HS, 58% More |
|
50 - 59 |
10% H, 12% B, 4% A, 2% O |
11% Less, 31% HS, 58% More |
|
60 - 69 |
8% H, 10% B, 4% A, 2% O |
13% Less, 32% HS, 54% More |
|
70 - 79 |
7% H, 9% B, 4% A, 1% O |
24% Less, 37% HS, 39% More |
|
80 - 85 |
|||
Total |
|
|
Children:
Age |
|
Race / Ethnicity* |
Highest Educated Parent’s Educational Attainment |
3 |
25% H, 15% B, 5% A, 5% O |
16% Less, 29% HS, 55% More |
|
4 |
25% H, 15% B, 5% A, 5% O |
16% Less, 29% HS, 55% More |
|
5 |
23% H, 14% B, 5% A, 5% O |
12% Less, 27% HS, 61% More |
|
6 |
23% H, 14% B, 5% A, 5% O |
12% Less, 27% HS, 61% More |
|
7 |
23% H, 14% B, 5% A, 5% O |
12% Less, 27% HS, 61% More |
|
8 |
23% H, 14% B, 5% A, 5% O |
12% Less, 27% HS, 61% More |
|
9 |
23% H, 14% B, 5% A, 5% O |
12% Less, 27% HS, 61% More |
|
10 |
20% H, 15% B, 4% A, 4% O |
12% Less, 27% HS, 61% More |
|
11 |
20% H, 15% B, 4% A, 4% O |
12% Less, 27% HS, 61% More |
12 |
|
20% H, 15% B, 4% A, 4% O |
12% Less, 27% HS, 61% More |
13 |
20% H, 15% B, 4% A, 4% O |
12% Less, 27% HS, 61% More |
|
14 |
20% H, 15% B, 4% A, 4% O |
12% Less, 27% HS, 61% More |
|
15 |
19% H, 16% B, 4% A, 4% O |
11% Less, 31% HS, 58% More |
|
16 |
19% H, 16% B, 4% A, 4% O |
11% Less, 31% HS, 58% More |
|
17 |
19% H, 16% B, 4% A, 4% O |
11% Less, 31% HS, 58% More |
|
Total |
|
|
Existing recruitment databases will be randomly sampled and screened for household members’ age, gender, race/ethnicity, education and primary language. This will be done by Delve, the marketing research firm which will conduct the data collection activities. The goal is to accrue 150 participants for each of the adult cells, defined by age, gender, and language; and 75 participants in each of the child cells, as shown in Tables B2.1 and B2.2. Since very few school age children speak Spanish as their dominant language, this sample will not include Spanish speakers age 8-17, but additional English speaking children will be recruited to maintain the overall sample size across ages. The single-year age increments in the children are designed to capture the rapid developmental change that occurs during childhood, while the adult age groups were similarly designed to correspond with accepted periods of transition and decline. The ultimate goal of this project is to provide normative reference values that will be acceptable to the epidemiological and clinical research community. . The data collected in this study will provide needed normative values for sensory, motor, cognitive, and emotional function that are relevant and generalizable to community based samples of children, adolescents, adults and older adults of both genders, with varying educational levels, from urban and suburban locales, for both English or Spanish preferred language populations. Therefore, this sample size per cell was selected based on the precedent of other norming studies, in particular the Wechsler Adult Intelligence Scale. This sample size also provides sufficient precision in the estimates of within-cell normative values. In many instances precision will be improved by combining English and Spanish-speaking cells and/or combining males and females within an age group. For certain tests, such as the Reading and Vocabulary components of the Cognition domain, it will be important to provide normative values separately for English and Spanish-speakers, whereas for Motor domain measures this distinction is not necessary and the language groups may be combined for increased precision. Similarly, some instuments may warrant gender-specific normative values while other instruments will not. For the smallest cells of size 75, the 95% confidence interval for the cell mean will be estimated with a precision of ± 0.23 standard deviation units. Cells of size 150 will have precision of ±0.16, size of 300 corresponds to precision of ±0.11, size of 600 corresponds to precision of ±0.08, and size of 900 corresponds to precision of ±0.06. This provides us with precision of ±0.20 or better for all but the smallest cell size. It is widely accepted in the minimally important difference literature on patient-reported outcomes that differences less than 0.20 standard deviation units (a small effect size according to accepted criteria) are unlikely to be meaningful differences. The cells that are rarest in the U.S. population are the Spanish-speaking child and elderly cells.
Table B2.1. Target Completes for Children by Cell
Age |
English |
Spanish |
Total |
||
Male |
Female |
Male |
Female |
||
3 |
75 |
75 |
75 |
75 |
300 |
4 |
75 |
75 |
75 |
75 |
300 |
5 |
75 |
75 |
75 |
75 |
300 |
6 |
75 |
75 |
75 |
75 |
300 |
7 |
75 |
75 |
75 |
75 |
300 |
8 |
150 |
150 |
0 |
0 |
300 |
9 |
150 |
150 |
0 |
0 |
300 |
10 |
150 |
150 |
0 |
0 |
300 |
11 |
150 |
150 |
0 |
0 |
300 |
12 |
150 |
150 |
0 |
0 |
300 |
13 |
150 |
150 |
0 |
0 |
300 |
14 |
150 |
150 |
0 |
0 |
300 |
15 |
150 |
150 |
0 |
0 |
300 |
16 |
150 |
150 |
0 |
0 |
300 |
17 |
150 |
150 |
0 |
0 |
300 |
Total |
1875 |
1875 |
375 |
375 |
4500 |
Table B2.2. Target Completes for Adults by Cell
Age |
English |
Spanish |
Total |
||
Male |
Female |
Male |
Female |
||
18-29 |
150 |
150 |
150 |
150 |
600 |
30-39 |
150 |
150 |
150 |
150 |
600 |
40-49 |
150 |
150 |
150 |
150 |
600 |
50-59 |
150 |
150 |
150 |
150 |
600 |
60-69 |
150 |
150 |
150 |
150 |
600 |
70-79 |
150 |
150 |
150 |
150 |
900 |
80-85 |
150 |
150 |
|||
Total |
1050 |
1050 |
900 |
900 |
3,900 |
We plan to stratify the recruitment databases into strata corresponding to testing cells defined by age, sex, and language. We will select a sample of persons from those databases for participation in the study. We will select a conditionally equal probability sample of persons from each stratum. Interviewers will contact the households to recruit the selected persons into the study. We expect to potentially recruit multiple persons from some households, particularly from larger households containing members of the rarer cells, although we are not explicitly controlling for the number of persons per household.
Participants will complete a small portion of the assessment on paper prior to their scheduled visit. Trained Delve technicians will then administer the Toolbox measures to adult participants, child participants, and proxies, as appropriate. For one-week test-retest reliability testing, 375 children (and their parents for proxy measures) and 375 adults will be randomly selected and asked to schedule another appointment at which the Toolbox measures will be re-administered. Another 375 children (and their parents for proxy measures) and 375 adults will be randomly selected to be contacted by telephone to schedule a 3-month follow-up appointment to evaluate change in the Toolbox measures.
We have assumed that approximately 85% of the selected participants will actually complete the tests successfully. To achieve this response rate, in addition to offering geographically appropriate compensation, we will provide scheduled respondents with a detailed packet of information regarding what they can expect on the day of testing, directions to the testing site, and a telephone number to call if they have questions regarding the study (see Attachment 60). They will also receive a telephone call reminding them of their appointment 2-3 days in advance and offering them the opportunity to reschedule if the scheduled appointment is no longer convenient. Respondents who are unable to travel to the testing location due to the financial burden or unavailability of transportation will be provided compensation for travel. In addition, selected respondents who do not keep their testing appointment will receive a call offering them the opportunity to reschedule. These techniques combined will serve to maximize respondent cooperation. The 15% that do not complete the study may drop out in a variety of ways—not completing the telephone questionnaire and confirming eligibility, not giving parental consent, not showing up for the tests, etc. The tests require the participants to go to a testing site for about three hours of tests, which is not conducive to a higher response rate, even with compensation. This response rate is consistent with the OMB minimum acceptable response rate, and furthermore, the design provides an adequate number of respondents in rare population cells to accurately norm the measures.
We will do a nonresponse bias analysis. The sample will be selected disproportionately to achieve the target numbers within each cell, and the cases for a cell will not be combined with cases from other cells (with the possible exception of Spanish-speaking elderly). Therefore, the nonresponse bias of interest is strictly within each cell. We will compare the weighted respondents to the total U.S. for the same age, gender, and language for a very limited set of demographics available, such as geographical region, household income, and household size.
All of the measures administered as part of the assessment have undergone extensive review as part of their development and no additional testing of the procedures is planned under the scope of this proposed survey. The Toolbox measures are located in Attachments 1-38, and the additional measures are located in Attachments 39-59.
Role |
Name |
Affiliation |
Telephone number |
Consultation on design aspects |
Jennifer Beaumont, MS |
NU-MSS |
312-503-9805 |
Richard Havlik, MD, MPH |
Westat |
301-652-3427 |
|
Ron Hays, PhD |
UCLA |
310-794-2294 |
|
Kay Savio |
Delve |
314-851-3116 |
|
Kim Reale |
Delve |
314-851-3124 |
|
Interviewing contractor |
Judy Golas |
Delve |
740-392-9436 |
Data analysis |
Jennifer Beaumont, MS |
NU-MSS |
312-695-1324 |
Richard Havlik, MD, MPH |
Westat |
301-503-9805 |
|
Ron Hays, PhD |
UCLA |
310-794-2294 |
|
Devin Peipert, MSW |
NU-MSS |
312-503-3647 end_of_the_skype_highlighting |
|
Katy Wortman, MSW |
NU-MSS |
312-503-3609 end_of_the_skype_highlighting |
|
James Burns, MS |
NU-MSS |
312-503-3380 end_of_the_skype_highlighting |
|
Seung Choi, PhD |
NU-MSS |
312-503-9803 end_of_the_skype_highlighting |
|
Jin-Shei Lai, PhD, OTR/L |
NU-MSS |
312-503-3370 end_of_the_skype_highlighting |
|
Richard Gershon, PhD |
NU-MSS |
312-503-3453 end_of_the_skype_highlighting |
|
Natalie McKinney, BA |
NU-MSS |
312-503-3909 end_of_the_skype_highlighting |
|
Tracy Podrabsky, BS |
NU-MSS |
312-503-3908 end_of_the_skype_highlighting |
|
Jamie Griffith, PhD |
NU-MSS |
312-503-3538 end_of_the_skype_highlighting |
|
Elizabeth Hahn, MA |
NU-MSS |
312-503-9804 end_of_the_skype_highlighting |
1 US Census Bureau Civilian Noninstitutionalized Population estimates for 5/1/2009
2 Estimated using percentages reported in Census 2000 Brief: Language Use and English-Speaking Ability: 2000
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |