Sub-study1 miniSSB Test

Mini Supporting Statement B For

“Questionnaire Cognitive Interviewing and Pretesting (NCI)”

0925-0589, Expiration Date 04/30/2014

Title of Sub-Study: Reliability of Computer Adaptive Tests (CAT) Study for the NIH-SSA Collaboration to Improve Disability Determination

Daniel Hobbs, Management Analyst, Epidemiology & Biostatistics Section, Rehabilitation Medicine Department, National Institutes of Health Clinical Center, NIH

Section B

B1. Respondent Universe and Sampling Methods

A sub-contracted survey firm, YouGov, will administer two assessments of physical functioning and behavioral health to a normative study population of US adults and self-reported permanently work disabled US adults for the initial contact. Individuals in each of these two samples will be contacted a second time, for a second administration of the same instruments. Respondents will also be administered the legacy assessment, Veterans Rand 12-Item Health Survey (VR-12) at each time point, as well as the physical function and behavior health function computer adaptive tests (CATs). All assessments will be administered electronically through the company’s website. The study participants will be an opt-in pool of voluntary respondents, recruited through the standard practices of private survey firms.

B2. Procedure for Collection of Information

The two developed assessments of physical functioning and behavioral health (Attachments A, B, and C), along with the VR-12 assessment will be administered to participants twice (Attachment E). The second time point will be administered seven (7) days following the initial administration. Administration will occur on a rolling basis, to ensure that all study participants are tested at a uniform interval. This second series of tests will allow for analysis of any changes in scoring, specifically assessing the ability of the instruments to consistently score an individual’s function.

To ascertain those individuals whose functional status changes between administrations of the two tests, the legacy instrument VR-12, considered the current ‘gold- standard’ in functional testing, will be administered to all survey respondents. This will allow researchers to account for expected variations in CAT scores among those whose functional status has changed between the administrations of the two rounds, differentiating these scores from variations in scores due to limitations of the CAT instruments.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Based on outcomes of previous studies with a similar or greater number of respondents, and the previous performance of the subcontractor, there is minimal concern about nonresponse rates.

B4. Test of Procedures for Methods to be Undertaken

This study is part of a larger series of studies that are being conducted between the SSA and NIH. Three projects have already received OMB approval¹ and another project received NIH Clinical Exemption (CE #2013-07-001). In February, 2013, OMB recommended this study be submitted under Dr. Gordon’s Willis’ formative generic. The physical functioning and the behavioral health questions have undergone user simulation testing (OMB No. 0925-0642-33). This is the final formative study planned as part of the development of the physical function and behavioral health CAT instruments with the SSA.

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Boston University-Health and Disability Research Institute (BU-HDR) will be responsible for the analysis of the data and providing the final report to NIH.

List of instruments, instructions, and scripts submitted with this request:

Attachment A: All Behavioral Health CAT Items

Attachment B: All Physical Functioning CAT Items

Attachment C: CAT Instrument initial screenshot

Attachment D: Expedited IRB Review approval

Attachment E: VR-12 Legacy Assessment

Attachment F: IRB Approved Consent

1 The three projects include:

• OMB No. 0925-0659, “The SSA-NIH Collaboration to Improve the Disability Determination Process: Validation of IRT-CAT Tools (CC)”, approved on 6/12/2012;

• OMB No. 0925-0642-33 “User Simulation Study for NIH-SSA Collaboration to Improve Disability Determination”, approved on 11/7/2013; and

• OMB No. 0925-0642-35, “Cognitive Interviewing for Item Bank Fields in a Computer Adaptive Testing Instrument,” approved on 12/31/2013.

2