NSDUH OMB Generic Clearance_2017-2020 SS A_Amended

9

2017-2020 National Surveys on Drug Use and Health:

Methodological Field Tests

SUPPORTING STATEMENT

A. JUSTIFICATION

1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ), is requesting approval for an extension to the generic clearance for methodological field tests for the National Survey on Drug Use and Health (NSDUH) (OMB No. 0930-0290) which expires on May 31, 2017. NSDUH, sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA), is a national survey of the U.S. civilian, non-institutionalized population aged 12 and older. NSDUH has been conducted on a periodic basis from 1971-1988, and annually since 1990. The conduct of NSDUH (OMB No. 0930-0110) is paramount in meeting a critical objective of SAMHSA’s mission to maintain current data on the incidence and prevalence of substance use and mental health problems in the United States.

NSDUH is authorized by Section 505 of the Public Health Service Act (42 USC 290aa4 – Data Collection).  Section 505 specifically authorizes annual data collection for monitoring the incidence and prevalence of illicit substance use and mental health problems, as well as the abuse of licit substances in the United States population.

NSDUH data are used by SAMHSA, the National Institute on Drug Abuse (NIDA), the Centers for Disease Control and Prevention (CDC), the Office of National Drug Control Policy (ONDCP), other federal agencies, Congress, and various state and local government agencies interested in the incidence and prevalence of substance use and mental health statistics. The NSDUH questionnaire asks the minimum information necessary to meet the needs of federal policymakers and the substance abuse research, prevention, and treatment communities. In conjunction with other data sources, NSDUH data are used to:

• design prevention programs;

• respond to inquiries on the extent of substance use;

• estimate treatment need;

• study the social and economic impact of substance abuse;

• identify the correlates of substance use;

• evaluate the overall impact that federal and state programs have on drug demand and reducing youth substance use;

• assess and improve outcomes of prevention and treatment services;

• measure program performance and improvement, including National Outcome Measures, Government Relations and Public Affairs (GRPA), and other requirements; and

• identify areas where serious substance abuse problems exist and provide assistance to states to help them develop and adopt targeted responses for those problems.

In order to continue producing current data, CBHSQ must periodically update NSDUH to reflect changing substance use and mental health issues. CBHSQ plans to revise the NSDUH questionnaire to address changing policy and research data needs, and to potentially modify the survey methodology to improve the quality of estimates and the efficiency of data collection and processing. In addition, CBHSQ may implement other changes to NSDUH that could be tested during 2017 through 2020.

These methodological tests will continue to be designed to examine the feasibility, quality, and efficiency of new procedures or revisions to existing survey protocol. Specifically, the tests will measure the reliability and validity of certain questionnaire sections and items through multiple measurements on a set of respondents; assess new methods for gaining cooperation and participation of respondents with the goal of increasing response and decreasing potential bias in the survey estimates; assess customer satisfaction; and assess the impact of new sampling techniques and technologies on respondent behavior and reporting. Research will involve focus groups, cognitive laboratory testing, and field tests. (Details are included under “2. Purpose and Use of Information”.)

Methodological testing activities are expected to also focus on assessing questionnaire modifications through cognitive interviews and improving response rates among persons residing in controlled access communities (locked apartment buildings, gated communities, college dormitories, etc.) and other hard-to-reach populations. Other activities currently under consideration are targeted at assessing the characteristics of nonrespondents and determining the feasibility of alternative sample designs and modes of data collection. Some studies may be combined to introduce efficiencies. Particular attention will be given to minimizing the impact of design changes so survey data continue to remain comparable over time. If these tests provide successful results, current procedures or data collection instruments may be revised.

This submission is for generic approval for these methodological testing activities. As these tests are developed, their materials will be submitted to OMB for an expedited review. This will enable CBHSQ to continue methodological testing activities in a timely manner, given the tight data collection schedule. CBHSQ requests that OMB review the individual submission on an expedited basis and provide comments or approve the request within two weeks of receipt.

2. Purpose and Use of Information

The methodological field tests will assess the potential effectiveness of proposed revisions to the NSDUH data collection instruments and procedures, and will determine their impact on trends before implementation. The intent of each study will be to assess, maintain, or improve the overall quality of the NSDUH data. The individual studies to be submitted under this clearance will not be designed to produce population estimates. The methods proposed for use under this generic clearance are briefly described below:

Focus Groups

Focus groups have been a useful tool in developing new questions and materials for NSDUH. They are useful as an early step in exploring a new issue or gathering opinions about a topic that has social relevance. Under previous generic clearance, focus groups were conducted to assess potential changes in the NSDUH contact materials. As new topics and requests arise frequently, CBHSQ anticipates the need for additional focus groups as one tool used in the efforts to be responsive to changing times.

Cognitive laboratory testing

New questions and modules are introduced into the NSDUH questionnaire on a regular basis. Currently, they undergo cognitive laboratory testing to assess problems with question wording, meaning and flow within the questionnaire. Potential groups recruited for cognitive testing include prescription drug users, clients of drug treatment centers, adolescents and members of the general public. Under the current generic clearance, cognitive testing was utilized to test questions designed to produce prevalence estimates of substance use disorders (SUD) based on the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) for NSDUH. Cognitive testing was also conducted under the current clearance to evaluate whether the use of Text-to-Speech (TTS) software, designed to read questions to respondents, has any effect on the comprehension of specific survey items or administration of those items compared to the human voice. Final reports from both the DSM-5 and TTS tests are included as appendices with this submission (Appendix A, 2015 National Survey on Drug Use and Health: DSM-5 Final Report; Appendix B, 2015 National Survey on Drug Use and Health: Text To Speech Investigation and Report). Since federal requirements dictate OMB approval of studies dealing with 10 or more human subjects, the amount of testing that can be completed in a given survey year is limited due the time constraints involved in obtaining clearance. NSDUH annual cognitive testing would benefit from more time for development. Usability testing may also be conducted in a laboratory setting to explore different data collection modes, new software and/or hardware.

Field Tests

For the purposes of this clearance, field tests are defined as small data collections of 500 cases or less, designed to assess modifications in the survey instrument and/or data collection procedures. Field tests will incorporate the findings of any pre-tests and assesses the impact of the changes on a larger scale. The tests will provide a more in-depth examination of context effects associated with questionnaire changes with a small subset of the NSDUH population. They also present the opportunity to study how data collection protocol and materials changes can potentially affect response rates. Under the current generic clearance, a field test of 30 interview cases was completed to further assess the impact of TTS on interview administration time and detect any issues not identified during the cognitive interviews noted above. Field tests will involve administering the entire survey, including any new questions and procedures, to a random sample of respondents. Several potential design changes and methodological investigations have been proposed that would require a field test:

• Additional examination of the impact of the American Psychiatric Association’s adoption of DSM-5 criteria for mental disorders on the NSDUH.  This work may include exploring how DSM-5 revisions affect Major Depressive Disorder (MDE), SUD (and correspondingly treatment need for SUD), and Any Mental Illness (AMI) and Serious Mental Illness (SMI) estimates.

• Adaptive sampling techniques could be investigated to increase the yield of the drug-using population.

• A nonresponse follow-up study may be conducted in which a subset of selected respondents who initially did not participate in NSDUH are re-contacted. An incentive would be offered for these individuals to complete the interview and to provide information regarding the reasons for refusal, as well as information on their drug use and other characteristics to allow for the assessment of bias.

• A study could be conducted to assess the feasibility of obtaining interviews from three respondents within a given household where three or more potential respondents reside (currently up to two persons within a household can be selected).

• The feasibility of various longitudinal-type survey designs (e.g., following persons over time, rotating panel designs, etc.) could be assessed.

• “Responsive designs” could be investigated where an optimal call limit on screening and interviewing households is determined based on the examination of collected survey data

• New computer hardware and software may be tested prior to implementation.

• Alternative sampling frames may be explored.

• A pretest may be warranted to test modified procedures or to capture timing data for new questions in a given survey year. It would be conducted among a small subset of the NSDUH population, approximately 200 cases.

Field tests would be employed to determine the feasibility of these and possibly other design modifications with a small subset of the NSDUH population. They would be used to determine the potential benefits and costs of these changes if they were adopted in the main study.

3. Use of Information Technology

Information will be collected through the use of face-to-face interviews and/or focus groups, telephone interviews, self-administered questionnaires including customer satisfaction surveys, or clinical evaluations, depending upon the subject matter being addressed. The face-to-face interviews and self-administered questionnaires will be conducted using computer-assisted interviewing (CAI). The main NSDUH study has been administered via CAI since 1999.

4. Efforts to Identify Duplication

Before each new methodological test is developed, CBHSQ will review existing literature on the proposed topic, and consult with outside experts to evaluate available information in similar studies with comparable populations.

5. Involvement of Small Entities

The methodological tests will not include small businesses or other such entities as respondents.

6. Consequences If Information Collected Less Frequently

CBHSQ is responsible for providing quality, timely data to the public on an annual basis. Methodological tests are necessary to keep up with changes in substance use and mental health without affecting trend measurement. Therefore, timely methodological tests are key for NSDUH data quality. Prior to each methodological test, a separate clearance memo (under this generic clearance) will be presented to OMB for review of each test.

7. Consistency With the Guidelines in 5 CFR 1320.5(d)(2)

CBHSQ recognizes the need to collect information in a manner that places minimal burden on each respondent. Therefore, when CBHSQ recruits prospective participants for each methodological test, they will explain the purpose of the study, the approximate length of time that it will take, and the voluntary nature of participation. All efforts will be made to keep the data collection instrument for each test short and well focused. This data collection is fully consistent with 5 CFR 1320.5(d)(2).

8. Consultation Outside the Agency

A number of experts on survey methodology, substance abuse and mental health have provided consultation on key issues related to recent changes to the questionnaire and methodology currently employed on NSDUH. Consultations with experts in these fields will continue as methods study development progresses. Consultants will be identified based on the topic of the individual submissions (i.e., each methodological study) and their names and contact information will be provided in the respective clearance packages.

A Federal Register notice was published on December 22, 2016 (Vol. 81, No. 246, pages 93948-93949). There were X comments received.

There are no unresolved issues resulting from any consultation at this time.

The following persons are the current consultants on the main NSDUH study:

Paul Beatty, Ph.D

Chief, Center for Survey Measurement

U.S. Census Bureau

Wilson Compton, MD

Deputy Director, National Institute on Drug Abuse

National Institutes of Health

Don Dillman, Ph.D

Regents Professor, Department of Sociology

Deputy Director, Research and Development in Social and Economic Science Research Center

Washington State University

Benjamin Druss, MD

Professor, Health Policy and Management, Behavioral Sciences and Health Education

Rosalyn Carter Chair in Mental Health

Emory University

Barry Graubard, Ph.D

Biostatistics Branch, Department of Cancer Epidemiology and Genetics

National Cancer Institute

Christopher Jones, Pharm.D

Director, Division of Science Policy

Assistant Secretary for Planning and Evaluation

Sharon Lohr, Ph.D

Vice President, Westat

Silvia Martins, Ph.D

Associate Professor, Department of Epidemiology

Columbia University Mailman School of Public Health

Kathleen Merikangas, Ph.D

Senior Investigator

Chief, Genetic Epidemiology Research Branch, Intramural Research Program

Nation Institute of Mental Health

Mark Olfson, MD

Professor of Psychiatry

Columbia University Medical Center

Patrick O’Malley, Ph.D

Senior Research Scientist

Survey Research Center, The Institute for Social Research

University of Michigan

University of Maryland, School of Public Affairs

Steve Thompson, Ph.D

Shrum Chair in Science

Professor of Statistics

Simon Fraser University

Gordon Willis, Ph.D

Cognitive Psychologist, Office of the Associate Director of the Applied Research Program

National Cancer Institute

9. Payment to Respondents

Survey research literature suggests that monetary incentives have a strong positive effect on response rates and no known adverse effect on data quality. In some cases, the use of incentives has reduced differential response rates and nonresponse bias. Any proposal to offer an incentive to respondents to offset their cost of participation will be presented in the materials for each expedited review.

10. Assurance of Confidentiality

Concern for the confidentiality and protection of respondents’ rights has always played a central part in the implementation of NSDUH and will continue to be given the utmost emphasis. Information provided by respondents will only be used by authorized personnel for statistical purposes and cannot be used for any other purpose. Prior to any data collection, respondents will be advised of the following: the nature of the activity; the purpose and use of the data collected; SAMHSA sponsorship; and the fact that participation is voluntary at all times. Since responses are voluntary, respondents will be assured there will be no penalties if they decide not to respond, either to the information collection as a whole or to any particular questions.

On November 9, 2006, OMB approved CBHSQ as a statistical unit. As a result, CBHSQ now adheres to the Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA) implementation guidelines in their sponsored surveys, including NSDUH. CIPSEA provides a uniform set of confidentiality protections to all individually identifiable data collected for statistical purposes under a pledge of confidentiality. Under CIPSEA, penalties are imposed for willfully disclosing information to a person or agency not entitled to receive it; unlawful disclosure could be considered a class E felony with up to 5 years imprisonment or fines not to exceed $250,000.

As CIPSEA agents, all contractor staff complete an annual CIPSEA training and sign either a notarized Confidentiality Agreement (Attachment A) or a notarized Data Collection Agreement (Attachment B) – each specific to an individual’s role on NSDUH – certifying they will keep all respondent information confidential.

As a further assurance of confidentiality, all presentation of data in reports will be in aggregate form, with no links to individuals being preserved. Reports will only be used by the project staff for research purposes and for the development of specific data collection questions and procedures. There will be no publication of estimates from the methodological tests.

Although some personal information will be collected, data will not be retrieved by personal identifiers during data analysis and data file preparation, and thus the Privacy Act does not apply to these activities.

11. Questions of a Sensitive Nature

Some studies may require the inclusion of people who match the characteristics of the target population for specific questions. This sometimes requires asking a question about race/ethnicity, income, education, and/or drug abuse or mental health problems on the initial screening questionnaire used for recruiting. Potential participants are informed that the reason these questions are asked is to make sure CBHSQ speaks with the appropriate people for each particular test. Again, respondents will be assured the information is voluntary and will be handled in a confidential manner.

Since NSDUH collects data on drug use and mental health, some methodological tests may involve asking questions about (or discussing) personal experiences with such problems. Questions of this nature require some sensitivity in how they are worded and approached. In face-to-face data collections, every attempt will be made to ensure the interview is conducted in as private a setting as possible.

Raw data from data collections that include sensitive information (for example, screening questionnaires, paper cognitive interviewing questionnaires and audio tapes) will not be retained once the data have been extracted and aggregated; nor will the information become part of a system of records containing permanent identifiers that can be used for retrieval.

12. Estimates of Annualized Hour Burden

The number of respondents to be included in each field test will vary, depending on the nature of the subject being tested and the target population. However, the total estimated response burden is 8,225 hours. This estimate is based on our previous generic clearance submissions and activities anticipated for the next several years. The exact number of subjects and burden hours for each test are unknown at this time, but will be clearly outlined in each individual submission. The table below, however, describes the anticipated burden for each of the major testing activities for which generic approval is being tested.

Estimated Burden for NSDUH Methodological Field Tests

Activity	Number of Respondents	Responses per Respondent	Total Number of Responses	Average Burden per Response	Total Burden (Hrs.)	Hourly Wage Rate	Total Hour Cost
a. Focus Groups	270	1	270	2.0 hrs.	540	$15.58	$8,413.20
b. Respondent screening for a.	337	1	337	0.083 hr.	28	$15.58	$436.24
c. Cognitive laboratory testing	200	1	200	1.0 hr.	200	$15.58	$3,116.00
d. Respondent screening for c.	250	1	250	0.083 hr.	21	$15.58	$327.18
e. Field Tests	6,600	1	6,600	1.0 hr.	6,600	$15.58	$102,828.00
f. Household screening for e.	8,910	1	8,910	0.083 hr.	740	$15.58	$11,529.20
g. Screening Verification for e.	445	1	445	0.067 hr.	30	$15.58	$467.40
h. Interview Verification for e.	990	1	990	0.067 hr.	66	$15.58	$1,028.28
TOTAL	9,497	–	9,497	–	8,225	–	$128,145.50
Annual Average (Total divided by 3 years)	3,165	–	3,165	–	2,741	–	$42,715. 17

Estimated Annualized Burden for NSDUH Methodological Field Tests

Activity	Number of Respondents	Responses per Respondent	Total Number of Responses	Average Burden per Response	Total Burden (Hrs.)
a. Focus Groups	90	1	90	2.0 hrs.	180
b. Respondent screening for a.	112	1	112	0.083 hr.	9
c. Cognitive laboratory testing	67	1	67	1.0 hr.	67
d. Respondent screening for c.	83	1	83	0.083 hr.	7
e. Field Tests	2,200	1	2,200	1.0 hr.	2,200
f. Household screening for e.	2,970	1	2,970	0.083 hr.	246
g. Screening Verification for e.	148	1	148	0.067 hr.	10
h. Interview Verification for e.	330	1	330	0.067 hr.	22
Total	3,165	–	3,165	–	2,741

13. Estimates of Annualized Cost Burden to Respondents

Respondents participate on a voluntary basis as private individuals and, therefore, are subject to no direct costs other than their time to participate; there are no start-up or maintenance costs.

14. Estimates of Annualized Cost to the Government

As stated earlier, the exact number of tests and subjects are unknown at this time. Therefore, the cost estimate is based on a number of assumptions and is likely to change. The total estimated cost to the federal government will be approximately $672,360. This includes CBHSQ’s cost to manage the study. Each package developed for this generic clearance will have the estimated costs summarized in the supporting statement. The annualized cost burden is $224,120.

15. Changes in Burden

There is no burden change.

16. Time Schedule, Publication and Analysis Plans

The data collection plan, schedule and analysis for each field test will be determined by the objectives of the methodological test, and will be included in the materials for expedited review.

The process for developing the analytical plan for the methodological tests will be similar to that used in any formal evaluation. CBHSQ staff will review the material to be pretested, discuss the objectives with the individuals responsible for developing the materials, determine the analytic questions to be addressed in the test, and then after resolution of any problems, approve the test procedures, instruments, and data analysis plan.

Techniques will primarily include qualitative analyses (for example, content analysis for results of cognitive studies), although some results may be summarized quantitatively using descriptive statistics. No complex analytic techniques will be used.

17. Display of Expiration Date

Approval is requested to not display the expiration date on Methodological Field Test materials. The exclusion of the expiration date provides the opportunity to use any applicable main study materials during the field tests and reduces the time and expense of printing all new materials.

18. Exceptions to Certification Statement

The certifications are included in this package and fully comply with 5 CFR 1320.9.