National Mental Health Study Field Test
SUPPORTING STATEMENT
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
B1.1 Overall Design
The target population for the NMHS Field Test is the civilian, non-institutionalized population aged 13 years old or older living in the contiguous U.S. (excluding Alaska and Hawaii). The non-institutional population includes both housing units and group quarters units (collectively called dwelling units). Housing units include but are not limited to: single family houses, condominiums, apartments, mobile homes or trailers, seasonal residences, vacant houses intended for occupancy, and residential units under construction. Group quarters units include but are not limited to: rooming houses, college dormitories, fraternity and sorority houses, homeless shelters, halfway houses, migratory workers’ camps, and retirement residences that do not include nursing care, medical care, or psychiatric care by staff members. Nursing homes, correctional institutions, mental institutions, and hospitals are excluded from the target population.
Active duty military personnel are also excluded; however, family members or persons not on active duty, but living at the same address, may be eligible.
There will be no sample selected in Alaska and Hawaii, due to the restricted target population. In addition, since the NMHS Field Test instruments will be available only in English, non-English speakers will be coded as non-respondents. The proposed sample size for the field test will not be designed to support regional estimates.
Retired NSDUH segments from the 2016 NSDUH will be used, including group quarters, as the sample frame. The overall sample design will be clustered, multi-stage, and stratified.
First Stage of Selection
In the first stage a Probability Proportional to Size (PPS) sample of State Sampling Regions (SSRs) will be selected. The SSRs become the primary sampling units (PSUs). The NSDUH sample currently includes 750 State Sampling Regions (SSRs), essentially strata, with 5,808 listed segments. They are equal in size within each state but not necessarily so across states. Relatively equal-sized strata will be created based on states, or groups of states, using the state target population size as a measure of size. The number of SSRs to be allocated each stratum will be in proportion to the total population that resides in that stratum. This will differ from the current state allocation of SSRs which is based on the NSDUH sample design and results in a sample that supports state-level estimates. The overall sample of SSRs will represent a national sample with equal probabilities of selection for all households in the sample. For the field test, 60 of 726 SSRs in the contiguous 48 states and the District of Columbia (the 24 NSDUH SSRs in Hawaii and Alaska are not included) will be randomly sampled using Probability Proportional to Size (PPS) sampling. These 60 SSRs will serve as the Primary Sampling Units (PSUs).
Second Stage of Selection
In Stage 2, sample segments will be sampled within each SSR. Each of the SSRs in the NSDUH sample have 8 sampled segments for a total of 6,000 (750 x 8) segments. For each annual NSDUH survey, the project implements a counting-and-listing operation in all sampled segments and retires half of these segments at the end of the survey year. These retired segments then become available for use by other surveys. Two of the 4 retired NSDUH segments will be selected from within each of the PSUs sampled in Stage 1 using PPS sampling where the measure of size is the composite measure defined for the NSDUH sample. The Stage 2 sample will result in a final sample total of 120 (60 x 2) listed segments.
Third Stage of Selection
In Stage 3, the Contractor will select a sample of households within each segment sampled in Stage 2. Segments that have undergone changes since the listing operation will be subject to a refreshing process to ensure that all new dwellings units are added to the segment and deleted dwellings units are eliminated before the sampling takes place. The Contractor will include instructions to the field staff for handling dwelling units found to be new or deleted during the field visit. Within each listed segment an average of 29.7 households will be sampled, with equal probability of selection, for screening and interviewing – excluding all dwelling units selected for NSDUH. This will result in a total sample of 3,563 (120 x 29.7) households.
Applying the expected yield rates listed below, the original sample of 3,563 households is expected to yield approximately 1,200 completed interviews with eligible adult and adolescent respondents (900 adults and 300 adolescents).
Eligibility Rate 84%
English-speaking Rate 95%
Screening Response Rate 82%
Interview Response Rate 71.5%
These assumed rates are based on rates reported for recent NSDUH surveys. Given that the NMHS will use NSDUH sample segments and similar screening protocols it is reasonable to assume that the NSDUH eligibility and response rates will prevail in the NMHS. For example, in the 2014 NSDUH methodology report,1 the reported eligible dwelling unit rate is 83.5% (p.117) and the screening and interview response rates are, respectively, 82.6% and 74.1% (Table 4.1). The proportion of interviews carried out in English is 96.3% (p.353, Table 7.30).
Fourth Stage of Selection
In the fourth and final sampling stage, 0, 1, or 2 individuals will be sampled per household (see Section B1.4 below).
B1.2 Sample Size
The sample size is motivated by these overall objectives of the field test. The sample size of 1,200 is an optimal compromise between cost and benefit. The proposed sample size supports testing of field procedures, sampling algorithms, and data processing steps. The sample size is sufficiently large to support estimates with acceptable levels of precision. For example, assuming a design effect of 2, the 95 percent two-sided confidence interval for a proportion in the 50 percent range would be (46 percent, 54 percent). For a rare characteristic, say, in the 5 percent range, the corresponding confidence interval would be (3.3 percent, 6.7 percent). The sample size allows for the identification and measurement of correlates of mental health and is sufficiently large to identify subgroups of interest and support analysis for even relatively small subgroups.
B1.3 Key measures
The NMHS Field Test sample design has as its prime objective to generate reliable and accurate point estimates with corresponding confidence intervals for the following set of 11 key measures:
Depression
Social Anxiety Disorder
PTSD
Generalized Anxiety Disorder
Panic Disorder
Mania/Bipolar Disorder
Eating Disorders
Psychotic Experiences/Symptoms
Borderline Personality Disorder (adults only)
ADHD (adolescents only)
Conduct Disorder (adolescents only)
B1.4 Selection of more than 1 person per household
Within each eligible household, the Contractor will implement a random selection of 0, 1, or 2 eligible members. Probabilities of selection will be assigned in advance to all sampled households with the objectives of achieving the following distribution of the sample across three age groups: 13-17, 25 percent; 18-30, 25 percent; and 31+, 50 percent. Given this target, the two youngest age groups, especially the 13-17 group, will be oversampled. In some households, especially those with adolescents, two individuals will be sampled whereas in others, especially those with no adolescents, it is possible to not sample any individuals. Probabilities of selection will be calculated for all possible pairs in households with two or more eligible members; using probabilities to determine the pair to be interviewed. Prior to sample implementation, the expected distribution across three age groups (13-17, 18-30, 31+) will be calculated and discussed with SAMHSA.
B1.5 Sample selection
SAS and SUDAAN will be used as the sample selection software. The quality control procedures will include a comparison of the sample and population distributions to ensure that the two are very much in line, at least within sampling error.
For every respondent, the following geographic identifiers will be collected and retained for future analytic use:
Federal information processing standards (FIPS) State, county, place codes and the full names of each;
Census tract, census block group, and census block numbers;
Minor Civil Division (MCD) FIPS code and the full names (if applicable);
Urban/rural designation of census block,
Mailing address zip code and city name,
Core Based Statistical Area (CBSA), Consolidated Statistical Area (CSA), Metropolitan Division codes and the full names of each;
New England City and Town Area (NECTA), Combined NECTA, NECTA Division codes and the full names of each (if applicable);
Principal city name associated with the CBSA or NECTA, and
Longitude and latitude (coordinates representing the centroid of the census block may be considered as an alternative to the coordinates of the resident’s address).
GPS coordinates for block assignments (obtained from respondents who agree to have coordinates taken).
B1.6 Clinical Reappraisal Study (CRS) Sample Design and Selection
As soon as the NMHS in-person interview is completed, disorder screening data will be utilized for each respondent to determine a selection probability for the CRS interview. The selection probabilities for the CRS interview have been calculated as a function of the prevalence rate for each disorder and the target sample size. They will be programmed as part of the main survey questionnaire and will be implemented dynamically at the end of the main interview. At the end of the main interview, the respondent’s disorder profile will generate a corresponding sampling probability which in turn will determine whether that respondent will be invited to participate in the CRS. The CRS is designed to obtain clinical interviews with respondents who screen threshold positive, sub-threshold, and negative for each disorder. Psychotic experiences and borderline personality disorder (BPD) will be treated as continuums with probabilities assigned relative to intervals across the continuum of screening scores. For disorders scored on a continuum, the range will be divided into a manageable number of categories (say, 3-5) and to each a sampling probability will be assigned based on the incidence rate for that category and the desired sample size. For BPD, the continuum of scores will be broken into three pre-determined categories for screening: 0=no symptoms; 1-6=subthreshold; 7 or more = threshold. Sampling fractions will vary initially across disorders due to differences in prevalence among the disorders, and will most likely require modification over the CRS study period to achieve the sample quotas. Dynamic application of sampling probabilities will involve applying an algorithm to respondent data once the main interview is complete. The parameters in the algorithm can be modified over the study period but the basic algorithmic process will not change.
A subsample of approximately 200 adult and adolescent/parent respondents will be selected to participate in a telephone-based CRS clinical follow-up interview within 4 weeks of completing the main NMHS interview. Each respondent will be classified as having threshold (i.e., meeting DSM-5 disorder criteria), sub-threshold (i.e., CIDI 4.0 screen positive cases not meeting full disorder criteria), or negative (i.e., neither threshold nor sub-threshold) for each disorder, except Psychotic Experiences. Cases will similarly be classified based on responses to the McLean Screening Instrument for Borderline Personality Disorder (MSI-BPD).2 For Psychotic Experiences, each respondent will only be classified as either negative or threshold. These classifications will be used to sample cases for the CRS.
Since the purpose of the NMHS Field Test is testing operational procedures, a reduced and simplified sample design is proposed for the field test. The field test will have 1,200 completed cases – 25 percent (300 interviews) with adolescents and 75 percent (900 interviews) with adults. To test the CRS operational procedures, it is important that at least one module for each disorder-level combination be administered during the CRS portion of the NMHS Field Test. In addition, the CRS Field Test will be designed to support an assessment of Major Depressive Disorder validity in adults by comparing CIDI scores with the SCID. Consequently, the minimum requirement for the CRS Field Test is to complete at least one module for each disorder-level combination and 30 adult modules for each of the three levels of the Major Depressive Disorder. This will most likely require sampling all adult Major Depressive Disorder threshold and subthreshold cases found in the field test. Whereas this is a realistic target for the adult section of the sample, the field test sample size of 300 adolescents is too small to support a sufficiently large sample to test concordance of the adolescent CIDI Major Depressive Disorder module with the K-SADS depression module. As is true for the adult sample for the CRS interviews, it is important to have an adolescent CRS sample of sufficient size that at least one module for each disorder-level combination be administered.
In determining the CRS sample size, statisticians will incorporate the cooperation rate, estimated conservatively to be 65.5 percent based on the five years of the NSDUH MHSS, and inflate the target accordingly to ensure the correct number of completed interviews. This response rate is based on the NSDUH/MHSS experience which reported overall annual response rates of 64.4% to 67.2% over the period 2008-2012.3
To meet the stated goals for the CRS interviews, 100 adult respondents and 50 adolescent/parent respondents will be sampled. Assuming a 65.5 percent response rate, 153 adults and 77 adolescents will be sampled and recruited. The adult sample of 100 respondents will generate approximately 300 completed CRS modules, assuming an average of three modules per adult. The adolescent sample of 50 respondents will generate approximately 150 completed CRS modules. CRS adolescent interviews that include the depression, ADHD, or conduct disorder modules will require a parent CRS interview to assess those disorders as well. All other disorder modules require only an adolescent clinical interview, with the exception of mania/bipolar disorder. This is because the K-SADS-PL combines depression and mania into one module, so parents will by default complete the mania/bipolar module.
The NMHS Field Test will yield approximately 900 completed adult in-person interviews. The expected prevalence of threshold major depression is 16.9 percent; yielding approximately 152 adults with threshold depression. The threshold depression cell and the subthreshold and the negative depression cells, both of which have higher prevalence rates, will be filled. The least prevalent cell to be filled (among those that have prevalence data) is any eating disorder (2.7 percent). Thus, 24 adults with a threshold eating disorder are expected to be found, yielding at least one CRS interview.
The NMHS Field Test will yield approximately 300 completed adolescent in-person interviews. The least prevalent cell to fill (among those that have prevalence data) is bipolar disorder (1.3 percent). Thus, 3 or 4 adolescents with a threshold bipolar disorder will be sampled from 300 cases; yielding at least one case for the CRS interview. These cases will be monitored carefully with a view to maximizing the sample size of bipolar disorder cases for appraisal using the CRS.
For the Psychotic Experiences/Symptoms disorder module, the adolescent sample will be limited to those aged 15-17 years. For the remaining adolescent disorder modules, all adolescents (aged 13-17) will be sampled in the NMHS field test sample.
Screening Procedures
Many of the procedures implemented in the NMHS Field Test will mirror those currently used during at-the-door screenings with NSDUH respondents. Prior to the FI’s arrival at the SDU, a Lead Letter (see Attachment B) will be mailed to the resident(s) briefly explaining the survey and requesting their cooperation. It will be printed on DHHS letterhead with the signature of the DHHS National Study Director and National Field Director, and the Contractor’s National Field Director.
Upon arrival at the SDU, the FI will refer the resident to this letter and answer any questions. If the resident has no knowledge of the Lead Letter, the FI will provide another copy, explain that one was previously sent, and then answer any questions. The lead letter and other screening and initial contact materials will include references to the $5 or $10 cash screening incentive based on the experimental condition assigned to the household. Instructions for administering the $5 or $10 cash incentive and obtaining a signature on the incentive receipt can be found in Attachment I, Housing Unit and Group Quarters Unit Screening Questions.
If no one is home during the initial visit to the SDU, the FI may leave a Sorry I Missed You Card (Attachment Y) informing the resident(s) the FI plans to make another callback at a later date/time. Callbacks will be made as soon as feasible following the initial visit. FIs will attempt to make at least four callbacks (in addition to the initial call) to each SDU in order to complete the screening process and complete an interview, if yielded. If the FI is unable to contact anyone at the SDU after repeated attempts, the FS may send one of the Unable-to-Contact (UTC) letters (Attachment C). These UTC letters reiterate information contained in the Lead Letter and present a plea for the resident to participate in the study. If after sending the UTC letter, an FI is still unable to contact anyone at an SDU, a Call-Me letter (Attachment C) may be sent to the SDU requesting that the resident(s) call the FS as soon as possible to set up an appointment for the FI to visit the resident(s).
When in‑person contact is made with an adult resident of the SDU and introductory procedures are completed, the FI will present a Study Description (Attachment G) and answer any questions that person might have concerning the study. A Question & Answer Brochure (Attachment J) that provides answers to commonly-asked questions may also be given. In addition, FIs will be supplied with copies of NIMH Articles and Information Sheets (Attachment R) for use in eliciting participation, which can be left with the respondent.
If a potential respondent refuses to be screened, the FI will be trained to accept the refusal in a positive manner, thereby minimizing the possibility of creating an adversarial relationship that might preclude future opportunities for contact. The FS may then request one of several Refusal Letters (Attachment D) be sent to the residence. The letter sent will be tailored to the specific concerns expressed by the potential respondent and will ask him or her to reconsider participation. Refusal letters will be customized and also include the FS’s phone number in case the potential respondent has questions or would like to set up an appointment with the FI. Unless the respondent calls the FS or the Contractor’s office to refuse participation, an in-person conversion will be attempted.
With respondent cooperation, the FI will begin screening the SDU by asking either the Housing Unit Screening questions, or the Group Quarters Unit Screening questions, as appropriate. The screening questions will be administered using a 7-inch touch screen Android tablet computer. A paper representation of the housing unit and group quarters unit screening process is shown in the Showcard Booklet (Attachment T).
Once all household members aged 13 or older have been rostered, the hand-held computer will perform the within-dwelling-unit sampling process, selecting zero, one, or two members to complete the interview. For cases with no one selected, the FI will ask for a name and phone number for use in verifying the quality of the FI’s work, thank the respondent, and conclude the household contact.
For each person selected to complete the full interview, the FI will follow these steps:
If the selected individual is aged 18 or older, or aged 17 and living independently from his or her parent or legal guardian, and is currently available, the FI will immediately seek to obtain informed consent. Once consent is obtained, the FI will begin to administer the questionnaire in a private setting within the dwelling unit. As necessary and appropriate, the FI may make use of the Appointment Card (in Attachment F) for scheduled return visits with the respondent.
If the selected individual is 13 to 17 years of age, except in rare instances where a 17-year-old is living independently from his or her parent or legal guardian, in which case the 17-year-old provides his or her own consent, the FI will read the parental introductory script (Attachment U) to the parent or legal guardian before speaking with the adolescent about NSDUH. Subsequently, parental consent will be sought from the present adolescent’s parent or legal guardian using the Parent section of the adolescent version of the Introduction and Informed Consent Scripts (Attachment H). Once parental consent is granted, the adolescent will then be asked to participate using the Adolescent section of the same document. If adolescent assent is received, the FI will begin to administer the questionnaire in a private setting within the dwelling unit with the present parent or legal guardian in the home throughout the interview.
The NMHS screening and interview will be conducted only in English. FIs will be equipped with an Other Language Card (Attachment Z) to use if a screening respondent is unable to communicate in English. This Other Language Card introduces the FI in 11 languages and states he or she would like to speak with someone in the household who speaks English. If no one who speaks English is available, the screening will not be completed. If, as a result of the Other Language Card, the respondent identifies someone who does speak English, a translator can be used for the screening only. This translator does not necessarily need to be a member of the household. Adolescents are eligible to serve as translators. The FI will read the screening questions to the translator in English, and the translator will translate them to the screening respondent. Procedures for using a translator do not differ if the translator is an adolescent. If anyone is selected for the interview, that person must speak English; non-English speakers will be coded as nonrespondents. This decision was made given the very small proportion of the population that are not able to speak English. For the 2015 NSDUH, only a very small portion of the population (< 5%) elected to complete the NSDUH interview in Spanish.
Adult/Adolescent In-Person Interview Procedures
As mentioned in section A.3, the FI will administer the NMHS Field Test in-person interview in a prescribed and uniform manner with sensitive portions of the interview completed via ACASI, similar in many ways to the current administration of the NSDUH interview.
There is the possibility the questions contained in the adult and adolescent mental health NMHS Field Test modules may cause some respondents emotional distress. Since these questions are answered within the ACASI section of the interview, there is no way for the FI to know which respondents will see these questions, or what the respondent’s answers to those questions are. Therefore, the ACASI portion of the questionnaires will contain scripts with national hotline numbers if respondents screen positive for certain mental health disorders or provide certain answers to the suicidality questionnaire module. In addition, all FIs will be trained on the Non-Clinician Distressed Respondent Protocol (Attachment AA) which provides a full summary of procedures and scripts for handling respondent distress. National hotline number information will also be included at the bottom of the Incentive Receipt (Attachment E).
Race/ethnicity questions will be FI-administered and meet all of the guidelines for the OMB minimum categories. The addition of the finer delineation of Guamanian or Chamorro and Samoan, which collapse into the OMB standard Native Hawaiian/Other Pacific Islander category, were a requirement of the new HHS Data Collection Standards. They are included in the NMHS questionnaires.4
At the end of the in-person interview, the script provided for the FI to read briefly explains why collecting Global Positioning System (GPS) coordinates at respondents’ residences is helpful and asks for permission to collect GPS coordinates outside the respondent’s home. After the interview is completed and before the verification procedures begin, each respondent will be given a $40.00 cash incentive and an Incentive Receipt (Attachment E) signed by the FI.
For verification purposes, at the end of the interview adult respondents and parents are will be asked to provide current address and phone number in the tablet on an electronic Quality Control Form (Attachment AB) for possible follow‑up to ensure the FI did his or her job appropriately. Respondents will be informed that providing the information is voluntary.
A random sample of those who provide contact information for verification purposes will be contacted via telephone to answer a few questions verifying that the in-person interview took place, that proper procedures were followed, and that the amount of time required to administer the interview was within expected parameters. The CATI Verification Scripts (Attachments AC-1 and AC-2) contain the scripts for these interview verification contacts via telephone, as well as the scripts that will be used when verifying a percentage of certain completed screening cases in which no one was selected for an interview or the SDU was otherwise ineligible (vacant, not primary residence, not a dwelling unit, dwelling unit contains only military personnel, respondents living at the sampled residence for less than half of the quarter). For verification purposes, a Quality Control letter (Attachment W) will be mailed to a respondent’s address when a phone number is not available.
FIs may give a Certificate of Participation (Attachment AD) to interested respondents after the interview is completed. Respondents may attempt to use these certificates to earn school or community service credit hours. As stated on the certificate, no guarantee of credit is made by SAMHSA, NIMH or the Contractor. The respondent’s name is not written on the certificate. The FI will sign his or her name and date the certificate, but for confidentiality reasons the section for recording the respondent’s name will be left blank. The respondent can fill in his/her name at a later time so the FI will not be made aware of the respondent’s identity. It is the respondent’s choice whether he or she would like to be identified as a NMHS respondent by using the certificate in an attempt to obtain school or community service credit.
GPS coordinates will be recorded outdoors after all interviews are completed using the equipped GPS receiver on the tablet for only those locations where permission is obtained to collect them. In households where multiple adult or adolescent interviews are completed, both subjects will need to agree to the GPS request for the reading to be collected.
As noted above, all interview data will be transmitted on a regular basis via secure encrypted data transmission to the Contractor’s offices, where the data will be subsequently processed and prepared for reporting and data file delivery.
Parent Interview Procedures
After an adolescent begins the self-administered portion of his or her in-person interview, the FI will identify the present, knowledgeable parent/legal guardian, provide the Parent Study Description (Attachment K) and invite that adult to participate in the parent interview. The FI will record the parent questionnaire ID as well as the child’s initials on a Parent Interview Information Card (Attachment V). The adolescent’s initials will be recorded on the card to remind the parent of the adolescent about which the parent should be providing information in the parent interview. This will be especially important in those situations where more than one adolescent is interviewed in the same household. In addition to the parent interview ID and adolescent’s initials, this information card will provide the parent/legal guardian respondent with the NMHS parent interview web address, a technical support telephone number and an option of calling to complete the survey with a NMHS-trained TI. The secure web link for the parent interview will adhere to government https protocols. This information will be accessible via the website as well.
The FI will encourage the parent/legal guardian to complete the questionnaire while the FI is still in the household. The FI will also let the parent know he or she may complete it later, if needed, either on the website or by calling. Once the FI has explained participation and asked the parent to participate, if the parent indicates he/she will complete the interview, the FI will provide the cash incentive to the parent. The FI will then ask the parent to sign an incentive receipt form.
If the present parent/legal guardian refuses to participate, the FI will offer information and provide a Parent Study Description (Attachment K) that explains the importance of the study and ease of participation. If the in-person visit ends in a refusal to complete the parent interview, no further recruitment attempts will be made.
Section B3 summarizes the steps taken to remind and encourage initially cooperative parents to complete their parent interviews.
Adult and Adolescent CRS Interview Procedures
At the end of each in-person interview, an algorithm will determine whether the respondent should be invited to participate in the CRS interview. If a respondent has been selected to complete the CRS interview, a series of recruitment screens will automatically display on the FI’s computer.
If an adult in-person respondent is selected for the CRS, the FI will hand the CRS Study Description Sheet (Attachment M-1) to the adult and attempt to recruit the adult to complete the CRS interview. If the adult agrees to participate, the FI will provide the cash pre-incentive and gather first name, telephone number and best times for the CI to contact the adult to complete the interview. The FI will complete a reminder card for the respondent that includes the days and times the respondent indicates are best for the follow-up interview (Attachment M-1).
If an adolescent is selected for the CRS, the FI will briefly explain the selection to the adolescent and then ask to speak to the parent/legal guardian who agreed to complete the parent instrument. The FI will hand the CRS Study Description Sheet (Attachment M-1) to the parent/legal guardian and explain the importance of the CRS interviews. In addition, the FI will ask if the selected adolescent can complete the CRS interview on the telephone from home without an adult present. If the parent/legal guardian gives permission, the FI will hand the Adolescent CRS Study Description Sheet (Attachment M-1) to the adolescent and attempt to recruit the adolescent. If the adolescent agrees to participate, the FI will provide the cash pre-incentive and gather adolescent first name, first name of the parent/legal guardian present during the interview, telephone number, and best times to call from the parent/legal guardian.
CIs will be provided with supplemental consent information for adolescent interviews. They will receive the first name of the parent/legal guardian who provided consent in the home and an indicator if the parent/legal guardian provided approval for the adolescent to complete the CRS interview without a parent/legal guardian present in the home. CIs will obtain consent from the parent/legal guardian via telephone before proceeding with the adolescent telephone assent procedures.
Parent CRS Interview Procedures
If an adolescent’s selection algorithm indicates the need for a parent CRS interview (i.e., the adolescent screens in for presence of an externalizing disorder for which parent report is necessary), screens will display automatically on the FI’s computer for the FI to read aloud, asking to speak to the present parent/legal guardian who was selected to complete the web/telephone parent interview for the adolescent. The FI will explain to the parent/legal guardian that he/she was also selected for a follow-up CRS interview. The FI will hand the CRS Study Description Sheet (Attachment M-2) to the parent/legal and attempt to recruit the parent/legal guardian to complete the CRS interview. If the parent/legal guardian agrees to participate, the FI will provide the cash pre-incentive and gather first name, telephone number and best times for the CI to contact the parent/legal guardian to complete the interview.
Additional CRS Interview Procedures
Clinical interviewers trained in administering the SCID and K-SADS-PL will be recruited to conduct the NMHS Field Test. These interviewers will be certified before administering the SCID or K-SADS-PL. Adult, adolescent and parent volunteer respondents will be recruited from mental health treatment centers and paid $40 for participating in a telephone interview, which, with the respondents’ consent, will be audio recorded. An expert in the SCID and K-SADS-PL will listen to the audio recording of the interview and review the paper instruments to determine whether the clinical interviewer administered the instruments properly. These clinical interviewers will be supervised throughout the data collection period to maintain the integrity and reliability of clinical assessment and to resolve any clinical issues or questions that emerge.
CRS contact information will be released to the field after confirmation that the corresponding main interview has been completed. Adult and adolescent respondents will only be invited to the CRS after they complete their in-person interviews, so their information will be released to the CRS data collection team shortly after the FI leaves the household and transmits completion codes to the central data repository. A parent/legal guardian can complete a primary (web/telephone) interview during an FI’s visit, but may complete it days or weeks after the FI leaves the household. A parent/legal guardian’s information will be released to CIs once the adolescent’s information is released. CIs will prompt the parent/legal guardian to complete the main parent interview if they have not done so and remind them that they can complete the main parent interview via web or telephone.
Once a respondent’s contact information is made available, the CI will contact the respondent to complete the CRS interview at the respondent’s earliest convenience. Cases will be assigned to CIs via the CMS based on CI availability and the respondent’s preferred time for completing the follow-up interview. When a case is assigned to a CI, an e-mail will be sent to the CI alerting him/her of the case. CIs will complete the follow-up CRS interview via telephone as soon as feasible; ideally within two weeks, but no later than four weeks following the corresponding in-person interview to minimize the chance of discordance due to real changes in symptomatology over time. More calendar time will be allowed for difficult-to-contact respondents.
Before starting the interview, the CIs will read the introduction and informed consent/assent scripts (Attachment N) to remind respondents of the importance of privacy and to ensure that respondents receive all information necessary for informed consent. In addition, the script will remind respondents that information gathered during the process of data collection will be kept completely confidential. To safeguard the privacy for adolescent CRS interviews, the script will also require the CI to confirm with the adolescent they are in a comfortable and private location within the home before starting the interview. If the CI suspects that a parent/legal guardian or another individual is listening to the interview, he/she will offer an appointment to the adolescent. CIs will complete a problem report for any cases where they feel confidentiality of the interview is compromised.
During the CRS interview, the CIs will read questions from the clinical instruments (Attachment A-3 and A-4) and ask follow-up questions as needed based on respondent answers per instructions received during training. CIs will be taught to make copious, handwritten notes to accurately depict the respondent’s mental health. In addition, the interview will be recorded using a centralized system owned and operated by the Contractor, subject to respondent permission.
A detailed clinician distressed respondent protocol (Attachment AE) provided for the CIs will ensure the safety of all respondents who may be at danger to themselves or others because of their current mental state. If a CI feels that an adult respondent is exhibiting moderate distress, the CI will be trained to provide the respondent with the telephone number for the National Suicide Prevention Lifeline. If an adult exhibits severe distress (expresses current thoughts of harming themselves), the CI will attempt to connect the adult directly with the National Suicide Prevention Lifeline to speak with a trained helpline counselor. The CI will stay on the line while the adult respondent talks with the helpline counselor. If an adolescent exhibits severe distress (expresses current and serious plan to harm themselves or others), the CI will ask to speak to the parent/legal guardian and will notify them via telephone of the situation before offering the appropriate hotline information. Also, if an adult or adolescent exhibits signs of distress, the CI will immediately contact the CRS task leaders and complete an incident report describing the situation in the CMS. After the CI completes the incident report, an automatic e-mail will also be sent to members of the NMHS management team who will review the incident report and evaluate if the appropriate actions were carried out by the CI (i.e., offer of a break and hotline number or contacting the parent if warranted). Event reports will be filed with the Contractor’s IRB within 48 hours for these cases and further action or follow-up might result after IRB review. Details of the clinician distressed respondent protocol (Attachment AE) have been discussed with and approved by SAMHSA and the Contractor’s IRB.
In cases where CIs note respondent cognitive impairment, if it is deemed temporary (i.e., alcohol or substance intoxication), the CI will politely end the interview and attempt to make an appointment to re-contact the respondent at another time. If the respondent refuses to agree to reschedule the follow-up interview, the CI will end the phone call and not attempt re-contact. If the impairment is deemed by the CI to be long term and consistent (i.e., possible brain damage or dementia), the CI will politely end the interview and not ask to re-contact the respondent.
Upon completing an interview, the CI will ship the paper instrument to the Contractor where the instruments will be receipted and stored in a locked facility.
Once each instrument is received at the Contractor’s facility, a clinical supervisor (CS) will complete a clinical edit to ensure proper interviewing procedures were followed. In addition, a 10 percent random sample of each CI’s completed interview will be reviewed by a CS in detail along with the respective audio file. The CSs will then provide performance feedback to CIs, providing coaching as necessary to ensure the CIs are administering the interview appropriately and consistently. An additional technical edit will be completed by specially trained staff to ensure responses are marked clearly. The data will then be keyed and processed for data analysis. As an extra quality control step, all clinical instruments will be subject to double-keying.
Questionnaires
As explained in section A.3, the versions of the questionnaires to be fielded in the NMHS Field Test include computerized adult, adolescent, and parent instruments and paper and pencil (PAPI) adult, adolescent, and parent clinical reappraisal (CRS) instruments.
Adult and Adolescent In-Person Interview Questionnaires
The proposed in-person interview CAI questionnaires are shown in Attachment A-1. The document shown is a paper representation of the content that is to be programmed. The interview process is designed to retain respondent interest, ensure confidentiality, and maximize the validity of response. The questionnaires will be administered in such a way that FIs do not know respondents’ answers to sensitive questions. Most questions for the adult and adolescent interviews will be self-administered using ACASI. The respondent listens to the questions privately through headphones so even those who have difficulty seeing or reading are able to complete the self-administered portion. Topics that are administered by the FI (i.e., the CAPI section) are limited to questions on demographics, health insurance, and income. Respondents will be offered the option of designating an adult member of the household who is available during the interview to serve as a proxy to provide answers to questions in the Health Insurance and Income sections.
The ACASI portion of the questionnaires are divided into sections based on specific dimensional scales and screens, screen positive modules and all screen positive and screen negative modules. The instruments contain specific modules of both the adult and adolescent versions of the DSM-5 World Mental Health Composite International Diagnostic Interview (WMH-CIDI) and several self-report measures from the NIH PhenX Project (i.e., suicide, PTSD, general mental health common data elements).
Parent Interview Questionnaire
The available parent/guardian who is knowledgeable about each adolescent respondent (aged 13-17 years old) will be asked to complete a parent interview (Attachment A-2) about that adolescent respondent. The parent interview will be completed independently over the web or via a telephone interviewer if the parent/legal guardian elects to call-in and complete the survey. It is based on the long version of the NCS-A Parent Questionnaire (NCS-A PQ).5 In order to limit the estimated administration time to 30 minutes, some questions were deleted from the NCS-A PQ. In order to verify that the parent/legal guardian reports on the correct adolescent (as instructed by the FI), a small number of demographic questions were added. Some of the NCS-A PQ questions were adapted for the web mode, but the documented scalar properties of NCS-A PQ were retained. The web instrument will be developed and implemented using the Contractor’s web survey tool. When accessed independently over the web, this will be a self-administered, text-only questionnaire with no audio content. The parent instrument will be nearly identical for the web and telephone administration except responses need to be read aloud over the phone.
Adult, Adolescent and Parent CRS Interview Questionnaires
The NMHS Field Test consists of three CRS interview instruments: one for adult respondents (Attachment A-3), one for adolescent respondents (Attachment A-3), and one for the adolescent respondents’ parent/legal guardians (Attachment A-4). All will be administered by a CI over the telephone using a PAPI instrument. For adults and adolescents, the CRS interviews are expected to average 60 minutes per respondent. The parent CRS interview is expected to average 30 minutes. These CRS interviews will include modules from the Structured Clinical Interview for DSM-5 for Axis-I disorders (SCID)6 and the International Personality Disorder Examination (IPDE)7 for adults and the Kiddie Schedule for Affective Disorders and Schizophrenia-Present and Lifetime Version 2013 (K-SADS-PL 2013)8 for adolescents and parent/legal guardians. Some adolescents age 15-17 will also complete the SCID Psychotic Experiences module since reporting of prodromal psychotic symptoms has not been shown to be reliable in young adolescents.
In 2014, the NSDUH weighted response rates were 82 percent for screening and 72 percent for interviews, with an overall response rate (screening * interview) of 60 percent. With higher incentive amounts for the NMHS Field Test, the Contractor expects the weighted response rates for adult and adolescent respondents in the NMHS to be about the same as the 2014 NSDUH rates. The estimated screening and interview response rates will be 82 percent and 71.5, respectively. The estimated parent web/telephone interview response rates will be 70 percent. The estimated agreement rate for the CRS adult and adolescent/parent interviews will be 84 percent. The estimated overall response rate for the CRS will be 78 percent.
Adult and Adolescent In-Person Interviews
As a way to maximize response rates, the NMHS Field Test will use existing, OMB-approved NSDUH materials as templates, though all will be updated to reflect the NMHS name, content, goals and purpose. CBHSQ revised the NSDUH lead letter and brochure through a review of contact materials used on other government-sponsored surveys, expert review, and feedback from 17 focus groups conducted in both English and Spanish in five metropolitan areas in 2014 (OMB No. 0930-0290). The primary focus of these revisions was to improve the 2015 NSDUH materials in ways likely to generate positive reactions from members of sampled households and, therefore maximize response rates. The NMHS Field Test materials will include a revised Lead Letter (Attachment B) and Question & Answer Brochure (Attachment J).
In addition to using cash incentives and well-designed contact materials to achieve the expected response rates, the NMHS Field Test will utilize NSDUH study procedures proven to maximize respondent participation. This begins with assignment of the cases prior to the start of data collection, accompanied by weekly response rate goals that are conveyed to the FIs by the FS. When making assignments, FSs will take into account which FIs are in closest proximity to the work, FI skill sets, and basic information (demographics, size, etc.) about the segment. FSs will assign cases to the FIs to ensure maximum production levels at the start of the data collection period.
Once FIs transmit their work, data will be processed and summarized in daily reports posted to the web-based CMS accessed by FSs. On a daily basis, FSs will use reports on the CMS to review response rates, production levels, and record of call information to determine an FI’s progress toward weekly goals, to determine when FIs should attempt contact with a case, and to develop plans to handle challenging cases such as refusal cases and cases where an FI is unable to access the dwelling unit. FSs will discuss this information with FIs on a weekly basis.
Response rate and nonresponse patterns will be tracked by various demographics and region of the country in the NMHS Data Collection Final Report. The report provides detailed information about noncontacts versus refusals, including reasons for refusals.
As noted in section B.2 above, FIs may use a Sorry I Missed You Card (Attachment Y), NIMH Articles and Information Sheets (Attachment R), and a Certificate of Participation (Attachment AD) to help make respondent contact and encourage participation. To aid in refusal conversion efforts, Refusal Letters (Attachment D) tailored to specific refusal reasons can be sent to any case that has refused. Similarly, an Unable-to-Contact Letter (Attachment C) may be sent to a selected household if the FI has been unable to contact a resident after multiple attempts. For cases where FIs have been unable to gain access to a group of SDUs due to some type of access barrier, such as a locked gate or doorperson, Controlled Access Letters (Attachment C, Unable-to-Contact Letters) can be sent to the gatekeeper to obtain his or her assistance in gaining access to the units. If those attempts fail, a Call-Me Letter (in Attachment C) may be sent directly to a selected household. These letters inform the residents that an FI has been trying to contact them and asks that they contact the FS by phone. If the resident calls the FS, the FS will attempt to get the resident to agree to an appointment so the FI can return to that address and screen the household in person.
Parent Interviews
Shortly after setting an adolescent up to complete the ACASI portion of the adolescent in-person interview, FIs will identify a knowledgeable parent or legal guardian about the selected adolescent, explain why participation is worthwhile, and ask if the adult would be willing to participate. If the parent/legal guardian agrees to participate, the FI will give the parent/legal guardian a Parent Interview Information Card (Attachment V), collect key contact information for subsequent follow-up contact attempts (Attachment I, pg. 3-1), and give the parent/legal guardian a $30 cash incentive. The parent/legal guardian will have the option of completing the interview over the web using his or her own computer or calling a toll-free number and completing it over the telephone. The FI will encourage the parent/legal guardian to complete the interview before the FI leaves the home.
If the parent/legal guardian has not already completed the web/telephone interview by the time the FI transmits data back to the study’s central data repository, an automated messaging system will send an invitation via email and/or text message. The invitation (Attachment AF) will include a link to the web interview as well as the toll-free number they may call to complete the interview over the telephone. The secure web link for the parent interview will adhere to government https protocols. Parent/legal guardians will be able to click on the link and access the interview without having to type in the web address. This may encourage some parent/legal guardians to participate that might otherwise have delayed or entirely foregone participation.
If a parent/legal guardian agreed to complete a questionnaire about an adolescent but has not completed it within three days of the initial invitation, the automated messaging system will send an email and/or text reminder to the parent/legal guardian (Attachment AF) if such information was provided by the parent at the time of recruitment. The system will send another email and/or text reminder if an additional five days’ elapse without completion of the parent interview and will continue this reminder pattern until either the parent/legal guardian completes the interview or a month has elapsed since the initial invitation. If the parent refuses after the FI leaves the household, the case will be finalized. The reminders will mention the importance of completing the interview within a month of the date that the adolescent’s in-person interview was completed (Attachment AF).
Approximately 70 percent of parent/legal guardians are estimated to complete the parent interview. The parent interview response rate is based on an evaluation of the NCS-A parent interview response rate which was in the low 60’s using a leave-behind Paper and Pencil (PAPI) questionnaire with limited follow-up prompting. A higher response rate is estimated for the NMHS Field Test Parent interview based on use of improved methodology with administration offered via a web and telephone and an expanded prompting protocol using e-mail and text reminders. In addition, parents will receive a pre-paid cash incentive.
Adult, Adolescent and Parent CRS Interviews
The NMHS has modeled its methods for maximizing response rates in the CRS interviews after those used in the NSDUH MHSS. Adaptions have been made for new components such as the parent interview and the adolescent follow-up clinical reappraisal interviews. The MHSS obtained high response rates in all its administration years, 2008 through 2012. In the most recent administration (2012), 83.7 percent of respondents who were asked to participate in the MHSS interview agreed to participate and 78.6 percent of those respondents that agreed to participate actually did so for an overall response rate of 65.8 percent.
FIs will not be recontacting households to convert CRS interview refusals, but they will be trained to answer respondent questions at the time of in-person recruitment as appropriate. Since follow-up respondent participation is so critical to the success of the NMHS, a CRS Unable-to-Contact Letter (Attachment AG) will be sent to respondents who agreed to participate in a follow-up interview and have been hard to reach or unable to contact, including those who do not answer the telephone, are always unavailable when a CI calls to set up an appointment for the follow-up interview, or whose phone number is incorrect or disconnected. The letter will be sent via UPS to the respondent at the address at which he/she completed the initial interview. The letter will provide respondents with a toll-free telephone number for contacting the Contractor to set up an appointment for the follow-up interview.
To maximize adult, adolescent, and parent CRS interview response rates, the CIs will use the best day/best time information obtained by the FIs to schedule interviews, but they will also be flexible in scheduling a time for the CRS interview that is convenient to the respondent. If a respondent is unavailable when the CI calls to complete the follow-up interview, the CI will schedule a callback appointment. CIs will be trained to thoroughly explain the study, its purpose, and answer questions from all respondents.
In addition, the study is designed so that follow-up of adolescents for the CRS is not contingent upon the parent/legal guardian completing their initial interview. In cases where a parent/legal guardian never completes his or her interview, an adolescent will be contacted regardless of parent/legal guardian non-response, and an adolescent-only interview will be considered a complete case for both initial interview (CIDI-A) and follow-up (K-SADS-PL). Although the parent interview adds more information on presence of adolescent disorders, parent interviews are not necessary for making an adolescent diagnosis. Severing the adolescent interview process from the parent interview process in this way will prevent parent/legal guardian non-response from impacting the total response rate for the adolescent survey.
Average Interview Length Estimates. The average length for each interview has been estimated before submission of these supporting statements (Stage 1). The Contractor will estimate in-person and web/telephone interview lengths again before the field test, after those questionnaires are fully programmed (Stage 2). Last, the Contractor will use field test data to further refine interview length estimates, in preparation for the full implementation (Stage 3).
Stage 1 – Prior to OMB Statement Submission.
Four different timing estimates were prepared for the adult and adolescent in-person interview instruments. First, the shortest path through the instrument was estimated. This estimate identified how respondents could answer the questions to be routed through the fewest number of items in the instrument. The timing estimate for the shortest path was then created using the assumption that each item will take an average of 10-15 seconds to administer. The number of items in the shortest path was multiplied by the average time per item, resulting in a range of the estimated time to complete the shortest path through the instrument.
The second timing estimate was the typical path through the instrument. A scenario was designed for each instrument to represent how a typical respondent might answer the questions. For the adult in-person interview instrument, for example, the typical respondent was assumed to be married with two biological children, employed, covered by private health insurance, has minor chronic health conditions, and will screen into the Depression module and the Anxiety and Worry module, in addition to several other criteria. Like the shortest path estimate, the typical path estimate was prepared by multiplying the number of items in this typical path by the assumed administration time of 10-15 seconds per item.
The third estimate was prepared by reading through the questions to simulate an interview, rather than using an assumed time per question as was done in the first two estimates. For each instrument, two to three testers read through the questions following the criteria outlined for the typical path, described in greater detail above. For the adult and adolescent in-person interview instruments, each tester simulated a respondent who listened to the audio of the question but then selected an answer before the audio for all response options was read. For the parent interview, the testers simply read the question and response choices in their head and selected an answer much as a respondent completing a web survey would. The third timing estimate is presented as a range of the shortest and longest times reported by the testers.
The fourth estimate was prepared by modifying the timing data captured when preparing the third estimate. The fourth estimate omits modules appearing in Part 2 of the instrument, since some respondents will be randomized to skip this section. Part 2 of the adolescent and adult instruments is comprised of modules that contained detailed questions on specific disorders. These modules are administered to 1) respondents who, through the answers they provide in the CIDI Screener module, endorse symptoms that indicate they may have a specific disorder and 2) a random 25 percent of the respondents who do not endorse symptoms as a means to validate the screening tool. For each module included in this estimate (i.e., those modules not in Part 2), the time per module was assumed to be the average time spent on that module by the testers in the third estimate. These module averages were then summed to create a total time estimate for typical respondents skipped out of Part 2. This fourth estimate was skipped for the parent interview, which does not include Part 2.
For the parent interview, two estimates were calculated for each mode of interview (web and telephone) – a shortest path estimate and a more typical path. However, all estimates for the parent interviews are based on simulated interviews (similar to the approach described above for the third and fourth estimates).
The interview lengths for the adult and adolescent in-person interview instruments estimated through Stage 1 time testing (described above) are shown in Table 3. The interview lengths for the web and telephone parent interviews (also described above) are shown in Table 4. In addition, module-level estimates are shown in Tables 5 (adult/adolescent) and 6 (parent).
Length estimation for the CRS interviews is based on NSDUH MHSS timing data. The NMHS Field Test will entail administering either one psychosis module or up to three SCID/K-SADS-PL/IPDE disorders for each respondent.
Stage 2 – After Programming, Before Field Test.
Only the in-person and web/telephone questionnaires have been programmed, so only those interview types will be subject to Stage 2 time testing.
For Stage 2, the same scenarios used in Stage 1 time estimations will be used. As noted above, these scenarios include the shortest path and a typical path through the instrument. When estimating the in-person interview lengths, each scenario will follow through the programmed instrument using a slow method and a fast method. The slow method will model a poor reader, so the tester will allow the audio to fully play each question before entering a response. The fast method will model a proficient reader, so the testers will read the question text for themselves and enter answers whenever ready to answer, regardless of whether the audio has finished reading the onscreen text or not.
For the FI-administered portions of the interviews, the testers will simply read the question text aloud and then enter the response from the scenario script. An estimate of the proportion of the sample expected to fall into each category will then be applied (i.e., poor readers with few disorders, poor readers with more disorders, proficient readers with few disorders and proficient readers with more disorders) to weight the timing data by those proportions.
For Stage 2 testing, the collection of item-level timing data will be enabled on the laptop computers used by testers. This fine level of data will allow for quick and efficient determination of exactly how much each item contributed to the overall average length estimate for each interview type.
If any interview’s Stage 2 weighted and averaged results are significantly longer than the times shown in section A12, SAMHSA, NIMH, and the Contractor will work together to edit and/or remove questions as needed to bring estimates into alignment with those shown in A12.
Stage 3 – After Field Test.
Programs will be set up to automatically collect timing data for the adult and adolescent in-person interviews and the parent interview. These timing data will be available at the module level, as well as overall. CRS interviews are administered via PAPI, so they are not programmed. CIs will manually record the time at the beginning and the end of each interview. These data will be captured from the paper questionnaires and then used to calculate overall interview length. Field test results from the CRS interviews will be used to refine estimates of interview length before the full implementation.
Table 3. Timing Results from Stage 1 Estimations for Adult and Adolescent Instruments
|
Estimate 1: Shortest path |
Estimate 2: Typical path (average time per question) |
Estimate 3: Typical path (timed read through) |
Estimate 4: Typical path (timed read through), Part 2 excluded |
Adult Instrument |
41-62 minutes |
73-110 minutes |
65-84 minutes |
38 minutes |
Adolescent Instrument |
39-59 minutes |
63-95 minutes |
70-84 minutes |
43 minutes |
Table 4. Timing Results from Stage 1 Estimations for Web and Telephone Parent Instruments
|
Estimate 1: Shortest path (timed read through) |
Estimate 2: Typical path (timed read through) |
Web Instrument |
15 minutes |
25 minutes |
Telephone Instrument |
21 minutes |
31 minutes |
Table 5. Timing Results from Adult and Adolescent Simulated Interviews (Typical Path)
|
Estimate 3 (Typical path) |
Estimate 4 (Part 2 excluded) |
Estimate 3 (Typical path) |
Estimate 4 (Part 2 excluded) |
|||
Module Name |
Adult Timing #1 |
Adult Timing #2 |
Adult Timing #3 |
Adult Avg. Timing |
Adolescent Timing #1 |
Adolescent Timing #2 |
Adolescent Avg. Timing |
Core Demographics |
0:01:56 |
0:01:54 |
0:01:45 |
0:01:52 |
0:02:04 |
0:01:40 |
0:01:52 |
Military Service |
0:01:26 |
0:01:24 |
0:01:49 |
0:01:33 |
N/A |
N/A |
N/A |
Beginning ACASI & ACASI Tutorial |
0:04:34 |
0:04:01 |
0:04:27 |
0:04:21 |
0:05:26 |
0:05:06 |
0:05:16 |
Your Health |
0:07:55 |
0:07:28 |
0:06:21 |
0:07:15 |
0:07:29 |
0:08:17 |
0:07:53 |
Columbia Impairment Scale |
N/A |
N/A |
N/A |
N/A |
0:01:29 |
0:01:42 |
0:01:35 |
CIDI Screener |
0:04:08 |
0:03:45 |
0:03:19 |
0:03:44 |
0:05:45 |
0:04:50 |
0:05:18 |
Depression |
0:01:10 |
0:04:36 |
0:04:09 |
0:00:58 |
0:02:22 |
0:05:31 |
0:01:10 |
Mania (High Mood) |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
Anxiety and Worry (GAD) |
0:02:03 |
0:03:50 |
0:03:33 |
0:01:05 |
0:02:55 |
0:05:09 |
0:01:05 |
Social Anxiety |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
Agoraphobia |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
Panic Disorder |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
Eating Disorders |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
Intermittent Explosive Disorder (Anger Attacks) |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
N/A |
N/A |
N/A |
Suicidality |
0:00:18 |
0:01:12 |
0:00:16 |
0:00:19 |
0:00:18 |
0:00:24 |
0:00:21 |
CIDI Psychotic Experiences |
0:01:40 |
0:01:39 |
0:01:34 |
0:01:38 |
0:01:53 |
0:01:41 |
0:01:47 |
Treatment of Emotional Problems |
0:03:56 |
0:04:37 |
0:00:12 |
0:01:04 |
0:04:01 |
0:05:34 |
0:02:41 |
Pharmacoepidemiology |
0:02:44 |
0:03:30 |
0:03:52 |
0:01:20 |
0:00:00 |
0:04:00 |
0:01:20 |
Trait Fear |
0:00:00 |
0:00:00 |
0:01:51 |
0:00:00 |
0:01:58 |
0:00:00 |
0:00:00 |
Disinhibition |
0:01:40 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:00:00 |
0:01:44 |
Personality |
0:00:00 |
0:02:24 |
0:00:00 |
0:02:24 |
0:00:00 |
0:00:00 |
0:00:00 |
Affective Reactive Index |
N/A |
N/A |
N/A |
N/A |
0:00:00 |
0:00:45 |
0:00:00 |
Borderline Personality Disorder |
0:01:12 |
0:01:24 |
0:01:00 |
0:01:12 |
0:01:27 |
0:01:16 |
0:01:21 |
Attention and Concentration |
N/A |
N/A |
N/A |
N/A |
0:03:39 |
0:04:39 |
0:00:00 |
Oppositional-Defiant Disorder |
N/A |
N/A |
N/A |
N/A |
0:00:00 |
0:00:00 |
0:00:00 |
Conduct Disorder |
N/A |
N/A |
N/A |
N/A |
0:00:00 |
0:00:00 |
0:00:00 |
Separation Anxiety Disorder |
N/A |
N/A |
N/A |
N/A |
0:00:00 |
0:00:00 |
0:00:00 |
|
|
|
|
|
|
|
|
Juvenile Justice and Detention |
N/A |
N/A |
N/A |
N/A |
0:00:12 |
0:00:14 |
0:00:13 |
Prison |
0:00:12 |
0:00:11 |
0:00:08 |
0:00:10 |
NA |
N/A |
N/A |
Homelessness |
0:00:09 |
0:00:09 |
0:00:06 |
0:00:08 |
0:00:10 |
0:00:10 |
0:00:10 |
Head Injuries |
0:00:30 |
0:00:23 |
0:00:27 |
0:00:27 |
0:00:30 |
0:00:43 |
0:00:37 |
Stressful Experiences (Post-Traumatic Stress Disorder) |
0:07:07 |
0:09:55 |
0:08:06 |
0:00:00 |
0:03:38 |
0:03:03 |
0:00:00 |
Family Medical History (ACE) |
0:02:55 |
0:03:38 |
0:03:46 |
0:00:00 |
0:03:31 |
0:03:00 |
0:00:00 |
Tobacco, Alcohol, Drugs, and Treatment |
0:05:05 |
0:07:07 |
0:02:02 |
0:00:00 |
0:03:53 |
0:07:43 |
0:00:00 |
Relationships and Social Networks |
0:02:52 |
0:04:35 |
0:04:16 |
0:00:00 |
0:03:01 |
0:02:59 |
0:00:00 |
Childhood Demographics |
0:02:10 |
0:03:15 |
0:02:25 |
0:00:00 |
0:01:46 |
0:01:52 |
0:00:00 |
Childhood Experiences |
0:02:49 |
0:02:57 |
0:02:49 |
0:00:00 |
0:04:19 |
0:04:04 |
0:00:00 |
Employment |
0:01:02 |
0:01:08 |
0:00:48 |
0:00:59 |
0:00:19 |
0:01:00 |
0:00:40 |
Household Roster |
0:01:54 |
0:01:41 |
0:01:24 |
0:01:47 |
0:02:09 |
0:02:10 |
0:02:09 |
Proxy Information |
0:00:00 |
0:00:32 |
0:00:26 |
0:00:29 |
0:00:51 |
0:01:05 |
0:00:58 |
Income |
0:02:32 |
0:04:23 |
0:03:03 |
0:03:19 |
0:03:13 |
0:03:28 |
0:03:21 |
Health Insurance |
0:00:38 |
0:00:57 |
0:00:36 |
0:00:44 |
0:00:43 |
0:00:58 |
0:00:50 |
Verification |
0:00:56 |
0:01:17 |
0:00:48 |
0:01:00 |
0:01:03 |
0:01:06 |
0:01:04 |
Total |
1:05:33 |
1:22:52 |
1:05:18 |
0:38:38 |
1:10:04 |
1:24:09 |
0:43:25 |
Table 6. Timing Results from Parent Simulated Interviews (Shortest Path & Typical Path)
Module Name |
Estimate 1 (shortest path) |
Estimate 3 (typical path) |
||||
Web |
Telephone |
Items |
Web |
Telephone |
Items |
|
Introduction Screens |
0:01:01 |
0:01:45 |
2 |
0:01:01 |
0:01:45 |
2 |
Birth and Early Development |
0:00:20 |
0:00:17 |
4 |
0:00:26 |
0:00:32 |
6 |
Education |
0:01:19 |
0:01:36 |
13 |
0:01:19 |
0:01:36 |
13 |
Columbia Impairment Scale |
0:01:22 |
0:01:54 |
14 |
0:01:22 |
0:01:54 |
14 |
Attention and Concentration |
0:00:50 |
0:00:54 |
9 |
0:02:18 |
0:02:47 |
23 |
Restlessness |
0:01:01 |
0:00:57 |
10 |
0:01:01 |
0:01:04 |
10 |
Problems with Separation |
0:01:03 |
0:01:14 |
11 |
0:01:03 |
0:01:14 |
11 |
Low Mood |
0:00:12 |
0:00:13 |
1 |
0:02:36 |
0:03:17 |
25 |
Anger and Disobedience |
0:00:41 |
0:00:60 |
11 |
0:00:41 |
0:00:60 |
11 |
Breaking Rules |
0:01:16 |
0:01:35 |
17 |
0:01:16 |
0:01:35 |
17 |
Services |
0:02:00 |
0:03:07 |
23 |
0:06:07 |
0:05:54 |
51 |
Family and Medical History |
0:04:01 |
0:07:03 |
65 |
0:04:01 |
0:07:03 |
65 |
Military Service |
0:00:08 |
0:00:09 |
2 |
0:01:22 |
0:01:46 |
7 |
Total |
0:15:14 |
0:21:44 |
182 |
0:24:33 |
0:31:27 |
255 |
The design, instrumentation, and CRS elements of the NMHS Field Test were reviewed by national and international consultants, all considered experts in their fields. These experts, grouped by area of expertise, included:
Statistics and Design
James R. Chromy, PhD, RTI Retired
Former Director of NSDUH Sampling Operations and Statistical Reports
Steven Heeringa, PhD
Associate Director, Survey Research Center
University of Michigan
Adult Psychiatric Epidemiology
Evelyn Bromet, PhD
Distinguished Professor of Psychiatry and Preventive Medicine
State University of New York at Stony Brook
Ronald C. Kessler, PhD, Professor
Department of Health Care Policy
Harvard Medical School
Robert F. Krueger, PhD, Hathaway Distinguished Professor
Department of Psychology
University of Minnesota
Adolescent Psychiatric Epidemiology
Kathleen Merikangas, PhD, Senior Investigator
Chief of the Genetic Epidemiology Research Branch
National Institute of Mental Health
Health Services Research
Benjamin G. Druss, MD, MPH, Director
Center for Behavioral Health Policy Studies
Emory University
Kimberly E. Hoagwood, PhD, Professor
Department of Child and Adolescent Psychiatry
New York University
Mark Olfson, MD, MPH, Professor
Department of Psychiatry
Columbia University Medical Center
Clinical Reappraisal Methods
Ronald C. Kessler, PhD, Professor
Department of Health Care Policy
Harvard Medical School
Kathleen Merikangas, PhD, Senior Investigator
Chief of the Genetic Epidemiology Research Branch
National Institute of Mental Health
Current Incentive Amounts on Other Federal Surveys
Survey Name |
Survey
Component |
Incentive Amounts |
National Survey of Child and Adolescent Well-being (NSCAW) Wave 1 |
|
|
National Survey of Child and Adolescent Well-being (NSCAW) Wave 2 |
|
|
Primary Health Care Patient Surveys |
Patient Survey: 1.1 |
|
National Survey of Family Growth 2006-2007 experiment |
|
If household in either Group 1 or Group 2 did not complete a screener at the end of Phase 1, $5 prepaid incentive was offered to complete the screener in Phase 2. |
National Survey of Family Growth 2009-2012 |
|
$40 |
Panel Study of Income Dynamics (PSID) |
1.28 |
$60 |
National Health Interview Survey (NHIS) 2014-2016 (2014 experiment) |
|
|
Medical Expenditures Panel Survey (MEPS) Household Component, years 2007 to 2010
|
|
$30 per household per interview |
Medical Expenditures Panel Survey (MEPS) Household Component, 2008 experiment |
|
Incentive experiment that included three different respondent payment amounts: $30, $50, and $70. |
Medical Expenditures Panel Survey (MEPS) Household Component, 2014 |
|
|
National Adult Training and Education Survey (ATES) Pilot Study |
ATES Pilot Study questionnaire: 0.25 |
$2 for initial screener mailing
Pilot study also included an experiment:
|
National Longitudinal Study of Adolescent to Adult Health (ADD Health) Wave IV |
|
$40 for Wave IV in 2008-2009. In the final 3 months of data collection, the incentive was increased to $100. |
2011-2012 National Health and Nutrition Examination Survey (NHANES) |
|
Sample people who agreed to the exam component of the survey, which was conducted in mobile examination centers (MEC), could qualify for several monetary incentives.
Base Exam Incentives:
Parental Incentive: Non-sample person parents of sampled people under 16 years $20
Other Exam Incentives:
Transportation Allowance Mileage to MEC: Ranged from $30 to $70 in cities and from $25 to $65 in rural areas. |
Population Assessment of Tobacco and Health (PATH) Study Wave 2 |
|
|
National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) Wave 2 |
|
$40 prepaid incentive and another $40 postpaid incentive for those who completed the interview |
National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) Wave 3 |
Screener uses a full household enumeration process to collect information about the household.
|
|
National Household Food Acquisition and Purchase Survey (FoodAPS) |
|
|
Survey of Consumer Finances (SCF) 2007 |
|
$20 |
Survey of Consumer Finances (SCF) 2010
|
|
$50 |
National Survey on Drug Use and Health (NSDUH) 2010 |
|
|
National Survey on Drug Use and Health (NSDUH) 2015 |
|
$30 upon completion of the interview since 2002 |
Attachment A-1. Adult and Adolescent Questionnaire Specifications
Attachment A-2. Parent Questionnaire Specifications
Attachment A-3. CRS Adult and Adolescent Questionnaire Specifications
Attachment A-4. CRS Parent Questionnaire Specifications
Attachment B. Lead Letter
Attachment C. Unable-to-Contact Letters and Call-Me Letters
Attachment D. Refusal Letters
Attachment E. Incentive Receipt
Attachment F. Appointment Card
Attachment G. Study Description
Attachment H. Introduction and Informed Consent Scripts
Attachment I. Housing Unit and Group Quarters Unit Screening Questions
Attachment J. Question & Answer Brochure
Attachment K. Parent Study Description
Attachment L. Parent Study Informed Consent
Attachment M-1. CRS Data Collection Materials
Attachment M-2. CRS Data Collection Materials (Parent Only)
Attachment N. Introduction and Consent for the Clinical Interview
Attachment O. Confidentiality and Data Collection Agreements
Attachment P. Federalwide Assurance
Attachment Q. NMHS Fact Sheet
Attachment R. NIMH Articles and Information Sheets
Attachment S. SAMHSA Authorization Letter
Attachment T. Showcard Booklet
Attachment U. Parental Introductory Script
Attachment V. Parent Interview Information Card
Attachment W. Quality Control Letter
Attachment X. CRS Cover Sheet and Transmittal Forms
Attachment Y. Sorry I Missed You Card
Attachment Z. Other Language Card
Attachment AA. Non-Clinician Distressed Respondent Protocol
Attachment AB. Quality Control Form
Attachment AC-1. CATI Verification Scripts – Screening Only
Attachment AC-2. CATI Verification Scripts – Interview Only
Attachment AD. Certificate of Participation
Attachment AE. Clinician Distressed Respondent Protocol
Attachment AF. Parent E-mail and Text Prompts
Attachment AG. CRS Unable-to-Contact Letter
Attachment AH. Public Comment from Trevor Project
Attachment AI. Response to Public Comment from Trevor Project
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.
Beebe, T. J., Davern, M. E., McAlpine, D. D., Call, K. T., & Rockwood, T. H. (2005). Increasing response rates in a survey of Medicaid enrollees: the effect of a prepaid monetary incentive and mixed modes (mail and telephone). Medical Care, 43(4), 411-414.
Bielick, S., Cronen, S., Stone, C., Montaquila, J., and Roth, S. (2013) The Adult Training and Education Survey (ATES) Pilot Study: Technical Report (NCES 2013-190). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved 28Nov2016 from http://nces.ed.gov/pubsearch
Biner, P. M., & Kidd, H. J. (1994). The interactive effects of monetary incentive justification and questionnaire length on mail survey response rates. Psychology and Marketing, 11, 483–492.
Center for Behavioral Health Statistics and Quality. (2014). 2012 National Survey on Drug Use and Health: Methodological Resource Book (Section 16a, Mental Health Surveillance Study Operations Report [2008-2012]). Substance Abuse and Mental Health Services Administration, Rockville, MD.
Center for Behavioral Health Statistics and Quality. (2014). 2012 National Survey on Drug Use and Health: Methodological Resource Book (Section 16a, Mental Health Surveillance Study Operations Report [2008-2012]. Substance Abuse and Mental Health Services Administration, Rockville, MD., p.50, Table 7.1.
Center for Behavioral Health Statistics and Quality. (2014) 2014 National Survey on Drug Use and Health: Methodological Resource Book (Section 8), Data Collection Final Report. Substance Abuse and Mental Health Services Administration, Rockville, MD.
Creighton, K. P., King, K. E., & Martin, E. A. (2007). The use of monetary incentives in Census Bureau longitudinal surveys. Survey Methodology, 2.
Dillman, D. A. 2000. Mail and Internet Surveys: The Tailored Design Method. 2nd Edition. New York: John Wiley Co.
First, M. B., Williams, J. B. W., Karg, R. S., Spitzer, R. L. (2015). Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). Arlington, VA: American Psychiatric Association.
Food and Drug Administration’s Research and Evaluation Survey for the Public Education Campaign on Tobacco among LGBT (RESPECT): Supporting Statement. (n.d.). http://www.reginfo.gov/public/do/DownloadDocument?objectID=65626801
Groves and Couper, 1998. Nonresponse in household interview surveys. New York: Wiley.
Groves, R. M., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68(1), 2–31.
Groves, R. M., Singer, E., & Corning, A. D. (2000). A leverage-saliency theory of survey participation: description and illustration. Public Opinion Quarterly, 64, 299–308.
Groves, R. M., Mosher, W. D., Lepkowski, J., Kirgis, N. G. Planning and development of the continuous National Survey of Family Growth. National Center for Health Statistics. Vital Health Stat 1(48). 2009.
Hamilton, C. M., Strader, L. C., Pratt, J. G., Maiese, D., Hendershot, T., Kwok, R. K., Hammond, J. A., Haines, J. (January 01, 2011). The PhenX Toolkit: Get the Most From Your Measures. American Journal of Epidemiology, 174, 3, 253-260.
Han, D., Montaquila, J. M, & Brick, J. M. (2012). An evaluation of incentive experiments in a two-phase address-based mail survey. In Proceedings of the Survey Research Methods Section of the American Statistical Association (pp. 3765–3778).
Harvard Medical School Department of Health Care Policy has posted a copy of the NCS-A Parent Questionnaire here: http://www.hcp.med.harvard.edu/ncs/instruments.php
Jackle, A. and P. Lynn. 2008. Respondent incentives in a multi-mode panel survey: cumulative effects on nonresponse and bias. Survey Methodology 34: 105–117.
Kaufman, J., Birmaher, B., Axelson, D., Perepletchikova, F., Brent, D., Ryan, N. (2013). Kiddie Schedule for Affective Disorders and Schizophrenia for School Age Children–Present and Lifetime Version (K-SADS-PL), 2013. University of Pittsburgh School of Medicine, Department of Psychiatry.
Kennet, J., Gfroerer, J., Bowman, K. R., Martin, P. C., & Cunningham, D. B. (2005). Introduction of an incentive and its effects on response rates and costs in NSDUH. In Kennet, J., & Gfroerer, J. (Eds.), Evaluating and improving methods used in the National Survey on Drug Abuse (DHHS Publication No. SMA 05-4044, Methodology Series M-5). Rockville MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
Kessler, R. C., Berglund, P., Chiu, W. T., Demler, O., Heeringa, S., Hiripi, E., et al. (2004). The U. S. National Comorbidity Survey Replication (NCS-R): design and field procedures. International Journal of Methods in Psychiatric Research, 13(2), 69–92.
Kirlin, J. A., & Denbaly, M. (2013). FoodAPS National Household Food Acquisition and Purchase Survey. US Department of Agriculture, Economic Research Service.
Kulka, R. A., Eyerman, J., & McNeeley, M. E. (2005). The use of monetary incentives in federal surveys on substance use and abuse. Journal of Economic and Social Measurement, 30(2-3), 233–249.
Loranger, A. W., Janca, A., & Sartorius, N. (1997). Assessment and diagnosis of personality disorders: The ICD-10 international personality disorder examination (IPDE). Cambridge, U.K: Cambridge University Press.
Merikangas, K. R., Avenevoli, S., Costello, E. J., Koretz, D., & Kessler, R. C. (2009). National Comorbidity Survey Replication Adolescent Supplement (NCS-A): I. Background and measures. Journal of the American Academy of Child & Adolescent Psychiatry, 48(4), 367-379.
Singer, E. (2006). “Introduction: Nonresponse Bias in Household Surveys.” Public Opinion Quarterly, 70: 637-645.
Singer, E., and Ye, C. (2013). The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science, 645(1), 112–141.
The SAMHSA System of Record Notice covering NMHS is 09-30-0036 and 09-30-0049. See ttp://beta.samhsa.gov/privacy/pia for more information.
Trussell, N., & Lavrakas, P. J. (2004). The influence of incremental increases in token cash incentives on mail survey response is there an optimal amount? Public Opinion Quarterly, 68(3), 349-367.
Zanarini, Mary. (2008). McLean Screening Instrument for Borderline Personality Disorder MSI-BPD. Jones & Bartlett Pub.
1 Center for Behavioral Health Statistics and Quality. (2014) 2014 National Survey on Drug Use and Health: Methodological Resource Book (Section 8), Data Collection Final Report. Substance Abuse and Mental Health Services Administration, Rockville, MD.
2 Zanarini, Mary. (2008). McLean Screening Instrument for Borderline Personality Disorder MSI-BPD. Jones & Bartlett Pub.
3 Center for Behavioral Health Statistics and Quality. (2014). 2012 National Survey on Drug Use and Health: Methodological Resource Book (Section 16a, Mental Health Surveillance Study Operations Report [2008-2012]. Substance Abuse and Mental Health Services Administration, Rockville, MD., p.50, Table 7.1.
4 http://aspe.hhs.gov/datacncl/standards/ACA/4302/index.shtml
5 Harvard Medical School Department of Health Care Policy has posted a copy of the NCS-A Parent Questionnaire here:
6 First, M. B., Williams, J. B. W., Karg, R. S., Spitzer, R. L. (2015). Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). Arlington, VA: American Psychiatric Association.
7 Loranger, A. W., Janca, A., & Sartorius, N. (1997). Assessment and diagnosis of personality disorders: The ICD-10 international personality disorder examination (IPDE). Cambridge, U.K: Cambridge University Press.
8 Kaufman, J., Birmaher, B., Axelson, D., Perepletchikova, F., Brent, D., Ryan, N. (2013). Kiddie Schedule for Affective Disorders and Schizophrenia for School Age Children–Present and Lifetime Version (K-SADS-PL), 2013. University of Pittsburgh School of Medicine, Department of Psychiatry.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | 2003 National Survey on Drug Use and Health |
| Author | Mierzwa, Frank |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |