Mental and Substance Use Disorders Prevalence Study
SUPPORTING STATEMENT
Part B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
B1.1 Target Population
The target population for the Mental and Substance Use Disorders Prevalence Study (MDPS) pilot program is all adults aged 18 to 65 years currently residing in households or in non-household facilities including prisons, jails, psychiatric hospitals, and homeless shelters. The primary objective of the MDPS pilot program is to provide a method for producing national prevalence estimates of schizophrenia or schizoaffective disorder; bipolar I disorder; major depressive disorder; generalized anxiety disorder; posttraumatic stress disorder (PTSD); obsessive-compulsive disorder; anorexia nervosa; and alcohol, benzodiazepine, opioid, stimulant, and cannabis use disorders among U.S. adults ages 18 to 65 years. The MDPS pilot program will calculate mental and substance use disorder prevalence rates by combining a national probability-based household and prison incarcerated sample with targeted convenience samples of institutionalized and homeless non-household populations from facilities recruited by the research team. Four separate populations will be sampled: the household population, the incarcerated population in federal and state prisons, the institutionalized population in state psychiatric hospitals, and the homeless population residing in shelters. To determine the necessity of data collection in jails, a convenience sample of jails will be selected and a feasibility study conducted.
B1.2 Sampling Frame Used and Its Coverage of the Target Population
The household sample design will be a multi-stage, clustered, and stratified area sample of households including group quarters (supported housing facilities, board and care facilities, and assisted living facilities).
The household sample was drawn via a multistage, clustered, and stratified area sampling scheme. At the first stage, the household sample consists of 100 primary sampling units (PSUs), that are counties or county equivalents defined by the Census Bureau in 2019, or groups of counties. A total of 2,273 PSUs were in the final sample frame. The sample was first stratified by Census division and, within each stratum, PSUs were selected with probabilities proportional to size (PPS), as measured by the number of adults living in the PSU. Within each PSU, as the second stage, secondary sampling units (SSUs) that are defined as Census block groups (CBGs) or CBG groups were selected via a PPS sampling scheme to reduce travel costs. At the third stage, address-based sampling (ABS) was used to implement an unclustered random sample of households within the sampled SSUs. In preparation for anticipated non-response rates and other unexpected factors, a reserve sample of households was also selected within each of the sampled SSUs. We will then roster each selected household (including group quarters) that agrees to cooperate and will identify adults eligible for a subsequent clinical interview. For non-group quarter households with one or two eligible individuals, we will select all individuals for screening; for those with three or more eligible individuals, we will randomly select two individuals for screening. A maximum of two eligible individuals will be selected to complete a mental health screening interview in the group quarters as well. To determine sample overlap with the non-household populations, we will include screening questions about the history of prison, jail, and state psychiatric hospital institutional stays and periods of homelessness.
Non-household Sample
The core of the MDPS pilot program will be the household survey, but a key feature of the project will be the inclusion of non-household populations: prisons, state psychiatric hospitals, homeless shelters, and jails. The prison sample will be a national probability sample and will be merged with the household sample to help provide a method for making combined national mental health prevalence estimates. The remaining three non-household populations (hospitals, homeless shelters, and jails) will be limited to the sites selected by the university partners (Columbia, Duke University, University of Washington). The sampled facilities will be purposely selected with the goals of developing strategies for surveying these populations and studying how the results from these limited efforts may affect the overall national estimates.
B1.3 Design of the Sample (Including Any Stratification or Clustering)
Primary Sampling Unit
The primary sampling unit (PSU) area frame will be a list of U.S. counties or county equivalents, defined by the Census Bureau in 2019. Small counties, with a measure of size (MOS) less than a pre-determined threshold, will be combined with neighboring counties to form a single PSU with at least a minimum MOS. To avoid a certainty selection, counties with a large MOS will be divided into smaller PSUs, along Census tract boundaries. The cut points for combining or splitting counties will be determined based on a review of the population distribution of counties. The total number of PSUs will be approximately 3,000, the number of counties in the United States. Each PSU on average will contain about 65,000 households although the actual size will vary considerably.
A PSU’s MOS will be the total population 18-64 years old, based on the 2019 American Community Survey (ACS) 5-year estimates (release date, December 19, 2020), aggregated from tract-level estimates. The MOS age range differs slightly from the target population, because ACS only creates tract-level estimates for the age range 18 to 64, rather than 18 to 65. This small deviation will have no meaningful impact on the study.
The sample will be stratified by geography and population density. In the case of the MDPS pilot program, the data collection is relatively challenging in length, complexity, and sensitivity. Besides, the clinical interview requires trained clinicians rather than lay interviewers. The 100 PSU design is a compromise between the competing goals of sampling precision, which calls for a larger number of PSUs, and data collection feasibility, which calls for fewer.
The Secondary Sampling Unit (SSU) area frame will be a list of Census block groups (CBGs) within selected PSUs, as defined by the Census Bureau in 2019. A CBG is a geographical unit between a Census tract (that are small, relatively permanent statistical subdivision of a county or equivalent entity and can be further divided into Census blocks) and a Census block (statistical area bounded by visible features), with an average of approximately 550 occupied housing units per CBG. CBGs are the smallest geographical area for which the Census publishes ACS tabulations. CBGs with fewer than 250 locatable address units will be combined with neighboring CBGs to form a single SSU. The MOS will be the total population aged 18-64, based on 2019 ACS 5-year estimates at the CBG level. A random sample of 800 SSUs, 8 per PSU, will be selected with PPS. While every SSU has a chance (probability) to be selected, the selection probability is proportional to the population size (PPS).
Address based sampling (ABS) frames are a high-quality, cost-effective household sampling frame for in-person, mail, and multi-mode surveys of populations in residential households. The foundation of ABS frames is the U.S. Postal Service’s Computerized Delivery Sequence file, which is made available through non-exclusive license agreements with qualified private companies (Iannacchione, 2011). RTI maintains in-house a national ABS frame that is licensed from one of only two nationally qualified vendors and receives monthly updates to ensure that the frame data remain current.
The MDPS pilot program’s ABS frame will be limited to residential addresses whose geocodes are within the selected SSUs. The frame will include both city-style addresses and non-locatable addresses like OWGM1 PO boxes (McMichael and Brown, 2018), highway contract, and rural route boxes.
Selection Process for the Clinical Interview
The relatively low prevalence of high-priority conditions (e.g., an estimated 1% prevalence rate for schizophrenia or schizoaffective disorder) in the U.S population necessitates a mental health screening interview completed prior to the clinical interview. Therefore, once the household roster is completed by a household member, the rostering process will select up to two adults to complete the computerized mental health screening interview. Based on the screening results, one or both adults may be invited to participate in the clinical interview. The process of selecting participants for the clinical interview consists of first classifying them into one of three mutually exclusive strata, A, B, and C (see below), based on their responses to the computerized screening instrument. The proportion of participants selected within each stratum will be pre-determined. For example, we will select all participants predicted to have the highest probability of high-priority conditions based on their screener responses and so assigned to Stratum A. Because the high-priority conditions for MDPS include disorders with very low population prevalence (see above), this strategy both maximizes the potential number of individuals with the given rare disorder being interviewed and increases precision of the prevalence estimates. Selection probabilities within each clinical interview stratum, to determine who is selected for the clinical interview, will be set and adjusted as needed. Participants assigned to Strata B and C (see below) will be selected at lower rates for the clinical interview than will those assigned to Stratum A.
The following is an example of our preliminary proposed hierarchical structure and selection assumptions. Please note that these assumptions differ slightly depending on the screening instrument used, the Computerized Adaptive Testing for Mental Health Disorders (CAT-MH) or the World Health Organization’s Composite International Diagnostic Interview (CIDI), because of differences in the screening protocol items:
Stratum A (assumed 100% selection for clinical interview): All participants who (i) screen positive for psychosis on the CIDI or CAT-MH or report a manic episode of 8 days or more in the CIDI screening; or (ii) report receipt of a Supplemental Security Income/Social Security Disability Insurance benefit because of schizophrenia or schizoaffective disorder; or (iii) report being told by a physician they have a diagnosis of schizophrenia or schizoaffective disorder.
Stratum B (assumed 80% selection for clinical interview): Participants not in Stratum A who (i) screen positive for a mental or substance use disorder based on the CIDI or CAT-MH screening instrument; (ii) report receipt of a Supplemental Security Income/Social Security Disability Insurance benefit for any other psychiatric disorder other than schizophrenia or schizoaffective disorder; or (iii) report being told by a physician that they have depression, mania, or other seriously impairing mental disorder; or (iv) have signs or symptoms consistent with anorexia nervosa.
Stratum C (assumed 20% selection for clinical interview): Participants neither in Stratum A nor B.
The objective is to sample a sufficient number of screened persons for the clinical interview, to provide a solid basis for estimating prevalence rates not only for schizophrenia or schizoaffective disorder, but also for other disorders.
The non-household design includes 4 non-household populations: prisons, state psychiatric hospitals, homeless shelters, and jails. The prison sample will be drawn from a national probability sample. The remaining three population surveys (hospitals, homeless shelters, and jails) will be limited to convenience samples recruited by the study’s three partner sites (Duke University, Columbia University, and University of Washington).
Sample Design for the Prison Population
A probability-based multistage sampling design will be implemented to collect data for the long-term incarcerated population—those confined in federal and state prisons. A probability-based, methodologically rigorous survey of this population is feasible, and will serve to provide more complete coverage of the non-household population, traditionally overlooked in national surveys of mental health. We will first select a nationally representative sample of prison facilities, with probability proportional to size (PPS) (i.e., the number of inmates within each prison facility), which will facilitate integration with the national household survey. A sampling frame was provided by the Bureau of Justice Statistics (BJS), the primary statistical agency of the Department of Justice. Then, roster data provided by cooperative facilities will be used to select a random probability sample of inmates within each facility. Because SMI prevalence rates are expected to be higher among the prison population than among the general population, there will be no screening conducted in prisons.
The target number of cooperating prisons is 36, a number determined by the needs to fill multiple strata and maintain a low burden on each prison facility. However, some facilities may prove to be ineligible (i.e., they do not house adults) and other may not agree to participate. Therefore, we will select a main sample of 36 facilities and a reserve sample of 14 facilities, to compensate for ineligibility and nonresponse.
Within each facility, we plan to select around 20 inmates with a target total sample size of 714 inmates. Assuming the response rate among the selected inmates is 70% (United States Bureau of Justice Statistics 2004), we expect to complete 500 clinical interviews among the long-term incarcerated population.
After the Bureau of Prisons and relevant state departments of corrections approve the study, we will obtain the roster with inmates from each of the 36 participating facilities. Within each prison facility, the roster file will be filtered to only keep inmates who are aged 18 to 65 and are sentenced. Then, the filtered roster will be sorted by some key characteristics of inmates (e.g. housing unit, age, length of stay, race and ethnicity) and a random probability sample of inmates will be selected from the sorted roster via a systematic sampling scheme.
State Psychiatric Hospital Sample
The MDPS pilot program will collect data in four state psychiatric hospitals recruited by our university partners. We will acquire from each hospital agreement and permission to implement the survey, to sample patients, and to have access to information that would ensure an unbiased sample. We will use substitution if a hospital refuses to participate in the study.
Sex offenders will be excluded from the state psychiatric hospital sample. Sex offender residents may or may not have psychiatric conditions and typically are not housed within the psychiatric hospital because of any mental illness. Also, only those age 18 to 65 will be eligible for selection. Beyond age and sex offender status, no other state psychiatric hospital patient type will be excluded from participation. Selected patients will include individuals with mental illness, including both non-forensic and forensic patients (other than sex offender status), including those determined incompetent to stand trial.
We will work with each hospital administration to develop a roster and draw the sample. The hospital might be able to identify ineligible patients in advance or provide us with relevant flags, while we will be prepared to draw the sample. Ideally, we will obtain a roster with information on the currently hospitalized eligible population that includes age, gender, length of stay, and capacity to consent. Similar to the way in which the prison inmates are selected, we will use an implicit stratification sampling technique to first sort the roster by age and length of stay, and then select a random probability sample of patients via systematic sampling.
Our goal is to complete 200 clinical interviews across all the four hospitals (around 50 per hospital). Assuming a 60% response rate, we need to draw a sample of 333 adults.
Our goal will be to sample 18 homeless shelters recruited by our university partners. Anticipating some non-response from the shelter administration, we will identify more than 18 shelters for our recruitment effort. A complete list of all shelters2 will be compiled by the university partners, along with shelter characteristics such as the age and gender of the population served, the sponsor of the shelter (e.g., church, county, state), the type of facility (e.g., mental health treatment, sobriety-only), the total number of beds, and the average number of occupied beds per night. Shelter characteristics will be used to purposefully sample shelters for participation in the MDPS pilot program. The goal will be to deliberately select shelters that represent diversity (e.g., male- and female-only shelters, church-sponsored). We will use substitution if a selected shelter refuses to participate in the study.
We will contact each shelter administrator to request cooperation with the study. Once recruited, we will sample residents within each shelter on the day of the visit. Up to 4 visits per shelter will take place over the data collection period. Homeless shelters can pose data collection challenges related to the transient nature of the individuals who receive services there; people served at a shelter one day may not return the next day and some residents may only access services in the evenings. In addition, different shelters have different record-administration systems (e.g., hard-copy lists or electronic files) and organization structures (i.e., operated as one component of a broad public service agency assistance program versus an independent walk-in facility), which make it impossible to create a unified sampling method that can be applied to all shelters. Therefore, we will develop a customized sampling approach for each shelter, based on their ability to provide resident information, such as the number of adult residents or all residents, the accessibility of their resident list, the length of stay of residents, and the flow of residents’ arrivals.
Based on the information each shelter provides, we will first compile a resident list for each shelter. The residents will then be selected from this list via systematic sampling. We expect a total of 500 completed clinical interviews from the 18 shelters (around 28 per shelter). Assuming a 60% response rate, our plans call for an original sample total of 833 residents in homeless shelters. This means, to reach this goal in one one-week visit, the sample size will be set to 46 per shelter, to enable completion of 28 clinical interviews during the one-week visit. If this number of interviews is not achieved and the shelter is amenable to additional data collection sessions, they will be scheduled far enough in advance, to ensure that new shelter residents are available to participate. To account for variation across different shelters, for each shelter, the sample size per visit will be determined based on the number of visits we plan to make to this shelter and the sample size in later visits can be adjusted based on the response rates from our previous visits.
The jail component of the MDPS pilot program is a feasibility study and will include data collection in at least one jail identified by a university co-investigator. Sample jails will be contacted in advance, to ensure understanding of the survey and agreement with its implementation.
Once the jails are selected, agree to participate, and obtain the required approval we will obtain the roster about inmates from each selected jail. In each jail we plan to sample approximately 75 inmates, ages 18 to 65 and incarcerated during the data collection period of the visit. The roster file will be first filtered to keep eligible inmates, and then sorted by some key characteristics of inmates (e.g., age, length of stay, race and ethnicity). A random probability sample of inmates will be selected from the sorted roster via systematic sampling. Given the high turnover rates and short average duration of stay, the jail sample will initially only be invited to complete the mental health screener. The expected cooperation rate among the jail population is 70%, resulting in up to 208 completed mental health screening interviews, or approximately 35 respondents per jail. Upon release from jail, all participating inmates will be invited to complete the clinical interview, again for feasibility purposes.
B1.4 Size of the Sample and Precision Needed for Key Estimates
Because of the rarity of schizophrenia and schizoaffective disorder and of their clinical and public health significance, a high priority will be placed on positioning the study sample to accurately estimate schizophrenia/schizoaffective disorder prevalence. So, for sample size, we specifically focused on precision for estimating schizophrenia/schizoaffective disorder. Previous studies suggest that the prevalence rate of schizophrenia/schizoaffective disorder in the household population is low, approximately 0.5% to 1%, but the prevalence is higher in various non-household populations such as the incarcerated population (Bourdon et al. Estimated prevalence of schizophrenia in US adults from ECA survey, 1992). The SCID instrument used for the MDPS will not differentiate schizoaffective from schizophrenia. The diagnostic specificity measured will be a) schizophrenia or schizoaffective lasting at least 6 months, and b) schizoaffective and schizophreniform lasting at least one month. Greater diagnostic specificity would have required a significantly lengthier SCID interview for this module thereby increasing the interview length substantially.
With a schizophrenia or schizoaffective disorder point prevalence estimate of ~1% and a requirement of high precision, a sample size of 7,200 completed clinical interviews offers a good compromise between cost and precision, with an estimated 95% confidence interval (0.6%, 1.4%), assuming a design effect of 2. Given the importance of the non-household population, we allocate one-third of the ultimate target sample size (ranging from 390 to 510) of completed schizophrenia or schizoaffective disorder cases to the non-household populations. The three non-household populations together account for less than 5% of the total U.S. population. But, for a study of schizophrenia or schizoaffective disorder and other rare seriously impairing disorders, that relatively small population is important.
The household population is expected to have the lowest schizophrenia or schizoaffective disorder prevalence, requiring a large original sample size and an efficient screening process. We propose an initial sample of ~45,000 households for screening and selecting approximately 8,000 individuals for the clinical interview. Assuming at this stage a response rate of 75% and an adjusted schizophrenia or schizoaffective disorder prevalence rate of 4%-6% (because of sample stratification), we expect 240-360 completed clinical interviews with schizophrenia or schizoaffective disorder in the household sample. If the schizophrenia or schizoaffective disorder prevalence rate were 1.0%, the resulting 95% confidence interval of the estimate of the schizophrenia or schizoaffective disorder prevalence rate for the household sample would be (0.6%, 1.4%), assuming a design effect of 2.
The literature confirms higher prevalence rates for the non-household populations, and we expect to obtain approximately 150 completed clinical interviews with schizophrenia or schizoaffective disorder from the non-household population. Combining the household and non-household samples will yield a total of 390 to 510 schizophrenia or schizoaffective disorder cases which, even with a design effect from oversampling and weighting, will provide a sound basis for estimating national prevalence rates and for analyzing characteristics of this high-priority special population.
The estimated sample sizes and expected completes for the household and non-household populations are presented in Exhibit 1.
Exhibit 1. MDPS Pilot Program Expected Case Counts for Prevalence Estimates.
|
Number of completed household rosters/Target number of rostered facilities |
Number of adults to be selected for screening |
Expected number of completed screening interviews |
Number of adults selected to complete the clinical interview |
Expected number of completed clinical interviews |
Expected number of completed clinical interviews for adults with psychotic disorders |
Household |
45,000 |
60,000 |
45,000 |
8,000 |
6,000 |
240-360 |
Prison |
36 |
NA |
NA |
714 |
500 |
50 |
Hospital |
4 |
NA |
NA |
333 |
200 |
50 |
Shelter |
18 |
NA |
NA |
833 |
500 |
50 |
Jail |
6 |
298 |
208 |
208 |
63 |
15 |
Total |
45,000/64 |
60,298 |
45,208 |
10,088 |
7,263 |
405-525 |
B2.1 Data Collection Procedures
Household data collection involves the use of household rostering and screening processes to select an adult household member for the clinical interview. Clinical interviews will be conducted during a separate visit for selected adults who agree to the interview.
All household data collection is designed to be conducted in multiple modes in sampled households. The rostering and screening can be completed online, by phone, on paper, or in-person. A tablet computer is used for in-person rostering and screening. The clinical interview can be completed by video interview (i.e., Zoom), by phone or in-person. A laptop computer is used to administer the clinical interview and a tablet computer is used for the video and phone connection, as well as to record the interview for quality control purposes. Field interviewers (FIs) will be responsible for rostering and screening activities in the selected households. Clinical interviewers (CIs) will be responsible for scheduling and conducting the clinical interview with all adults selected from the screening process who agree to the interview.
Prior to household visits, every candidate household will be sent a lead letter and a study brochure inviting an adult household member to complete the roster online or by phone. Up to seven (7) follow-up letters and postcards are mailed to households that have not responded at the time of the mailing. This includes a thank you/reminder letter at week 1, a postcard at week 2, a paper roster at week 3, a reminder at week 6 and a postcard at week 7. Each of these mailings invites an adult household member to complete the household roster online or by phone. A subset of households will also be contacted by phone and invited to complete the roster by phone. This outbound calling is only conducted for a subset of households, whereas all will be offered the opportunity to call in to the central office to complete the household roster. Following these invitations, a letter is sent in advance of the field interviewer visiting sampled dwellings to complete the household roster in person. At the time of the in-person visit, the FI will confirm that the individual is a resident age 18 or older. S/he will then explain the reason for the visit, describe the COVID-19 safety protocols in place, answering questions and addressing concerns as needed, before asking the adult resident to complete the household roster. While the roster administration time varies depending on the number of adults living in the home, average completion time is currently 8 minutes. These materials are included in Attachment H.
Once the roster is populated, a sampling algorithm will immediately indicate whether zero, one or two adults in the household have been selected to complete the screening interview. If the household member who completed the roster is selected, the FI will notify him/her of the selection and ask if s/he is willing to complete the screen. If another adult in the household is selected, the FI will ask to speak to that person. The screening interview can be completed by the selected respondent using the FI’s tablet computer. This procedure ensures confidentiality at the time of the visit, given the sensitive nature of the screening questions. If the selected person is not home or is unable to complete the screen during the household roster visit, the FI will ask for the best time to reach the person and will leave a card for him/her indicating a return visit at a more convenient time. The FI will then thank the household member for completing the roster and make plans to return to speak with the selected person later. The FI will also provide information for either respondent to complete the screen online using a unique ID created for each individual. The unique ID and website login information are sent by email to each selected adult in the household within several hours of completing the household roster. This web screen will be accessible to the respondent via a unique secure link accessible on any computer, laptop or mobile device (i.e., tablet or smart phone) throughout the duration of the study period.
The brief screen is a self-administered instrument on a tablet hand-held computer and is designed to take no more than 15 minutes to complete. An informed consent statement is presented at the start of the screening interview. Whether the respondent is selected for a clinical interview will depend on his/her responses to the items in the screen.
Respondents who complete the screening questionnaire will be notified immediately whether they have been selected for the clinical interview. If not selected, the respondent will be asked for their preferred mode to receive the screening incentive (cash, check, electronic gift card) and contact information will be collected. If the screening is conducted in-person, the FI will then thank the respondent for his/her time. If selected, the respondent will be informed of their eligibility to participate in the clinical interview, asked if they are willing to do so, and then asked for their preferred mode for the screener incentive and for their contact information. Next, they will be presented with dates and times to schedule their clinical interview. If the clinical interview is scheduled electronically by the respondent, an email will then be sent to the respondent confirming the date and time of the interview. The recruitment status will be updated in the case management system and contact information for recruited respondents will be shared with the assigned clinical interviewer. The clinical interviewer (CI) assigned to the case will then send a secure Zoom link for the appointment date and time.
Trained CIs will conduct the clinical interviews by video, phone or in person using a laptop and tablet computer. Clinical interviews will be conducted in a private setting for both the respondent and the CI, such as a home office. The CI will administer informed consent process and obtain consent to record the interview, before beginning the clinical interview. Eligible people who are never reached by the CI, never schedule an appointment for the clinical interview, or miss their appointment and are not reachable may be contacted by an FI to remind them of the invitation to participate. An option to schedule the interview on their own using a web-based, self-scheduling system will be provided as well.
The CI will attempt to contact the respondent within 48 hours of receipt of the case to schedule an appointment for the clinical interview. If unable to reach the respondent on the first attempt, the CI will continue trying to reach the respondent during the preferred time frames, up to 3 additional times. After 3 or 4 unsuccessful attempts to reach the respondent during the preferred times, the CI will make 3 more attempts outside of those preferred time frames. If the CI reaches the respondent, he/she will verify that the correct respondent has been contacted and will schedule an appointment for the in-person interview. If a respondent avoids or refuses to schedule the interview at this point, the CI will probe for a reason and, if appropriate, will offer to call back at another time. Email and text reminders will be sent to respondents to remind them to schedule their interview. Once an appointment is scheduled, reminder emails or texts are sent in advance of the appointment. Emails and text reminders are also sent to respondents if a previously scheduled appointment is missed.
Administration time for the clinical interview will vary depending on the extent of a respondent’s mental health problems. The average completion time is approximately 80 minutes. Once the interview is complete, the respondent will receive a $30 electronic gift card.
Lead letters will be mailed to each candidate facility. The lead letter will serve two purposes: describe the study and describe what facility participation would entail. Additional study information will be included in the mailing in the form of a Frequently Asked Questions brochure and a facility data collection document. Facility administrators will be contacted to address any questions and to determine whether administrative approval is required. Research packets and supporting documentation will be provided to the facility regulatory board, if required. Once approval is obtained, facility points of contact will be asked for information about the facility, including the resident population size, whether there are special units within the facility (e.g., psychiatric units), what types of space would be available for group administration of the MDPS pilot program screening, information about any background clearance that is required for field staff, whether there is a need for special review (e.g., research approval or IRB), and any issues around privacy or consent (e.g., a requirement for a facility consent and study consent). Facility engagement will be documented to the degree required by the facility (e.g., through business agreements or memoranda of understanding).
MDPS pilot program facility points of contact will also be responsible for coordinating the collection of a facility roster, effective a specified date. Facility roster information will include fields such as resident name, age, sex, race/ethnicity, length of stay at the facility, and the location of the resident’s home. Approximately two weeks prior to data collection, the facility will be asked to send an electronic copy of the roster to the MDPS pilot program contractor. One week prior, a second roster will be sent to confirm those that are eligible for selection. A random sample of residents within each facility will then be selected to participate in the 80-minute clinical interview.
MDPS pilot program clinical interviews will be conducted as similarly as possible in both household and non-household settings. As in the household setting, CIs will conduct one-on-one interviews within facilities in a private location. The CI will schedule multiple interviews on each day to maximize efficiency, while still allowing sufficient time for each interview. If a person scheduled for an interview is not available, the CI will attempt to interview the person the next time he/she visits the facility. Non-household interviews will also be conducted virtually via videoconference, by phone or in-person.
Consent will be obtained in advance of the clinical interview and either a monetary or non-monetary incentive will be provided to the respondent upon completion of the clinical interview. A series of knowledge check questions will be administered following informed consent, to ensure the respondent understands the invitation and has the capacity to consent. During the interview, if the respondent becomes confused or is unable to respond adequately, the Short Blessed Test (Katzman, 1983), a screening tool to detect cognitive dysfunction, can be administered to determine capacity to continue. The incentive provided will be determined by each site using the guidelines provided by the study team. Study information on the format of frequently asked questions (FAQs) will be provided to respondents and COVID-19 safety protocols will be reviewed prior to the start of the interview. All respondent materials for non-household recruitment and data collection are included in Attachment I. All household and non-household instruments and respondent materials will be available in both English and Spanish.
Data Management
This study poses minimal risk to subjects, and it was reviewed and approved by the Advarra Institutional Review Board on March 5, 2020 and by the Office for Human Research Protection on April 8, 2020. However, a possible risk to subjects participating in the MDPS pilot program is the loss of privacy or confidentiality and the risk of identification. While the information collected in the clinical interview is sensitive, no personally identifiable information will be collected within the survey instrument. Identifiers uploaded within the case management systems for survey invitation distribution and to access the rostering and screening instruments will be removed from all data sets immediately after the raw data are downloaded. Any versions of the data set that contain identifiable data from the facility rosters will be destroyed after we obtain the deidentified data set.
Steps taken to minimize risks and to protect subjects’ welfare:
Data will be stored behind firewalls and with limited access within RTI’s secure portal. Only members of the research team will have access to the de-identified survey data and paradata.
Identifiable subject information used to contact respondents and for survey distribution (e.g., household addresses, email addresses, cell phone numbers) and deidentified survey responses will be kept in separate, encrypted files.
Survey results will only be reported at the aggregate level.
B3.1 Expected Response Rates
There is assumed to be a 60% response rate for the MDPS pilot program household screening interview. This is consistent with the screening response rate achieved on the recent 2017 National Mental Health Study, which used a similar household rostering and screening procedure. Of those adults who are selected and invited to participate in the clinical interview, the expectation is that 75% will accept and complete the interview. For the non-household sample, the assumed clinical interview response rate is 70% for the prison sample and 60% for the hospital and shelter samples.
B3.2 Maximizing Response Rates
The MDPS pilot program’s ability to gain the cooperation of potential respondents is key to its success. A household respondent’s willingness to participate first in the rostering process, then in the screening, and ultimately in the clinical interview is affected by a combination of circumstances unique to each selected respondent. Because the MDPS pilot program’s goals place such a heavy emphasis on vulnerable populations often not included in traditional survey efforts, we anticipate it may be a challenge to achieve a 75% response rate with this target population. These challenges have increased multifold by the onset of the COVID-19 pandemic, as the initial plans were to complete approximately 80% of the interviews in-person. However, there are several strategies to help maximize the MDPS pilot program response rates, starting with a multimode data collection design that is informed by the severely impeded in-person data collection design.
People vary in how they can respond (e.g., access to the internet) and, relatedly, in their likelihood to respond using a particular mode. Our design includes web, paper, telephone, and when and where possible, in-person data collection. This multimode design aims to achieve high participation rates and reduce the risk of nonresponse bias related to any single mode (e.g., internet access) or set of modes with common limiting features (e.g., illiteracy).
Incentives increase participation (e.g., Singer and Ye, 2013) and reduce the risk of nonresponse bias (Groves, Singer and Corning, 2000; Groves, Presser and Dipko, 2004; Groves et al., 2006), and play a particularly important role in a self-administration-centric design. As explained in Supporting Statement A—supported by other studies, the relatively large respondent burden in the MDPS pilot program three-stage design, and the multimode design with substantial reliance on self-administration imposed by COVID-19—cash incentives will be offered at different stages in the design: $2 prepaid household incentive, $10 for rosters, $20 for mental health screening interviews, and $30 for clinical interviews. In non-household populations, the equivalent of a $30 incentive will be offered, where feasible. For those who may be unable to accept a cash incentive (e.g., prison or jail inmates), non-monetary incentives will be offered.
The MDPS pilot program will utilize field study procedures designed to maximize respondent participation. This process will begin with the assignment of cases prior to the start of data collection, accompanied by weekly response rate goals that will be conveyed to the field interviewers (FIs), field supervisors, clinical interviewers (CIs) and clinical supervisors. When making assignments, supervisors will consider many factors, including which interviewers are in closest geographic proximity to the sampled households, interviewer skill sets, as well as basic information (demographics, size, etc.) about the specific non-household facility or household segment. Supervisors will assign cases to FIs, in order to ensure maximum production levels at the start of the data collection period. To successfully complete work in remote segments or in areas where no local FI or CI is available, a traveling FI from another area will be utilized to prevent delays in data collection.
Once interviewers upload interviews into the secure system, interview data will be processed and summarized in daily reports posted to a web-based case management system (CMS), which can be accessed by supervisors and which requires two-factor authentication for log-in, as part of a Federal Information Processing Standard (FIPS)-Moderate environment. Supervisors will be able to produce reports on the CMS to review response rates, production levels, and records of call information to determine specific interviewer’s progress toward weekly goals, to determine when interviewers should attempt contact, and to develop plans to handle challenging cases (e.g., refusal).
Interviewers may use a Sorry I Missed You Card or Appointment Card to help initiate respondent contact and to encourage participation. To aid in refusal conversion efforts, Refusal Letters tailored to specific refusal reasons can be sent to any respondent who has refused. Similarly, an Unable-to-Contact Letter may be sent to a selected household if the field interviewer has been unable to contact a resident after multiple attempts. For cases where field interviewers have been unable to gain access to a group of housing units due to some type of access barrier, such as a locked gate or doorperson, Controlled Access Letters can be sent to the gatekeeper to obtain his/her assistance in gaining access to the units.
If these attempts fail, a Call-Me Letter may be sent directly to a selected household. These letters inform the residents that a field interviewer has been trying to contact them and asks that they contact the field supervisor (FS) by phone. If the resident calls the FS, the FS will attempt to get the resident to agree to an appointment so the FI can return to that address and roster the household in person.
Clinical interviewers will send text messages as well as emails to respondents selected for the clinical interview, to offer ways in which an interview can be scheduled, to remind the respondent of a scheduled interview, and to follow up if the respondent misses their clinical interview appointment.
These respondent materials are included in Attachment H.
B3.3 Dealing with Nonresponse
In order to produce unbiased estimates, sampling weights will be used in such a manner that they account for the unequal sample selection at various stages and that they adjust for nonresponse, poststratification, and extreme values. This process is intended to reduce bias as well as reduce variance of estimates. The sampling weights will be derived from each sampling stage and will be calculated as the inverse probability of selection for the unit of observation of each stage. These weights, referred to as design-based weights, will be calculated when the samples are selected. After data collection, the design-based weights will be adjusted to account for nonresponse, under- and over-coverage of certain demographic groups, and extreme weights, resulting in fully adjusted sample weights.
A sample weight will be created for each key respondent, reflecting the varying probabilities of selection. Additionally, a generalized exponential model (GEM) will adjust for unit nonresponse, coverage error, and extreme weights. The weight will account for the purposive oversampling of individuals at high risk for mental disorders and psychotic disorders, as well as for nonresponse bias.
No pretests will be conducted. However, a pilot was conducted during the first two months of data collection to test various paths of the clinical interview and to obtain instrument and section level timings. The clinical interview instrument was revised to decrease repetition and to remove content that was unnecessary for the purposes of this study. All instruments were tested in all modes prior to fielding. As described in Supporting Statement Part A, many of the methods have been developed through prior studies (e.g., the National Survey on Drug Use and Health, National Mental Health Survey).
RTI International developed the plans for statistical analyses for this study. The team was led by the following individuals:
Heather Ringeisen, RTI International, MDPS PILOT PROGRAM Principal Investigator
Karol Krotki, RTI International, MDPS PILOT PROGRAM Design Contract Lead Statistician
Jill Dever, RTI International, MDPS PILOT PROGRAM Analysis and Reporting Lead
Andy Peytchev, RTI International, MDPS PILOT PROGRAM Study Design and Methodology Lead
Dan Liao, RTI International, MDPS PILOT PROGRAM Sampling Lead
References
Bourdon, K. H., Rae, D. S., Locke, B. Z., Narrow, W. E., & Regier, D. A. (1992). Estimating the prevalence of mental disorders in U.S. adults from the Epidemiologic Catchment Area Survey. Public Health Rep, 107(6), 663–668.
Katzman, R., Brown, T., Fuld, P., Peck, A., Schechter, R., & Schimmel, H. (1983). Validation of a short Orientation-Memory-Concentration Test of cognitive impairment. The American Journal of Psychiatry.
Kessler, R. C., Birnbaum, H., Demler, O., Falloon, I. R. H., Gagnon, E., Guyer, M., Howes, M. J., et al. (2005). The prevalence and correlates of nonaffective psychosis in the National Comorbidity Survey Replication (NCS-R). Biological Psychiatry, 58(8), 668-676.
Phillips, M. R., Zhang, J., Shi, Q., Song, Z., Ding, Z., Pang, S… Wang, Z. (2009). Prevalence, treatment, and associated disability of mental disorders in four provinces in China during 2001–05: An epidemiological survey. The Lancet, 373(9680), 2041-2053.
Singer, E., & Ye, C. (2013). The Use and Effects of Incentives in Surveys. The ANNALS of the American Academy of Political and Social Science, 645(1), 112-141. doi:10.1177/0002716212458082
van Os, J., Hanssen, M., Bijl, R.V.& Vollebergh, W. (2001). Prevalence of psychotic disorder and community level of psychotic symptoms: An urban-rural comparison. Archives of General Psychiatry, 58(7), 663-668.
Attachment A. Household Roster
Attachment B. Household Roster: PAPI Instrument
Attachment C. Household Screening Instrument
Attachment D. Household Screening Instrument: PAPI
Attachment E. Clinical Interview
Attachment F. Federal Register Notice (to be attached once received)
Attachment G. Informed Consent Forms
Attachment H. Household Respondent Materials
Attachment I. Non-household Facility Recruitment Materials
Attachment J. Non-household Respondent Materials
1OWGM stands for “Only Way to Get Mail”.
2 Each collaborative site will be reminded that they should consider removing shelters with the following characteristics from the list: 1) known to be impossible or very difficult to recruit due to their policy; 2) unique and unrepresentative of other shelters.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Grace Medley |
| File Modified | 0000-00-00 |
| File Created | 2022-03-30 |