SOCIAL SECURITY ADMINISTRATION
MENTAL HEALTH TREATMENT STUDY
Supporting Statement Part B
OMB No. 0960-0726
This study will use a randomized trial to evaluate the efficacy of an intervention (versus treatment as usual) for improving functioning and employment outcomes for SSDI beneficiaries. We selected twenty-two demonstration sites, located in diverse geographic areas, for the study. We selected these sites because their programs integrate evidence-based mental health and supported employment services. All sites have high fidelity to the supported employment model known as Individual Placement and Support (IPS). These sites have the best existing programs for testing the intervention. We intend the sample selected for the study to represent beneficiaries who reside within the catchment areas of these sites and who would be eligible for the program. The size of this population is currently unknown. However, if effective, we will eventually make the intervention available to all SSDI beneficiaries with either schizophrenia or affective disorder; there were 1,358,000 such persons as of June 2003.
This section describes the selection of the sample for recruitment of beneficiaries into MHTS in the geographic catchments surrounding the selected demonstration sites. The steps in this process include the development of the list of potential study participants, initial screening for eligibility, stratification, acquiring and refining contact information, determination of demonstration site sample sizes, sample selection, and processing the sample into release groups.
Obtain Beneficiary Data from SSA
The first step in this task will be to request a file from SSA of eligible SSDI beneficiaries in geographic areas surrounding the study demonstration sites. The request to SSA will include administrative data plus information for study inclusion and exclusion criteria. The administrative data will include name, telephone number, address, social security number, and other contact information.
Ideally, we would target the geographic areas closely toward where potential study participants live. However, Westat will initially request addresses for all beneficiaries living in ZIP codes located within the catchment area determined by each demonstration site. This radius should be adequate to include all beneficiaries within a reasonable catchment area of the demonstration sites. Furthermore, the demonstration sites are geographically distant enough that there should be no overlap between demonstration site catchment areas. Westat’s mapping and global information system (GIS) department will determine the ZIP codes. This department will also provide maps of the areas surrounding each site showing ZIP codes.
At the same time that we made this request to SSA, Westat worked with the demonstration sites to determine more realistic recruitment catchments around each demonstration site. We use the more restricted areas to narrow the list of potential MHTS participants after obtaining the full set of names from SSA.
Selecting eligible beneficiaries
Each record on the file delivered to Westat included administrative variables and screening variables used to conduct a preliminary assessment of eligibility based on the study’s inclusion criteria and a subset of the exclusion criteria (available on SSA files).
The administrative data appended included name, telephone number, address, social security number, and any other contact information available on the files.
SSA provided administrative data for beneficiaries that meet the study’s inclusion criteria described below:
(1) The beneficiary was receiving SSDI benefits;
(2) The beneficiary had a primary diagnosis of either schizophrenia or affective disorder; and
(3) The beneficiary was between the ages of 18 and 55.
We considered beneficiaries that meet these inclusion criteria provisionally eligible for the MHTS (provided they did not meet the conditions for exclusion), pending verification of the criteria at the enrollment interview.
SSA also identified beneficiaries that meet the study’s exclusion criteria described below:
(1) Living in a nursing home/other custodial institution; or
(2) Life threatening health condition that would make competitive employment impossible; or
(3) Had a legal guardian.
We did not consider these beneficiaries were for enrollment. Beneficiaries selected for potential enrollment based on these criteria will still have these criteria verified during the screener and baseline interview.
In addition to the three exclusion criteria above, MHTS staff later screened out beneficiaries based on the following three criteria.
(1) Failure to pass a screen to assess ability to give consent; or
(2) Currently receiving SE from one of the demonstration sites in sample; or
(3) Competitively employed within the past 30 days.
After Westat selected the sample of potential enrollees, each demonstration site reviewed their sample to identify potential enrollees who were currently receiving SE services. We excluded these beneficiaries from the sample prior to making recruitment calls.
Demonstration Site Sample Sizes
The MHTS recruited and enrolled 2,238 beneficiaries - we randomized 1,121 to the treatment group and 1117 to the control group. The sample at each demonstration site varies depending on the sample available in each site’s catchment.
Stratification
The final step in preparing the sampling frame for sample selection was to stratify beneficiaries into three groups. We used the following as a useful stratification scheme:
1. On SSDI < 24 months, not receiving SSI
2. On SSDI ≥ 24 months, not receiving SSI
3. Receiving SSI.
Note that the stratification is time dependent and that the data for stratification will be several months old at the time of the baseline interview. Unless taken into account, this would have the effect of moving beneficiaries from stratum 1 to stratum 2. One approach would have been to try to anticipate the time lag when defining strata. However, as discussed below, the sample allocation to these strata was proportionate to the number of beneficiaries in the strata in the overall catchment areas. Since we allocated the sample proportionately, the time lag would actually have no impact on the distribution of the sample. Thus, we used the strata as defined above.
Sample Selection
Westat selected an all eligible beneficiaries from the catchment areas of the 22 confirmed sites. Within each site, the beneficiaries divided into a series of smaller batches that we released sequentially for study recruitment as the enrollment process progressed. All eligible beneficiaries were loaded into the Study Management System (SMS); each site was able to access only cases from their specific catchment area.
Recruitment and Randomization
Participants were randomized to one of the two arms of the study; either the Treatment Intervention (supported employment + systematic medication management + mental/behavioral health services) or the Control (treatment as usual). Subjects were randomly assigned to the two arms in equal proportions (i.e., 1:1) using stratified permuted block randomization, with blocks of variable size (e.g., 4, 6 or 8). The use of variable block sizes gives extra protection against site personnel “decoding” the randomization scheme while ensuring a balance between arms after every 4, 6 or 8 participants within each stratum. For example, with two treatment arms, A and B, a permuted block randomization, with block size of 6, could yield the following allocations: ABABBA; ABBAAB, etc.
We used two variables for the stratified randomization. The first stratum is the type of beneficiary, a variable with 3 levels:
1. On SSDI < 24 months, not receiving SSI
2. On SSDI ≥ 24 months, not receiving SSI
3. Receiving SSI.
The second stratum is the site with 22 levels. Stratified randomization ensures that there is balancing of arms within each study site and within each beneficiary type.
Degree of Accuracy
With sample sizes of 1,120 per group (the final MHTS enrollment), we can detect an effect size (ES) as small as a 0.12 standardized difference (one tenth of a standard deviation) with 80 percent power using a two-sample t-test (2-sided) at the 5 percent significance level. An effect size of 0.15 SD would yield 94 percent power under the same assumptions. Similarly, subgroups of sizes 200 and 500 per arm effect sizes of 0.28 SD and 0.18 SD respectively. By the labels of Cohen (1988), we consider effect sizes up to 0.20 SD “small” and up to 0.50 SD “medium” sized.
Reference:
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (Second Edition). Lawrence Erlbaum.
Drawing on successful strategies used in prior studies, we used the following procedures to keep participants in the study: 1) the use of a research information group meetings to assure commitment and understanding of the project prior to randomization; 2) assertive outreach; and 3) collection of tracking information at each interview; and 4) payment for interviews (for control group only).
Another key element in study retention was the selection and training of research assistants to ensure appropriate interpersonal skills to establish a research relationship. We selected research assistants for their capacity to engage individuals with severe mental illness. Systematic training in interviewing skills was be provided by the Westat and Dartmouth-New Hampshire Psychiatric Research Center.
The presence of a Nurse Care Coordinator further enhanced program participation. Because of the shared decision-making framework of the study, consumer choice of treatment was enhanced and likely another motivator.
For the control condition, we also used the technique of obtaining names for multiple contacts of significant others, which would make it more possible to locate those who move between baseline and follow-up. This strategy has proven to be especially useful in following residentially unstable individuals. Another strategy used was quarterly telephone calls by the research assistant to study participants in both conditions to “check-in” about their employment and treatment status.
Westat conducted a pretest of the CAPI data collection instruments (baseline, quarterly, and follow-up) to confirm the estimated length of time to complete the questionnaires. We also used the pretest to ensure that the flow of items is appropriate and feasible.
Statistical Consultation:
Dr. James Bethel
Senior Statistician
Westat
Phone: (301) 294-2067
Site Visit Data Collection and Analysis:
Dr. William Frey
Principal Investigator
Westat
Phone: 301-610-5198
Exhibit-6
Construct of Quality Assurance Activities
Quality Assurance
All aspects of the MHTS intervention rely on established supported employment principles and evidence-based practices to assure consistent quality. The first step in the intervention process was to establish an accurate diagnosis, followed by development of an individualized treatment plan. This plan is the blueprint for services and is a key element for guiding Continuous Quality Improvement and Quality Assurance (CQI/QA) for the project. Each treatment group participant has an individualized treatment plan as a requisite element of participation in the MHTS. Each treatment plan is comprised of evidence-based supported employment and mental health services and algorithm-based medication management.
| File Type | application/msword |
| File Title | SOCIAL SECURITY ADMINISTRATION |
| Author | 177717 |
| Last Modified By | 177717 |
| File Modified | 2009-05-26 |
| File Created | 2009-05-26 |