20883 ID_0990-sash_ssb_110513

Supporting Statement B for

Support and Services at Home Evaluation:

Beneficiary Survey

RTI International

November 2013

B. Statistical Methods (used for collection of information employing statistical methods)

B1. Respondent Universe and Sampling Methods

The target population for the survey is Medicare beneficiaries participating in the Support and Services at Home (SASH) demonstration. SASH provides integrated, home-based services to beneficiaries in selected housing properties throughout Vermont. At this point, 1,685 intervention beneficiaries have been identified in 37 SASH properties. A total of 288 intervention beneficiaries came from the neighboring community rather than a housing property.

Identification of Comparison Beneficiaries

The effect of the SASH program will be estimated by contrasting the results for SASH participants with the results for a comparison group. Two types of effects are of interest – the effects of SASH only and the combined effects of SASH and the Multi-Payer Advanced Primary Care Practice (MAPCP) demonstration. Separate comparison groups will be created to estimate each of these effects. The first comparison group will be drawn from non-SASH housing properties in Vermont. The second group will be drawn from a similar geographic area in New York State that does not have a MAPCP program. Both groups will be selected using a two-stage procedure in which housing properties are identified in the first stage, and beneficiaries within those properties are identified in the second stage.

Vermont Comparison Group

The Vermont comparison group will be drawn from 56 non-SASH housing properties. Like the SASH sites, these properties are distributed across the state and exhibit a similar range of housing units per property. We will match property addresses to the MAPCP rosters to identify all MAPCP beneficiaries in these properties.

New York Comparison Group

The New York comparison group will be drawn from housing properties in a 16-county area in New York State. Located in the state’s northeast quadrant, these counties form the closest non-MAPCP area to Vermont. We have historical claims data for more than 75,000 Medicare beneficiaries in this region, although only a fraction of them will reside in multiunit housing. We will use the U.S Department of Housing and Urban Development’s Multifamily Reporting Database (MPRD) to identify active properties in this region. The MPRD contains a wide variety of characteristics on individual properties including number of units, occupancy rates, and ownership status. We will exclude properties designated for assisted living or nursing care.

B2. Procedures for the Collection of Information

Survey Materials. The mail survey is designed to collect outcomes that cannot be measured from claims data or other sources. We will use brief, standardized scales with demonstrated reliability and validity in older adults. The key survey domains, proposed instruments, and the location of each scale in the survey instrument are shown in Exhibit 2-1.

Exhibit 2-1. Beneficiary Survey Domains and Instruments

Domain/Instrument	Brief Description of Measures	Survey Location
Physical and Mental Health: RAND-12	Measures overall functioning and quality of life in adults.	Items 2, 3, 8
Functional Status: Activities of Daily Living Questionnaire	Measure of the ability to perform basic activities of daily living. Collects information on six standard activities—bathing, dressing, eating, getting in and out of chairs, walking, and using the toilet.	Item 5
Health Preference Utility: EQ-5D	Measures of quality-adjusted life years to perform cost-utility analyses. Consists of items about mobility, self-care, usual activities, pain, and anxiety/depression.	Item 4
Medication Problems: The Brief Medication Questionnaire	Measures how much difficulty respondents have with 5 common medication-management tasks.	Item 6
Dietary Problems: Dietary subscale of the Mini Nutritional Assessment	Measures overall nutrition status .based on dietary consumption	Item 7
Background Characteristics	Demographic characteristics such as race, ethnicity, educational attainment, and health insurance coverage.	Items 1, 9-15

We will distribute the English language version of the survey, although we will make Spanish language versions of the survey available upon request. The survey will be in scannable form, allowing for easy data capture of returned surveys. We will develop separate cover letters for each mailing. The first letter will contain all required elements of informed consent and a toll-free telephone number that subjects can call if they have questions. The cover letters will be printed on letterhead and signed by an appropriate official to enhance the survey’s legitimacy.

Survey Procedures. We will survey beneficiaries by mail, with reminder telephone calls to nonrespondents. We will use a multiple-mode, multiple-contact approach that has proven very successful on surveys conducted with the Medicare population and incorporates suggestions from Jenkins and Dillman’s (1997) best mail survey practices guidelines. Participation in the survey will be voluntary, and no incentives or remuneration will be given to sample members.

Survey Schedule. The survey data collection process will consist of an initial mailing to beneficiaries, followed two weeks later by a second mailing to all nonrespondents. The data collection period will end approximately 4 weeks after the second mailing.

Power Analysis. Our goal in this study is to be able to detect SASH program effect sizes of 0.35 or greater (0.35 standard deviations of the outcome measure) when contrasting SASH participants and comparisons. Applying statistical power analysis, a sample of 129 completed surveys will be required in each group to detect effects of this magnitude (power = 0.80, alpha = 0.05, two-sided test). A sample of the same size of SASH nonparticipants and comparison beneficiaries will also be collected. On the basis of our recent experience with Medicare beneficiaries, we anticipate a survey response rate of 58%. From the sample frame for each group, we will randomly select 129/0.58 = 223 beneficiaries per group. The size of the total sample to be contacted is therefore 669 beneficiaries, split evenly across the three types of respondents.

Quality Control. RTI will implement quality control procedures throughout the mail survey period. Our data preparation staff will match personalized cover letters with surveys using a unique identification number and will check a portion of all outgoing mail packages to make sure that the packages contain all required materials (i.e., cover letter, survey, business reply envelope). Responses for each sampled beneficiary will be tracked and assigned a unique disposition code in order to compute survey response rates. Our data receipt staff conducts a manual review of each returned survey to locate any written comments or enclosed materials, which will be referred to project staff for review. It is through this manual review that we may learn of sample members who are deceased, physically incapable, or ineligible. At different points during the data collection period, project staff will also pull a sample of hard-copy surveys and compare the responses to the scanned data to ensure that data are being accurately captured.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Mail Survey Practices. A number of “best mail survey practices” have consistently been shown to be associated with survey response rates (Herberlein and Baumgartner, 1978). These practices are:

• pre-notification letters

• number of follow-up mailings

• survey sponsorship

• saliency of the survey topics to the target population

• personalization of correspondence

• postage-paid return envelopes.

We have incorporated all of these elements in our protocol except for pre-notification letters. Response may also be enhanced by permitting proxies to assist beneficiaries to complete surveys.

Address Changes. Medicare beneficiary addresses will be supplied by CMS. The National Change of Address (NCOA) file will be used to ensure that we have accurate address information prior to the initial mailing.

Nonresponse Weighting. We will analyze the probability that each eligible, sampled beneficiary completed the survey. This response propensity model will be a logistic regression model in which the outcome is coded 1 if the sampled beneficiary completed survey and 0 if the survey was not returned or completed. The explanatory variables will consist of factors that are available for all housing property beneficiaries, including housing unit characteristics uniformly collected by HUD and demographic characteristics. The inverse of the predicted response probabilities will be used as survey weights.

B4. Test of Procedures or Methods to be Undertaken

Nearly all of the surveys items are drawn from widely used instruments with established reliability and validity that have previously been administered to older adults. As a result, we do not plan to engage in any further pre-testing for the purposes of this evaluation.

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Kevin W. Smith, M.A. (Survey Methodologist)

RTI International

Telephone: 781-434-1748

Ellen Stutts, M.A. (Research Survey Manager)

RTI International

Telephone: 919-541-6037

REFERENCES

Herberlein TA, Baumgartner R. Factors affecting response rates to mailed surveys: A quantitative analysis of the published literature. American Sociological Review 1978; 43:447-462.

5