19060 - ID Living Healthier Living Longer Part B Statistical Methods (1)

Office on Women’s Health Information Collection Request Living Healthier, Living Longer

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

Information collection is targeted at lesbian and bisexual women over the age of 60 who are patrons of the Services & Advocacy for Gay, Lesbian, Bisexual, & Transgender Elders (SAGE) community center in New York City. Currently, SAGE services 400 older LB women in the New York City area. All individuals enrolled in our healthy weight intervention hosted at the SAGE community center will be asked to complete surveys. Because participation in the intervention is contingent upon initial survey completion, we expect that all respondents will respond to the baseline survey, the health history questionnaire, and attend the health screening.

For the healthy weight intervention, statistical methods have been employed to ensure that we recruit an adequate number of individuals. Specifically, assuming 80% probability of rejecting the null hypothesis with 95% confidence:

• To decrease waist circumference by 8 cm (5-10%) (with an expected sample deviation of ± 10cm), a sample size of 40 is required

• To increase average daily steps by 2,000 (with an expected sample deviation of ± 2,350), a sample size of 36 is needed

• To decrease sugar sweetened beverage (SSB) consumption from an expected baseline of 25% of individuals consuming at least one SSB per day to no more than 5% of individuals consuming at least one SSB per day, a sample size of 27 is needed

• To increase proportion of individuals meeting dietary guidelines for fruit and vegetable consumption from expected baseline of 20% to at least 50%, a sample size of 17 is needed

We will continue to recruit until we reach 40 study participants. This number will ensure that all study goals are measureable within the desired statistical power level of 0.8 (80% probability) and confidence of 95% (.05 level of significance).

Analysis Strategy

The following statistical methods are traditionally used in comparing groups with pretest and post-test data:

1. Analysis of variance (ANOVA) on the gain scores,

2. Analysis of covariance (ANCOVA),

3. ANOVA on residual scores, and

4. Repeated measures ANOVA.

For all of these methods, the use of pretest scores helps to reduce error variance, thus producing more powerful tests than designs with no pretest data. In general, the power of the test represents the probability of detecting differences between the groups being compared when such differences exist. We will conduct all necessary tests of normality before developing multivariate models.

We will use SAS, version 9.2, 32-bit, to analyze the data. 

For qualitative data, we will examine raw data to produce descriptions, typologies, themes and categories. Typical steps in this process include data preparation, defining the unit of analysis (e.g., behavior change), developing a coding scheme (e.g., words associated with behavior change), coding all text, assess for coding consistency, and drawing conclusions.  We will use Atlas ti Version 7 to analyze the results.

2. Procedures for the Collection of Information

The questions included in this study will measure the relative effectiveness of a healthy weight program:

• Differences between baseline and end-of-study health metrics

• Differences between baseline and end-of-study physical activity and nutrition

• Respondent impression of whether/how the intervention was effective

Questions will be asked of the entire population of individuals enrolled in the healthy weight demonstration project. The demonstration will be accessible to current members of the SAGE community center and broader LB community in New York City. The healthy weight intervention will be advertised at the SAGE community center and will be promoted by the SAGE resident licensed social worker, with whom many of the SAGE patrons are well-acquainted. The older, LB women population who patronizes SAGE will not be further stratified for inclusion in the intervention; all will be able to participate if they so choose.

Respondent questions will be administered through three different venues:

1. Computer-assisted self-interviewing (surveys)

2. In-person completion and measurements (health metrics and health history questionnaire)

3. In-person group discussion (focus groups)

All data collection and analysis will be performed in compliance with OMB and Privacy Act requirements.

Surveys

Surveys will be administered using computer-assisted self-interviewing through an online survey tool. A trained researcher will compile and analyze survey responses. Respondents will be given a two-week window during which they can complete each of the surveys, with a reminder at one week. Paper surveys will be available to those respondents who prefer this method for survey completion, and the responses to paper surveys will be entered into an electronic database by a trained researcher. Studies have shown that mixed-methods survey administration (i.e., allowing completion by more than one form of media) results in higher response rates that single-method administration, as well as reduced response bias.¹

Health Metrics and Health History Questionnaire

Health metric measurements will be obtained by trained professionals (e.g., registered nurse; physical trainer) at health screenings. The health history questionnaire will be provided to respondents in advance of the initial health screening so that respondents are able to complete the questionnaire prior to the screening. Respondents will have the option of completing the questionnaire using PDF form completion and returning it electronically, or completing a paper copy of the form and bringing it to the initial health screening. If respondents need assistance completing the health history questionnaire, trained survey and health experts will be available at the initial health screening to assist. Respondents will be informed in advance of health screening dates.

Focus Groups

Focus groups will be conducted by a licensed social worker experienced in focus group moderation. Because these focus groups will take the place of another, regularly-scheduled meeting, respondents will know well in advance of the specific dates on which the focus groups will occur.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Participation in the healthy weight demonstration is contingent upon initial survey completion. All study respondents will have responded to the baseline survey and the health history questionnaire, and will have attended the health screening.

For subsequent surveys and health screenings, completion/attendance will be highly encouraged of respondents, and we will use procedures that have been proven effective in other studies as a means to maximize study retention and survey response rates:

Study retention:

• The primary individual who will lead the intervention will also lead recruitment (i.e., study promotion), which will establish relationships with the respondents

• We will use “like me” approaches to ensure that respondents feel comfortable with the various individuals with whom they will interact during the intervention

• We will create a social environment with the intervention to increase respondent desire to attend the various group sessions

• We will create a voluntary online group in which respondents can engage one another, share their progress, etc.

• Respondents will receive pedometers and therabands as a part of their participation in the intervention

• We will conduct telephone follow-ups with those individuals who have not recently attended intervention group sessions

• We will provide a level of attention in health screenings that respondents may not be accustomed to receiving through their usual sources of care

• We will identify in advance potential factors that might contribute to study attrition (through surveys and a preliminary focus group with fewer than 10 women)

Survey completion:

• We will provide monetary incentives for survey completion ($10 at two points during the study)

• We will conduct up to three telephone follow-ups with those individuals who have not completed surveys

• We will inform respondents of the importance of understanding the effects of the study, as measured through surveys, health screenings and focus groups

• Experienced and highly-trained staff will conduct all relevant data collection (focus groups, health measurements)

• We will employ multi-method collection approaches where possible (e.g., online and paper survey versions)

• Surveys can be completed at various points in time, to account for unintended interruptions that might occur

• The same individuals who lead the demonstration project will participate in recruitment, to establish relationships with the respondents

4. Tests of Procedures or Methods to be Undertaken

Approximately 74% of the survey questions included in our surveys and questionnaires (all but 19) were taken from previously validated survey instruments. More than 60% of our survey questions were taken from survey instruments or publications funded by the Federal government. The 19 questions which were not pulled from previously validated survey instruments include the following:

• 13 questions commonly used on medical forms (4 of which were based on recommendations of the National Heart, Lung, and Blood Institute, as discussed in the table below)

• 4 field tested questions

• 2 questions developed using statutory language from the Americans with Disabilities Act (As noted in the US Department of Health and Human Services’ Assessing the Need for a National Disability Survey: Final Report (2011), there is a lack of valid standardized questions to use for assessment of disability status. Thus, we developed two questions to capture disability status.)

The following table details the number of questions in our study that were pulled from each of the various survey instruments and publications (see Attachment H for the source of each specific question). In a limited number of instances, response options were modified to more accurately reflect the current study population (described in Attachment H).

Survey Instrument	Developing Entity	Validated Instrument	Specific2 Questions	Description of Instrument and Our Use
Behavioral Risk Factor Surveillance System (BRFSS)3	The Centers for Disease Control and Prevention		B2-3; B26-31; C19-24; D1; D3-4	The BRFSS is a system of surveys that collect information on health risk behaviors, preventive health practices, and health care access. The BRFSS was established in 1984 and is administered to more than 350,000 adults each year. We will use questions from the BRFSS to capture demographic and health behavior information, including the change in fruit and vegetable consumption between study baseline and completion.
Veterans RAND 12 Item Health Survey (VR-12)4	Veterans Health Administration and RAND Corporation		B18-25 C3-10	The VR-12 is a reliable and valid measure of physical and psychological health status. It is a short form survey derived from the longer VR-36, intended to reduce burden while maintaining reliable results. The VR surveys (12 and 36) have been administered more than 2.5 million times, including annually to Medicare beneficiaries in CMS’ Health Outcomes Survey. We will use the VR-12 to quantify changes in general health between study baseline and completion, as well as to understand effects of general health on respondent experience.
International Physical Activity Questionnaire (IPAQ)5	International Consensus Group of Physical Activity Experts		C11-17 D13-19	The IPAQ is a physical activity screener used around the world. It is a short form survey intended to reduce respondent burden and has been used in well over a hundred studies. The IPAQ was developed in1998 and underwent extensive reliability and validity testing across 12 countries in 2000. We will use the IPAQ to quantify changes in physical activity between study baseline and completion.
Lubben Social Network Scale (LSNS6)6	Dr. James Lubben		B12-17	The LSNS6 is a short survey instrument used to evaluate social isolation in older adults, which is known as a health risk for various conditions and is correlated with overweight and obesity. The LSNS5 has been used across many population groups, including in Veteran Health Administration and Medicare studies. We will use the LSNS6 to understand the effects of social isolation on respondent experience in our study (e.g., effects on drop-out; weight loss)
Diet History Questionnaire (DHQ)7	National Cancer Institute within the National Institutes of Health		B34; B36; B38; B40-42; C27; C29; C31; C33-35	The DHQ was developed at the Risk Factor Monitoring and Methods Branch of NCI. It includes a series of questions on food items and dietary supplementation, including frequency and quantity. We will use six of the DHQ questions to quantify beverage consumption changes between study baseline and completion.
California Health Interview Survey (CHIS)8	UCLA Center for Health Policy Research		B33; B35; B37; B39 C26; C28; C30; C32	The CHIS is the nation’s largest state health survey, surveying more than 50,000 individuals per survey administration. It was initiated in 2001, and has been funded by The National Institutes of Health, The Centers for Disease Control and Prevention, and the Agency for Healthcare Research and Quality. We will use 4 questions from the CHIS to measure the change in frequency of consumption of certain beverage types, from study baseline to completion.
3 Steps to Initiate Discussion about Weight Management9	The National Heart, Lung, and Blood Institute within the National Institutes of Health		D11-12; D21-22	NHLBI provides tools and information for healthcare providers to help individuals achieve long-term weight loss. One tool NHLBI has developed is a document recommending how to initiate conversations about weight loss, and includes assessing patient motivation to lose weight. NHLBI recommends that individuals be asked about willingness to lose weight, previous attempts at weight loss, feelings toward exercise, and potential barriers to success. We will ask questions addressing these topics to gauge respondent willingness to lose weight as well as to better understand their experience in this study.
Task Force on Recommended Alcohol Questions10	National Institute on Alcohol Abuse and Alcoholism within the National Institutes of Health		B43-45; C36-38	In 2003, an NIAAA task force developed alcohol consumption questions recommended for use in instances where researchers intend to ask a limited number of alcohol-related questions. We will use the 3-question series NIAAA developed to estimate the change in alcohol consumption between study baseline and completion (e.g., caloric changes), as well as to identify those individuals who may tend to binge drink. While moderate alcohol consumption has been shown to have a protective effect on weight gain (over no consumption), binge drinking has been shown to cause higher rates of weight gain.
Current Population Survey11	U.S. Census Bureau and Bureau of Labor Statistics		B3-4	The CPS is a joint effort between the US Census Bureau and The Bureau of Labor Statistics, and is administered monthly to households across the US. The survey includes questions on labor force, (un)employment, and other labor demographic characteristics. Two of our survey questions are based on the CPS, with modifications made to the response options to reduce respondent burden. We will use this demographic information to better understand for whom the intervention does (and does not) work. Additionally, we can use the information to tailor the intervention.
Americans with Disabilities Act (ADA)12	US Congress		D7-8	The legislative language of the Americans with Disabilities Act was used to develop a two-step question to identify respondents with disabilities. This information is important as it may directly affect an individual’s experience in our study (e.g., an individual with difficulty walking may not be receptive to a pedometer-based intervention; thus, we would try to accommodate this individual by identifying other sources of physical activity).
National Health Interview Survey13	National Center for Health Statistics within The Centers for Disease Control and Prevention (NCHS-CDC)		B6-7	The NHIS has been administered since 1957, and includes questions on a broad range of health topics. The survey helps DHHS monitor progress toward achieving national health objectives. Recently, NCHS developed and tested a series of questions about sexual identity and orientation. We will use two of these questions to understand the sexual identity of our study respondents, due to differences in health behaviors and weight status between heterosexual, bisexual, and lesbian women (discussed earlier).
National Health and Nutrition Examination Survey (NHANES)14	NCHS-CDC		D2	The NHANES studies began in the early 1960s are used to assess health and nutrition. Each year, the survey is administered to approximately 5,000 individuals. We will use one question from this survey to assess the menopause status of our respondents, as post-menopausal status is correlated with decreased metabolic rate and an increase in appetite-related hormones.15
National College Health Assessment II (ACHA-NCHA II)16	American College Health Association		B32; C25	The ACHA-NCHA is a nationally used survey that collects data on health habits, behaviors, and perceptions. It has been administered to approximately a million individuals since 2000. We will use a single question from this survey to measure the change in average daily fruit and vegetable consumption between study baseline and completion.
Two-Question Method of Assessing Gender Identity (2QAGI)17	Dr. Charlotte Tate		B5	The 2QAGI was developed to expand on typical male/female response options in social and medical studies, to accurately reflect gender and gender identity of study respondents. The 2QAGI approach differentiates between birth-assigned sex category (male, female, intersex) and current sense of self. We will use birth-assigned sex category in our study due to differences in metabolism, hormones, and other contributors to weight loss between birth-assigned females and males.
Various Medical Forms	Multiple		D(intro); D5-6; D9=10	Nine questions across our surveys were pulled from various medical forms and personal training screeners, and the American Heart Association Physical Activity Readiness Questionnaire. Variations of these questions are asked in National and Federal studies, for example NHANES, BRFSS, and NHIS.

In addition to questions allocated in the table above (pulled from existing surveys and government / task force recommendations), four questions have been either field tested in other research studies, pilots, or dissertations, and were not pulled from other forms or surveys (Questions B8-11):

1. Which of the following best describes your present relationship?

   1. In a committed relationship with a women (for example, cohabiting, domestic partnership, legally married)

   2. In a committed relationship with a man (for example, cohabiting, domestic partnership, legally married)

   3. Single, but somewhat involved with a woman, man, or both

   4. Single, and not involved with anyone

2. (If in a committed relationship) Do you currently live with your partner:

   1. All or most of the time

   2. Some of the time

   3. None of the time

   4. I do not have a partner (paper administration)

3. How “out” are you about your sexuality with your health care providers (doctors, nurses, nutritionists, mental health professionals, personal trainers, etc.)

   1. Out to all

   2. Out to some

   3. Out to a few

   4. Out to none

   5. N/A

4. Circle the number of the diagram that best depicts the approximate outline of your partner (image provided)

The physical measurements that will be done as a part of this study are all valid tests commonly employed by healthcare providers.

Pre-test

All survey questions (regardless of previous validation) were piloted across seven lesbian or bisexual women. Specifically, volunteers were asked to provide the following information:

• Time to complete each survey/questionnaire

• Any confusion with the questions

• Any questions perceived as sensitive or personal

• Preference of the wording and response options for three different questions

In response to the pilot results, 24 questions were modified. If further refinements are made to study questions, we will immediately inform OMB of such changes in a revised ICR.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Dr. Suzanne Haynes

Senior Science Advisor

The Office on Women’s Health

200 Independence Avenue, SW, Room 719E

Washington, DC 20201

Suzanne.Haynes@hhs.gov

Dr. Cindy Gruman

Vice President

The Lewin Group

3130 Fairview Park Drive, Suite 500

Falls Church, VA 22042

Cindy.gruman@lewin.com

Allison Rizer

Consultant

The Lewin Group

3130 Fairview Park Drive, Suite 500

Falls Church, VA 22042

allison.rizer@lewin.com

Felicia Sobel

Women’s Program Coordinator

305 Seventh Ave, 15th Floor

New York, NY 10001

fsobel@sageusa.org

1 Baum HM, Chndonnet A, Fentress J, Rasinwich C. Mixed-mode methods for conducting survey research. Data Recognition Corportation. 2012. Available online at http://www.google.com/url?sa=t&rct=j&q=mixed%20mode%20survey%20administration&source=web&cd=1&ved=0CDQQFjAA&url=http%3A%2F%2Fwww.datarecognitioncorp.com%2Fsurvey-services%2FDocuments%2FMixed-Mode-Methods-for-Conducting-Survey-Research.pdf&ei=-csCUdrWDcm00AGZyYDABw&usg=AFQjCNE2_f79lDgHLR81Thy3pNA-in3cjA

2 “Specific questions” refers to the question number in each attached form. For example, B-22 is question 22 in Attachment B.

3 Centers for Disease Control and Prevention (CDC). Behavioral Risk Factor Surveillance System Survey Questionnaire. Atlanta, Georgia: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 2011. (Applies OMB Directive 15: Race and Ethnic Standards for Federal Statistics and Administrative Reporting)

4 The Veterans Health Administration and RAND Corporation. The Veterans RAND 12 Item Health Survey (VR-12)

5 International Physical Activity Questionnaire, Self-administered Short Version.

6 Lubben, J., Gironda, M. (2004). Measuring social networks and assessing their benefits. In Social Networks and Social Exclusion: Sociological and Policy Perspectives. Eds. Phillipson, C., Allan, G., Morgan, D. Ashgate.

7 Diet History Questionnaire, Version 2.0. National Institutes of Health, Applied Research Program, National Cancer Institute. 2010.

8 California Health Interview Survey, Adult Questionnaire. CHIS 2009. UCLA Center for Health Policy Research

9 The National Institutes of Health (NIH). 3 steps to initiate discussion about weight management with your patients. NIH Publication No. 02-5211 November 2002

10 NIAAA The Task Force on Recommended Alcohol Questions

11 The US Bureau of Labor Statistics and the US Census Bureau. Current Population Survey, 2012. Available online at http://www.census.gov/cps/methodology/questions.html.

12 ADA definition for disabled 29 C.F.R. 1630.2(i)

13 Centers for Disease Control and Prevention (CDC). National Health Interview Survey Atlanta, Georgia: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 2013.

14 Centers for Disease Control and Prevention (CDC). National Center for Health Statistics (NCHS). National Health and Nutrition Examination Survey Questionnaire. Hyattsville, MD: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention

15 Barone Gibbs B, Kinzel LS, Pettee Gabriel K, et al. Short- and long-term eating habit modification predict weight change in overweight, post-menopausal women: results from the WOMAN Study. J Am Diet Assoc. In press.

16 America College Health Association. -National College Health Assessment II (ACHA-NCHA II) Survey. Spring Version, 2008.

17 Tate, C. C., Ledbetter, J. N., & Youssef, C. P. (2012). A two-question method for assessing gender categories in the social and medical sciences. Journal of Sex Research.