Supporting Statement B
For
Assessing an Online Process to
Study
the Prevalence of Drugged Driving in the U.S.:
Development
of the Drugged Driving Reporting System
NATIONAL INSTITUTE ON DRUG ABUSE (NIDA)
National
Institutes of Health
Department of Health and Human Services
March
25, 2015
Harold Perl, Ph.D.
Chief, Prevention Research Branch
Division
of Epidemiology, Services & Prevention Research
National
Institute on Drug Abuse (NIDA)
National Institutes of Health (NIH)
6001
Executive Blvd., MSC 9589
Rockville, MD 20852
Phone:
(301) 443-6504
Fax: (301) 443-2636
Email: hperl@nida.nih.gov
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 3
B.1 RESPONDENT UNIVERSE AND SAMPLING METHODS 3
B.2 PROCEDURES FOR THE COLLECTION OF INFORMATION 4
B.3. METHODS TO MAXIMIZE RESPONSE RATES AND DEAL WITH NON-RESPONSE 5
B.4 TEST OF PROCEDURES OR METHODS TO BE UNDERTAKEN 5
The universe for the Drugged Driving Survey (DDS) survey will include all adult individuals possessing a valid driver’s license and/or motor vehicle registration issued by their state of residence within each of the three participating states. Within each state, survey respondents will be a convenience sample of the first 1,250 individuals making an in-person visit to the department or registry of motor vehicles (DMV) who receive a receipt for services and respond to recruitment materials received during the visit by going on-line and completing the survey. No formal sampling methodology, stratification, or over-sampling of responding units will be used; only the natural stratification by geographic location (state residency) will occur, with an approximately equal number of respondents obtained from each state. The AHP/Carnevale Team expects that each of the three state’s DMV locations will provide recruitment materials to individuals in sufficient numbers to generate approximately eleven (11) completed surveys per day over the four-month data collection period. During the survey data collection period the team will calculate and monitor survey response rate based on the number of completed surveys as a proportion of the number of total recruitment materials distributed to DMVs. In addition, with the assistance of the American Association of Motor Vehicle Administrators (AAMVA) organization, the team will obtain data from each participating state regarding the actual volume of total in-person visitors to each state’s respective DMV locations during the four-month period to calculate the proportion of all visitors who would have been provided with recruitment materials.
The team conducted formal power analyses to determine that within-state sample sizes of 950 respondents would provide sufficient power (80%) to detect the range of effect sizes reported on responses to surveys similar to the DDS at a 5% significance level (NSDUH and NRS). The sample redesign to collect data from 1,250 respondents per state (3,750 total) would maintain sufficient sample size for planned analyses. Planned analyses include between- and within-group comparisons on characteristics and responses, such as between individual states, and between incentive levels. Thus, the team anticipates needing approximately 1,000 completed surveys from each of the three states (the project is targeting 1,250 per state), and a total sample size of 3,750. It is not possible at this time to determine how many DMV customers will need to be made aware of the survey in order to obtain 3,750 completed surveys. Methods will be used to maximize the response rates, as described in B.3 (below).
The team anticipates that the individuals choosing to participate in the survey will be generally representative of the universe of adult individuals possessing a valid driver’s license and/or vehicle registration within each participating state. It is possible that the sample of individuals who agree to participate in the survey may differ in some systematic ways from the universe. While the team does not expect this, it is possible that individuals obtaining DMV services in-person, rather than via the web, may differ in some characteristics from the general population. For example, they may potentially be less likely to conduct business over the internet, and therefore less likely to own a computer, tablet, or other smart device, and may be older than the individuals in the driver universe. The team will compare the characteristics of survey respondents (age, education, gender, type of license) with those of the driving age population within each state (from statistics from the National Highway Traffic Safety Administration and the American Association of Motor Vehicle Administrators) to determine the representativeness of the survey samples and note any systematic differences that may emerge.
No personally-identifying data will be collected from survey participants; their participation will be entirely anonymous. Data for each state will be collected within the same four-month time period. Individuals who are at least 18 years of age or older obtaining services at their DMV office will each receive recruitment material (a sticker attached to their DMV receipt of services or a postcard or flyer distributed to individuals, providing the web link to the survey site. Some stickers or cards will include information about incentives for completing a survey: In each state, initially, within the initial set of recruitment materials provided to DMVs, 75% of the total number distributed will offer an incentive (50% of the materials will have a $10 gift card incentive, while 25% will have a $20 gift card incentive.) The remaining 25% of the materials will not offer any incentive. Each sticker or card will have a two or three character alphanumeric recruitment code that respondents who participate in the survey will enter in a survey field when they complete the survey. The code will only identify (1) the state in which the respondent visited the DMV, and (2) the level of incentive (none, $10, or $20). The recruitment code will not identify the respondent individually. Upon completing the survey the survey software will generate a redemption code for the respondent, in the appropriate gift card amount (if any). The respondent will then use this redemption code at the National Gift Card Rewards (NGC) website to select their choice of gift (as described in Section B.3., below).
At regular intervals during the survey data collection process, the AHP/Carnevale team will examine response rates associated with each of the three types of recruitment materials (no incentive, $10 incentive, $20 incentive) to estimate whether the project is on target to meet the response goal of 3,750 total respondents. If it is not, the team will adjust the incentives within the subsequent set of recruitment materials next provided to each state’s DMV. If a higher response rate is needed, recruitment materials will be distributed based on the number of responses already received for that state. For example, if not enough responses have been received, the team will provide more materials advertising $10 gift cards.
One quarter (25%) of the aforementioned stickers (or cards), distributed to individuals receiving a receipt for services at their DMV office will merely provide the information for the survey web-link, as noted in B.2. The remaining stickers distributed will additionally inform participants that they will be eligible to obtain a gift card should they chose to participate and complete the survey. Fifty percent of the individuals will be offered a gift card of $10; the remaining 25% will receive a $20 gift card. Upon completing the survey, survey instructions will inform those participants entitled to receive a gift card how to navigate to the web page of the National Gift Card Rewards (NGC), and a second identifier code that survey participants can enter to claim their gift card from NGC, if applicable. In addition, the team will maximize response rates with a second distribution of materials with increased incentives if necessary to obtain a large enough sample size.
The team expects that the promise of a gift card as a thank-you gift for participating in the survey will increase participation in the survey beyond what it would be without the incentive. The team will measure the effect of each level of incentive on survey participation in comparison to participation where no gift card is offered. The team will not be able to identify or track individuals provided with the stickers inviting them to participate, and therefore will not be able to identify non-respondents in order to encourage their participation.
Preventing repeat respondents - While it is not possible to know exactly how many people might visit the DMV more than once in a 3-6 month period, the percentage is quite low according to the American Association of Motor Vehicle Administrators, a sub-contractor on this project. The survey asks if the respondent has taken the survey already, which may deter some repeat respondents. While the IP addresses of respondents will not be stored on a long-term basis, the web site hosting the survey can be adjusted to prevent the same IP address from being used in a certain time period (e.g. 8-24 hours).
Pilot testing/cognitive interviews with fewer than 10 individuals
Advocates for Human Potential (AHP) and Carnevale Associates conducted cognitive interviewing on the survey in April 2014. This process is not part of this application under the Paperwork Reduction Act (PRA); it is described here to provide reviewers with an understanding of the process that took place prior to OMB submission. AHP’s Institutional Review Board approved this cognitive interview process. The 8 respondents included 5 men and 3 women, and ranged in age from 21 to 65. Respondents completed the survey online while in a conference room at AHP’s Germantown office, and then immediately participated in cognitive interviews lasting between 1 to 1.5 hours. The cognitive interview process provided insights into how respondents understood the survey questions that were used to revise the survey.
The goal of the cognitive interview was to better understand how respondents interpret the survey questions, to ensure that the meaning of each question is interpreted as intended, and to understand the mental process by which respondents go about constructing their answers to each question to ensure that they understand the response options and are able to answer the questions accurately. This process allowed AHP to make improvements to the survey.
AHP has two individuals in its Research and Evaluation Division who will be consulted on statistical aspects of data analysis. In addition, Carnevale Associates will also have two staff members reviewing the data analysis process. Those individuals are listed below:
NAME/TITLE |
ADDRESS |
CONTACT INFORMATION |
Kristin Stainbrook, Ph.D. Deputy Director of Research |
Advocates for Human Potential, Inc. 4 State Street Suite 500 Albany, NY 12207 |
Phone: (508) 729-1241 kstainbrook@ahpnet.com |
David Centerbar, Ph.D. Sr. Research Associate |
Advocates for Human Potential, Inc. 490-B Boston Post Rd Sudbury, MA 01776 |
Phone: (978) 261-1423 dcenterbar@ahpnet.com |
John Carnevale, Ph.D. President & CEO (Contract Role: Senior Subject Matter Expert) |
Carnevale Associates, LLC Gaithersburg, MD 20883 |
Phone: 301.977.3600 John@carnevaleassociates.com |
Erika Ostlie, M.A. Managing Director (Contract Role: Research Manager) |
Carnevale Associates, LLC Gaithersburg, MD 20883 |
Phone: 301.977.3600 Erika@carnevaleassociates.com |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | David Centerbar |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |