Supporting Statement B for
Simulations for Drug Related Science Education (NIDA)
Bethesda, MD 20892
Prepared by:
Robert Levine
Address: 3600 FAU blvd., Boca Raton, FL 33431
Telephone: 305-981-4830
Fax: 305-981-4831
Email: Rlevine@archiemd.com
Submitted By:
Cathrine Sasek, Ph.D., Project Officer
National Institute on Drug Abuse
National Institutes of Health
6001 Executive Boulevard, Room 5230
Rockville, MD 20892
Phone: (301) 443-6071
Fax: (301) 443-6277
Email: csasek@nih.gov
Submission Date 03/17/09
Table of contents
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 3
B.1 Respondent Universe and Sampling Methods 3
B.2 Procedures for the Collection of Information 4
B.3 Methods to Maximize Response Rates and Deal with Non-response 6
B.4 Test of Procedures or Methods to be Undertaken 7
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting
and/or Analyzing Data………………………………………………………….8
The respondent universe for this evaluation is high school students in the South Florida Region. The South Florida Region is defined as Palm Beach County, Broward County and Dade County. There are a total of 102 high schools and a total of 228,642 students enrolled in these schools. Each county has approximately 50% males and 50% females enrolled in their high schools. Palm Beach County is 43% White, 29% African American and 21% Hispanic (23 schools and 51,691 high school students enrolled). Broward County is 30% White, 37% African American and 26% Hispanic (32 schools and 71,374 high school students enrolled). Dade County is 11% White, 27% African American and 60% Hispanic (47 schools and 105,577 high school students enrolled).
A sample size of 360 was chosen to ensure statistical significance for a medium effect size for the attitude measures and knowledge measures in this study. According to Cohen (Cohen, J (1988). Statistical power analysis for the behavioral sciences (2nd edition). Hillsdale, NJ: Erbaum), we will need at least 66 participants to detect a medium behavioral effect size with a power of .80 and two-sided alpha of .05. Sample size has been increased to 120 per major ethnic group (white non-Hispanic, African American and Hispanic) in order to be able to measure differences within and between ethnic groups. Our sample size is more than adequate, even with the expectation of attrition of 15% due to absenteeism and school changes over the course of the study. Our sample size will allow us to measure smaller effects on individual attitude questions within each ethnic group.
Additionally, we are requesting permission to add an ethnic group for data collection purposes. We would like to add “Caribbean Native” to the ethnic groups since it is not already included in the NIH ethnic classifications and a significant portion of the population in South Florida is represented from the Caribbean (i.e. Haiti, Jamaica, Bahamas, etc.) Individuals from these populations do not identify themselves as Hispanic nor African American. Our past research in South Florida schools has identified the need to include this ethnicity on our data collection forms. (Appendix 5 presents the official demographic profile of Miami Dade County, Florida 1960-2000).
A sample size of 360 was chosen to ensure statistical significance for a medium effect size for the attitude measures and knowledge measures in this study. According to Cohen (Cohen, J (1988). Statistical power analysis for the behavioral sciences (2nd edition). Hillsdale, NJ: Erbaum), we will need at least 66 participants to detect a medium behavioral effect size with a power of .80 and two-sided alpha of .05. Sample size has been increased to 120 per major ethnic group (white non-Hispanic, African American and Hispanic) in order to be able to measure differences within and between ethnic groups. Our sample size is more than adequate, even with the expectation of attrition of 15% due to absenteeism and school changes over the course of the study. Our sample size will allow us to measure smaller effects on individual attitude questions within each ethnic group.
This evaluation/study design is a quasi-experimental design. It is a pre-test/post-test with entire classrooms of students to be assigned to either the intervention group (i.e. using the computed based module) or the control group (i.e. reading appropriate text material from a high school science text book). We submitted our protocol to Copernicus Group IRB (One Triangle Drive, Suite 100, P.O. Box 110605, Research Triangle Park, NC 27709) for an expedited review (45 CFR 45.110).
Upon receiving approval from Copernicus Group IRB, we will submit requests to perform the evaluation to West Palm Beach County Board of Education, Broward County Board of Education and Dade County Board of Education. Once approval from school boards has been granted, principals and science teachers for both public and private schools will be contacted for participation in the evaluation. Science teachers volunteering to participate will be able to contact the evaluation team to set up a meeting to discuss the educational module, assessments and time commitment. If the science teacher decides to have his/her class participate in the evaluation, letters will then be sent home with those students to inform their parents/guardians of their classroom participation in this evaluation. Parents/guardians will have the opportunity to contact Archie MD regarding any questions they may have regarding the evaluation or educational module.
Once all recruitment has been completed, a schedule of dates and times to implement pre/post-tests and the intervention will be coordinated with the science teacher to ensure that the project is non-disruptive to the teachers planned educational activities and fits into their lesson plans. Once this has been determined, the evaluation team will go to the classroom to administer pre-tests to the classroom of students. It will be stated by the evaluation team member before passing out the pre-test, post-test or intervention that participation is voluntary and that the students grades, standing within the class or their standing within the school will be impacted on whether they choose to participate in the evaluation. All pre-tests will be administered and collected by the Archie MD team. The pre-test will take approximately 25 minutes to complete by the students. Once collected they will be placed in a sealed manila envelope and be entered into the database by the research team.
One to two weeks later (exact time depending on the teachers schedule and lesson plans), the evaluation team will return to administer the intervention. At this time, preloaded laptops will be given to classrooms of students to participate in the learning module (intervention group) or standard reading materials will be handed out (control group). Both the intervention and control groups will utilize their given materials for the 50 minute classroom time period. At the end of the class, the research team members will collect the computers or reading materials. At this time, students who used the computer based learning module will be given a brief (3-5 minute) anonymous questionnaire regarding their opinion of the computer based learning module (i.e. did you like the graphics? Was the sound clear? Suggestions for improvement?, etc). This will occur within the 50 minute time block designated for the intervention. Evaluation team members will collect this feedback form from students before they leave the room for their next class.
Approximately 3-4 weeks later, the evaluation team will return to the classrooms to administer the post-test. This post-test will take approximately 25 minutes to complete and will be administered and collected by the team member. Once collected they will be placed in a sealed manila envelope and be entered into the database by the research team. After all participating classrooms have completed post-tests, data analysis will begin to determine if there are any differences between the intervention and control groups in the areas of knowledge acquisition and attitudes towards science education and drug use.
A final post-test will be administered 6 months after the initial post-test to collect data. At this time, the evaluation team member will return to classrooms to administer and collect this attitudinal post-test. This post-test will take approximately 25 minutes to complete and will be administered and collected by the team member. Once collected they will be placed in a sealed manila envelope and be entered into the database by the research team. After all final post-tests have been administered, data analysis will begin to analyze if there are any longer term differences between the intervention and control group. Additionally, final results will begin to be written up for the final report to the funding agency (National Institutes of Health’s National Institute on Drug Abuse).
Evaluation team members will be present during the pre-test and post-test. The evaluation team members will be responsible for explaining to the students that their participation is voluntary and explaining the purpose of the evaluation. The science teachers are not responsible for the administration and collection of pre-tests and post-tests. Evaluation team members will monitor the classroom while students complete the pre-tests and post-tests to ensure that students do not exchange pre/post-tests, and to monitor if pre-tests and post-tests are complete. As for any other measures to maximize response rates there are none. This evaluation is voluntary. It is hoped that the outside presence of individuals explaining the project will motivate students to complete the pre/post-tests and intervention. We expect an 85% response rate (and 15% attrition due to absenteeism).
Pre-tests and post-tests have been developed in house and reviewed by an expert panel (expert panel: Dr. Leslie Miller – Expertise: Applying technology to education, Charlie O’Brien – Expertise: Science of Addiction, Michael Lang – Expertise: Science education, Charlie Parsons – Expertise: CEO of D.A.R.E. America, Dr. Eden Evins – Expertise: Clinical psychiatrist)
Knowledge based questions that will be used to measure knowledge acquisition were developed in line with information presented in the multimedia module and text based readings. Attitudinal questions for the section regarding drugs and the perception of harm from drugs have been adapted from the Florida Youth Risk Behavior Survey, and from prior evaluations measuring attitudes regarding alcohol and alcohol/substance abuse (which have been pre-tested with high school populations to ensure clarity and proper language). Science attitudinal questions have been adapted from the Scientific Attitude Inventory and additional questions have been developed by the development team. The pre-tests and post-tests were reviewed by the team, and changes have been made to make certain that knowledge questions use the same language used in the educational module, questions are ordered logically, and to check that the wording of the attitudinal questions is appropriate.
Pre-tests and post-tests take approximately 25 minutes to complete. A qualitative feedback form has also been developed for students to comment on their experience using the computer based learning module. This feedback form takes approximately 3-5 minutes to complete.
Pre-tests and post-tests will be confidential and will be coded so that no names are associated with responses. The questionnaires also ask for participants to include full date of birth (as proof of age), last four digits of their current home phone number. If no phone is available than the option to use mother’s month and day of birth will be presented as an alternative. These numbers will only be used for the purpose of code numbers to match pre- and post- intervention questionnaires and as proof of age. Students’ names will not be matched with any responses. The intervention and questionnaires will be completed during school hours, and the students will not be asked for any time outside of school.
Methods used during data collection for this evaluation were adapted from past school evaluations conducted on alcohol education interventions and safe driving education interventions. The pre/post test method used has been successful in the high school population (for alcohol and safe driving educational interventions, unpublished data). The evaluation team will administer the intervention, pre-tests and post-tests. Additionally, the evaluation team present during this time will be able to monitor the completion of pre-tests and post-tests, and be able to answer any questions or to ensure that forms are not switched.
All sample size and statistical aspects of the design were initiated by:
Lee Crandall, PhD
University of Miami Miller School of Medicine
Deputy Director, Comprehensive Drug Research Center
Professor of Epidemiology and Public Health
305-243-3021
Dr. Crandall is a Professor of Epidemiology and Public Health and Deputy Director, Comprehensive Drug Research Center, University of Miami School of Medicine. He is a medical sociologist, health services researcher, and social epidemiologist. He has over 30 years of faculty experience and numerous publications in the area of health disparities, epidemiology and prevention of substance abuse.
Data collection oversight and analysis will be performed by:
Jill Graygo MA, MPH
Lead Evaluator
Archie MD
305-562-0278
Ms. Graygo is a Research Manager that currently oversees the development, study design, implementation, analysis and dissemination of multiple federally funded research studies. She has extensive knowledge in study design, ethical issues and Institutional Review Board approval process. Additionally, Ms. Graygo has also been involved in assessment tool development and evaluations for multiple projects/programs, and developing strategies for dissemination of results and /or interventions based upon findings.
| File Type | application/msword |
| File Title | Supporting Statement 'B' Preparation - 02/12/2008 |
| Subject | Supporting Statement 'B' Preparation - 02/12/2008 |
| Author | OD/USER |
| Last Modified By | curriem |
| File Modified | 2009-03-17 |
| File Created | 2009-03-17 |