Att_OMB Part A Final

Appendix A Active Consent
Appendix B Student Survey
Appendix C Schoolwide Records Collections Form
Appendix D School-Level Drug Testing Collection Form
Appendix E Staff Interview Topical Question Guide

Exhibits

Exhibit 1 Sampling For Impact Study 7

Exhibit 2 Data Collection Plan 8

Exhibit 3 Respondent Burden Estimates For Data Collection 23

Exhibit 4 Schedule of Key Activities 28

Request for Reinstatement, without change, of a previously approved collection

This document presents the supporting statement for the request for regular approval for the Impact Evaluation of Mandatory-Random Student Drug Testing (MRDT). This collection was granted emergency approval [OMB Control Number 1850-0818] by the Office of Management and Budget (OMB) on February 21, 2007, with expiration on August 31, 2007. The current document contains no changes from the collection approved by OMB.

A. Justification

1. Circumstances Necessitating Data Collection

This study seeks to determine the impact of mandatory-random drug testing (MRDT) on high school students who do and do not participate in activities subject to drug testing. This section summarizes the study approach and is divided into 7 parts:

overview of the intervention,
identification of grantees and school study sites,
random assignment of schools,
student sampling,
data sources,
impact analysis plan, and
estimation of contamination in control schools.

Overview of the Intervention

Although illicit substance use has declined in recent years, it remains fairly widespread among secondary school children (NIDA, 2003). By Grade 12, 50% of students have used some type of illicit drug at least once and 23% report having used illicit drugs within the past 30 days (Johnston, O’Malley, Bachman, & Schulenberg, 2006). High schools have adopted a wide array of strategies either to prevent initiation or curb the level of drug use among their students, including mandatory-random drug testing of students enrolled in school-sponsored, competitive, extracurricular activities. In MRDT, parents must give consent to school authorities for their children to be tested for drug use as a condition of their child’s participation in competitive, extracurricular activities. Several times each year, students enrolled in these activities are randomly selected and tested for substance use. The goals of MRDT are to identify students who are in need of counseling or treatment, to reduce current drug use, and to prevent future use.

However, MRDT has not been rigorously evaluated. With this study, ED seeks to address this gap in evaluation evidence, particularly in light of a recent expansion of Federal funding to support MRDT. In fiscal year 2003, OSDFS received an appropriation of $2 million to award discretionary grants for the implementation of school-based MRDT programs and in fiscal year 2005, Congress increased this appropriation to $10 million.

The current study design is based on experiences with the 2005 round of drug testing grantees. During that year competitive preference points were awarded to applicants who agreed to participate in the national evaluation of the MRDT program. However, an insufficient number of grantees sought this preference and the evaluation could not be conducted. For the 2006 round of grantees, participation in the national evaluation was made an absolute priority. ED anticipated that districts would be reluctant to apply if they were required to delay implementation of drug testing in control schools for more than one year (even though the theory of action suggests that more than one year may be required for the program to have a full effect, especially among non-athletes who are not subject to school district student drug testing policies).The current study design’s one-year experimental period balances ED’s desire for a long experimental period and districts’ desire to implement their student drug testing programs quickly.

The primary research questions the study will address include:

Do high school students who are subject to MRDT report less use of alcohol, tobacco, and illicit substances than comparable students in high schools without MRDT?
To what extent does student participation in the activities that subject them to MRDT change?
Do students in high schools with MRDT policies who are not subject to such testing report less use of alcohol, tobacco, and illicit substances than comparable students in high schools without MRDT?
What are the characteristics of the MRDT programs?
What prevention and intervention policies, programs, or strategies other than MRDT exist in the treatment and control schools?

To address these questions, we will:

randomly assign schools, within districts awarded grants, to be in either a treatment group (that implements MRDT during the evaluation period) or a control group (that delays implementation of MRDT until the end of the 2007–2008 school year);
collect data on students’ self-reported drug use, school-level data on drug-related incidents and drug test results, and characteristics of the drug testing programs and other substance use prevention programs in treatment and control schools; and
analyze the data and report the results of the analysis.

In addition, it is possible that the process for applying for an MRDT grant and approving a district MRDT policy may stimulate change in student behavior in the either treatment or control schools, and some level of awareness could continue even in control schools (i.e., contaminate the control schools). Therefore, comparable student survey data will be collected in a sample of external schools with characteristics similar to the grantee schools to assess the extent to which any contamination of the control group may have occurred.

The Office of National Drug Control Policy (ONDCP) asked ED for the study of contamination to be included as part of the overall evaluation design for two reasons. First, school districts were required in their grant application to demonstrate community support. The process of generating community support on behalf of a district’s underlying grant application, as well as the public notification of the grant award, may cause a change in the behavior of students in grantee schools prior to random assignment. Second, and post random assignment, schools assigned to the control condition will be required to implement student drug testing following the end of the experimental period in spring 2008. If students in the control schools become aware of this prior to the end of experimental period, it may affect their behavior. Data collected from the external school sample will be used to better understand the extent to which contamination has occurred. This allows the study team to assess the extent to which trends in substance use in control schools are similar to (or different than) trends in substance use in comparison schools.

Identification of Grantees and School Study Sites

In fall 2006 OSDFS awarded 11 new discretionary demonstration grants including 52 schools for student drug testing (2 grantees subsequently decided not to implement the program).Each grantee included written commitments from 2 to 12 high schools in their district that did not have an existing student drug testing program and agreed to allow the evaluation contractor to randomly select half of the proposed high schools to implement the program during the evaluation period and the other half to serve as control sites until the end of the first 3 rounds of data collection that comprise the impact evaluation period for this contract (through the end of the 2007–2008 school year). Grantees are responsible for ensuring that control high schools neither promote nor implement any student drug-testing programs until the end of the impact evaluation period and that treatment and control schools cooperate with evaluation data collection activities.

Restricting the study to include schools with these characteristics is necessary to ensure that (i) treatment and control schools are equivalent at the start of the evaluation in terms of their interest in or need for MRDT, (ii) treatment schools have resources to implement MRDT, (iii) control schools are willing to refrain from student drug testing for the length of the impact evaluation, and (iv) neither treatment nor control schools implement voluntary or suspicion-based student drug testing programs, the effects of which could be confounded with the effects of MRDT.

Random Assignment of Schools

We view drug testing as an environmental intervention, which could potentially affect all members of the school environment. Knowledge that drug testing is occurring in the school might affect drug use among all students, even those who are not subject to testing. The school (as opposed to the student) is therefore the appropriate unit of random assignment to test the effectiveness of MRDT. The 45 high schools included in these grants will be randomly assigned as part of the impact study. Within each district, we will form blocks of schools that are similar in terms of key characteristics that are available in secondary data sources, such as the Common Core of Data. We will then randomly assign schools to treatment and control groups within these blocks, thereby reducing the potential for imbalance between groups with respect to these key characteristics.

Student Sampling

Our plan for sampling students is designed to achieve two main objectives. First, we want a sample of students in each school that is representative of the students in the school at the time of each of the follow-up surveys (fall 2007 and spring 2008). Second, we want a sample that includes a sufficient number of likely activity participants and nonparticipants^¹ to support estimation of impacts for each of these two key subgroups.

To achieve these objectives, we plan to draw a random sample of students, stratified by grade, prior to the administration of the baseline survey in the spring semester of 2007. Students from this initial sample who remain in the study schools in fall 2007 and spring 2008 will continue to be included in the study. Students who leave the study schools will be replaced by a sample of students who are new to the school in order to maintain a representative sample of students in the school at that point in time.^² Thus, our sample of students at follow-up will consist of a longitudinal component for which we will have baseline data available and a cross-sectional component for which baseline data for individual students will not be available.

Exhibit 1 illustrates the sampling plan for the impact study. Cells marked “new” indicate that all students in that grade are new to the study sample at that point in time. Cells marked “refresh” indicate that the only new students to the study in that grade at that time are those sampled to replace students who have left the study. Twelfth grade students are not sampled in spring 2007 because they will no longer be in study schools in the follow-up data collection period.

Exhibit 1
Sampling For Impact Study

Spring 2007	Fall 2007	Spring 2008
Grade 9 (new)	Grade 10 (refresh)	Grade 10 (refresh)
Grade 10 (new)	Grade 11 (refresh)	Grade 11 (refresh)
Grade 11 (new)	Grade 12 (refresh)	Grade 12 (refresh)
--	Grade 9 (new)	Grade 9 (refresh)

Note. In spring 2009 the sample will be refreshed for Grades 10 and 11. In spring 2010 a new
sample will be drawn from grade 10 and the grade 11 sample will be refreshed. spring 2009
and 2010 surveys will be conducted to fulfill Annual Performance Report (APR) requirements.

Data Sources

The following data will be collected (details are provided below):

Student surveys. These provide student-level outcome data on substance use and attitudes about substance use.
School administrative records. These provide school-level outcome data on academic achievement and disciplinary actions and baseline information on these outcomes.
School-level drug testing collection form. This provides data on the implementation of the drug testing process at each drug testing round.
Staff interviews. These provide detailed information about the testing programs being implemented and on the presence of other drug and alcohol prevention activities in each school.

Exhibit 2 details the data collection plan, including the timeframe for data collection.

Exhibit 2
Data Collection Plan

Instrument Mode	Respondent	Spring 2007	Fall 2007	Spring 2008	Spring 2009	Spring 2010	Key Data
Telephone calls or site visits	Schools	x	X	X	x	x	Student rosters and activity participation lists

Consent Form	Parents	x	X	x^a	x^a,b,c	x^a,b,c	N/A

Student Survey	Students	x	X	x	x^b,c	x^b,c	Demographic information Drug use Attitudes about drug use and school

Administrative records data^c	Schools	x		x	x^b	x^b	Schoolwide drug-related incidents

Anonymous drug test records data^c	Schools	x		x	x^b	x^b	Drug test records

Semi-structured interviews obtained during site visits^c	School Staff	x	X	x			Characteristics of drug prevention programs and policies, including mandatory-random drug testing policies, implementation of drug testing policies

^aA cross-section of students will be recruited and parental consent sought each year to refresh the sample of students who leave the study schools and add a sample of students for the analysis of repeated cross sections between grades. ^bSurvey and school records data will be obtained to meet performance reporting data requirements. ^cData will not be collected from nonparticipating external schools.

Impact Analysis

The sampling plan and data collection described above will support two types of impact analysis. First, we will calculate the impact of drug testing on drug use at the school level. Second, we will calculate the impact of drug testing on drug use at the student level, by separately examining MRDT’s impact on likely activity participants and likely non-participants.

The primary issue that must be addressed in order to calculate impacts on the drug use of activity participants is that drug testing might affect who participates in activities. If activity participation is affected by the intervention, comparing the drug use of activity participants in treatment schools to the drug use of activity participants in control schools could be misleading. For example, some students may choose to stop participating in activities in order to continue to use drugs. In that case, drug use of activity participants in treatment schools will appear lower than in control schools, but not because overall drug use has declined.

To address this issue, we will calculate impacts on the drug use of likely activity participants rather than actual activity participants. Students who are likely to be activity participants will be identified using the baseline student survey, which will be administered prior to random assignment. This survey will ask students about their activity participation in the current (2006–2007) school year and their plans to participate in activities in the 2007–2008 school year. Because these survey questions are asked prior to random assignment, students’ responses cannot be affected by implementation of the intervention. The longitudinal sample will be used for this analysis, since those are the only students for whom we will have baseline survey data.^³

Technical details of the impact analysis are described in section A.16.

Estimation of Contamination in Control Schools

The contamination analysis calls for selecting one nonparticipating external school per grantee (note that two external schools will be selected for the 3 largest grantees, resulting in a sample of 12 external schools). Our proposed approach to identify schools is to use NCES’ Common Core of Data (CCD) to find schools in nearby districts that are similar to study schools in terms of characteristics such as racial composition, Title 1 status, and other characteristics available in the CCD. Physical distance from grantee schools will also be considered. Recruiting schools to participate in the study for the purposes of evaluating contamination will be challenging, given that these schools are receiving no services and have little incentive to cooperate. For this reason, we suggest an annual incentive of $1,500 to schools that agree to participate. To encourage well-matched external sample schools to agree to participate in this study, we also plan to offer to share descriptive summary statistics on drug use in the external schools with school officials upon the conclusion of the impact study’s data collection activities.

Our preferred method of identifying the external schools (using a grantee with 4 schools as an example) is to form 2 matched pairs (or blocks) among the multiple schools in a grantee district, and find the external school that best matches the average school for one of those 2 blocks^⁴. For example, suppose our only matching variable were percentage approved for free or reduced-price lunch, and that the average percentage for schools in the grantee district are 20, 30, 70, and 80. We could form 2 blocks—one with the 2 lower-percentage schools and one with the 2 higher-percentage schools—and then look for an external school that is the best match for 1 of the 2 blocks. For example, suppose the candidate external schools have percentage eligible for free or reduced-price lunch of 25, 45, 50, and 90. We would pick as the best match the one with 25%, since that is the closest match to one of the 2 blocks. For the contamination analysis, we would compare the school in the low-percentage score block assigned to the control group to the selected external school.

How, by Whom, and for What Purpose Information is to be Used

ED has contracted with RMC Research Corporation and its subcontractors, Mathematica Policy Research and COSMOS Corporation, to collect and analyze data for the Impact Evaluation of Mandatory-Random Student Drug Testing. IES will share the information from this study with OSDFS and ONDCP to help them improve their substance abuse prevention grant programs. This evaluation responds directly to a request made by the Budget Office within OMB to conduct a rigorous evaluation of mandatory random student drug testing programs.

The information collected will support policy decisions about future funding of MRDT and other substance abuse prevention programs within ED and at the Federal, State and local levels. In addition, the data will be a resource to support additional research on school-based substance abuse prevention strategies by academic researchers or other interested parties. Restricted-use data files from the evaluation will be submitted to IES and disseminated accordingly, and can be used for independent studies on other topics of interest to the substance abuse prevention research and policy community.

Parental Consent Forms

A parental consent form is necessary to initiate student level data collection activities. The contractor will work with district and school officials to identify the most appropriate and effective methods of distribution, drawing on a review of the literature on consent form distribution (Ellickson & Hawes, 1989; Ji, Pokorny, & Jason, 2004; Leakey, Lunde, Koga, & Glanz, 2004; MacGregor & McNamara, 1995; McMorris et al., 2004). When applicable, school staff will only be involved in the dissemination (and not the collection) of the parental consent forms, since schools will not have Institutional Review Board (IRB) approval for data collection. The approach for disseminating and collecting forms will be similar in treatment, control and external schools. The form has been included in Appendix A.

To facilitate the study team’s collection of the consent forms, we will request parental contact information from each school in order to make follow-up contact with parents. Parental consent forms will be returned to COSMOS in individual postage-paid envelopes. COSMOS will monitor return rates at each school and follow-up with nonresponding parents as appropriate. Other communication will be in the form of reminder announcements at PTA and other school meetings attended by parents, reminders letters, and reminder e-mail notifications, when possible.

Student Surveys

The primary outcome measures for the study will come from student surveys. Baseline student survey data will be collected to assess the similarity of the treatment and control groups, to increase the precision in the measurement of outcomes, and to allow for estimation of changes in self-reported drug use for specific students over the course of the study. The follow-up student survey provides outcome data on students’ drug use and perceptions about drug use. The survey will be brief to minimize the data collection burden on schools and students. We are requesting OMB clearance to administer the same survey on as many as 5 data collection periods (as detailed in Exhibit 2 above). Student surveys will be administered by staff hired and trained specifically for this purpose and selected based on their experience working with school administrators and staff. The form survey has been included in Appendix B.

School Administrative Records

Collecting schoolwide records data on substance-related incidents will allow for estimating the impact of the testing program on school outcomes such as the number of suspensions. We are requesting OMB clearance to collect these data in all 4 data collection periods (as detailed in Exhibit 2 above; the same form will be used each time). The form has been included in Appendix C.

The schoolwide record form will be shared with schools early in the study. Interviewers, who will be on site in spring 2007 and spring 2008, will assist in compiling the school wide records as necessary and identify a point of contact in the school for follow-up communication regarding the school wide records form. Completed forms will be collected by the site visitors or mailed to RMC Research in a self-addressed stamped envelope. For the 2 data collection points when interviewers will not be on site (spring 2009 and spring 2010), the RMC Research data coordinator will mail the record forms to school contacts, along with a self-addressed envelope for returning the forms, maintaining frequent communication with the site until the data are received.

Drug Testing Collection Form

The drug testing collection form will allow the study team to collect data at the school level. Because drug testing is to be conducted only in treatment schools, the study cannot estimate impacts on drug use according to drug test results. However, the data will be used to describe the basic drug testing activities that occurred at each treatment school and examine the trend in drug test results over the duration of the study. A copy of the drug testing collection form is included in Appendix D. The study will collect drug testing records from treatment schools each time a mandatory random drug-test is implemented starting after the baseline survey (spring 2007) through spring 2010. We are requesting OMB clearance to collect these data up to an average of 3 times in a school year during the first year and 9 times in a school year during the next three years (the same form will be used each time). Procedures for collecting these forms will be similar to those for the schoolwide record form.

School Staff Interviews

Interviews with school staff will address two of the proposed research questions. First, the staff interviews will help identify the characteristics of the MRDT programs. Second, staff interviews will help to determine what prevention and intervention policies, programs, or strategies other than MRDT are being implemented in the treatment and control schools. At or near the date of the first three rounds of student surveys, in each of the participating treatment and control schools, one interviewer will interview up to three school staff who are knowledgeable about the school’s substance abuse prevention strategies. We are requesting OMB clearance to conduct the interviews at this time. Staff interviews will be conducted with key school staff who are in leadership positions and are knowledgeable about the school’s substance abuse prevention strategies or who administer the school’s drug-testing program. The data collection coordinator will identify interviewees in conversation with the primary district contact person. School staff may be excluded from the interview if they have not worked at the school for more than 1 year and/or do not feel like they are familiar with the school’s substance abuse prevention strategies. Potential interviewees may also decline to participate. The list of topics to be covered in the school staff interviews, an introductory script, and and interview protocol are included in Appendix E.

3. Use of Automated, Electronic, Mechanical or Other Technological Collection Techniques

The data collection plan reflects sensitivity to issues of efficiency, accuracy, and respondent burden. Where feasible, information will be gathered from existing data sources, such as school records, using straightforward reporting forms. School records information will be gathered via computer files if a school district prefers this method. In many cases, however, the data can be obtained only from students and school staff.

The student surveys will be administered in a group setting and will be completed with paper and pen. For collection of schoolwide drug-related incidents, we will use existing data in electronic format where possible. Where manual collection or abstraction is required, we will use standardized forms. In compliance with OMB directives, we will provide an electronic version as an option for all paper data collection instruments.

In addition, the contractor’s electronic mail address and toll-free telephone number are included on the front of questionnaires for respondents who may have questions. These procedures are all designed to minimize burden on respondents.

4. Efforts to Identify and Avoid Duplication

The purpose of the data collection is to estimate the impacts of MRDT on student drug use in schools that implement such policies. While one peer-reviewed study has attempted to estimate the effects of mandatory-random drug testing, it used a nonexperimental design and therefore the evidence must be viewed with caution given the likely selection bias affecting the results. Conducting a randomized trial to evaluate the impacts of MRDT requires randomly assigning schools into either a treatment group or control group and collecting data within those schools. The information to be collected from student surveys and site visits in these randomized schools will not be available elsewhere.

5. Impact on Small Business and Other Small Entities

The primary entities for this study are schools and the districts to which they belong, along with the students who attend them. No small businesses or other small entities participate in this study. Burden is reduced for all respondents by requesting only the minimum information required to meet study objectives. The burden on schools and districts has been minimized through the careful specification of information needs, restricting questions to generally available information, and designing the data collection strategy—particularly the survey methods—to minimize burden on respondents. The evaluation contractor will coordinate all data collection so as to minimize burden on school and district staff, students, and their parents.

6. Consequences to Federal Programs or Policies if Data Collection Is Not Conducted

Not conducting this data collection would significantly impede ED’s ability to assess the impacts of MRDT on student drug use. OMB has specifically asked that ED conduct a study of MRDT as there is no rigorous research on this intervention. Without information on the impacts of MRDT it will be difficult for the OSDFS to efficiently award program dollars and provide technical assistance in the future.

7. Special Circumstances

There are no special circumstances.

8. Solicitation of Public Comments and Consultation With People Outside the Agency

a. Federal Register Announcement

A 30-day notice to solicit public comments was published in the Federal Register on December 21, 2006.. Any comments received in the comment period were addressed prior to submission to OMB for final approval.

b. Consultations Outside the Agency

Consultations on the research design, sample design, and data sources occurred during the study’s design phase and will continue to take place throughout the study. The purpose of such consultations is to ensure the technical soundness of the study, and to verify the importance, relevance, and accessibility of the information sought in the study. An evaluation advisory panel provided input on the initial study design and data collection plan in the conceptualization phase. The advisory panel members represent a number of the nation’s leading researchers on student drug testing and evaluation design. The panel included:

Ms. Lisa Brady, South Hunterdon Regional High School, West Amwell, NJ.

Professor Linn Goldberg, Oregon Health and Science University.

Professor Robinson Hollister, Swarthmore College

Professor Rebecca Maynard, University of Pennsylvania

Dr. Wilson Compton, National Institute of Drug Abuse

To provide advice on the study, the evaluation team formed a Technical Work Group (TWG) of researchers who combine expertise especially in large-scale random assignment studies and impact evaluation, as well as knowledge of substance abuse prevention strategies. The evaluation team has consulted (and will continue to consult) with the TWG on the overall study design, the data collection plan, and the survey instruments. Members of the TWG include:

Dr. Robinson Hollister, Swarthmore College

Dr. Mark Lipsey, Vanderbilt University

Dr. Jeff Smith, University of Michigan

Dr. Lawrence Scheier, Washington University and LARS Research Institute

Dr. Patrick O’Malley, University of Michigan

Dr. Michael Fendrich, University of Wisconsin-Milwaukee

Dr. Linn Goldberg, Oregon Health and Science University

The contracted evaluators also provided input on the revised design and data collection plan.

9. Respondent Payments

Sampled Students

Students in treatment and control schools and in external schools who are asked to complete a survey will receive a movie ticket for a local theater (valued at approximately $7) for returning the completed consent form, regardless of whether or not consent is granted. We believe that such an incentive is necessary to achieve an adequate return rate and to ensure an unbiased respondent sample. Only by achieving high rates of participation among sampled students can the study be representative both of likely substance users and non-users. Students will complete the surveys during regular school hours.

Grantee Schools

Schools participating in the grant program are already receiving funds from the OSDFS grants and no payments will be made to offset burdens in those sites.

External Schools

Recruiting external schools to participate in the study for the purposes of evaluating contamination will be challenging, given that these schools are receiving no services and have little incentive to cooperate. Following the National Center For Education Evaluation (NCEE) guidelines for incentives or payments for the recruitment of comparison schools, we propose an incentive of $1,500 per school per year for the external schools to participate in the evaluation. We also plan to offer to share descriptive summary statistics on drug use in the external sample of schools with school officials upon completion of the impact study’s data collection activities.

10. Assurance of Confidentiality

RMC Research Corporation and its subcontractors will follow procedures for ensuring and maintaining participant privacy, consistent with Education Sciences Reform Act of 2002. Title I, Part E, Section 183 requires of this Act “All collection, maintenance, use, and wise dissemination of data by the Institute” to “conform with the requirements of section 552 of title 5, United States Code, the confidentiality standards of subsection (c) of this section, and sections 444 and 445 of the General Education Provision Act (20 U.S.C. 1232g, 1232h).” These citations refer to the Privacy Act, the Family Educational Rights and Privacy Act, and the Protection of Pupil Rights Amendment. In addition for student information, “The Director shall ensure that all individually identifiable information about students, their academic achievements, their families, and information with respect to individual schools, shall remain confidential in accordance with section 552a of title 5, United States Code, the confidentiality standards of subsection (c) of this section, and sections 444 and 445 of the General Education Provision Act. Subsection (c) of section 183 referenced above requires the Director of IES to “develop and enforce standards designed to protect the confidentiality of persons in the collection, reporting, and publication of data.” Subsection (d) of section 183 prohibits disclosure of individually identifiable information as well as making any the publishing or communicating of individually identifiable information by employees or staff a felony.

Research participants will be informed about the nature of the information that will be requested and privacy protection afforded them. Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district, school, or individual. Respondents will also be informed that their names will not be associated with their answers and that no one outside the study team will have access to this information except as may be required by law, regulation, or subpoena or unless permission is given by both the parent and participating child.

We will use the following process to inform parents/guardians of data collection and privacy procedures and to obtain their consent. Parents/guardians of sampled students will be informed of the nature of the study through a letter. Accompanying this letter will be a consent form, which parents will need to fill out to provide their consent for their child to participate in study activities. Parents will be informed at this time that the study is voluntary. The active consent form is included in Appendix A.

The contractor has a long history of protecting privacy of records, and considers such practice a critical aspect of the scientific and legal integrity of any survey. In addition, ED will enforce the following standards designed to protect the confidentiality of persons in the collection, reporting, and publication of data:

Identifying information on schools, students, and parents is maintained on separate forms, which are linked to the surveys only by a sample identification number. These forms are separated from the surveys as soon as possible.

Access to the file linking sample identification numbers with respondents’ identification and contact information is limited to a small number of individuals who have a need to know this information. At the conclusion of the research these data are destroyed.
Access to the hard copy documents collected from respondents is strictly limited. Documents are stored in locked files and cabinets. Discarded material is shredded.
Computer data files are protected with passwords and access is limited to specific users. With especially sensitive data, the data are maintained on removable storage devices that are kept physically secure when not in use.

11. Justification for Sensitive Questions

The objective of the study is to estimate the extent to which there is a reduction in the reported use of tobacco, alcohol, and illicit substances among particular groups of students (e.g., athletes, participants in extracurricular activities). Illicit substance use is considered a sensitive topic; however, it is necessary to collect these sensitive data from students to answer the key research questions. In particular, the student survey will ask questions about student substance use in 3 reference periods: over their lifetime, during the last 6 months, and during the last 30 days. These questions have been derived from prior surveys of substance use, including the national Monitoring the Future Study (NIDA, 2003). The questions have been tested for their reliability and stability in repeated measures data collection and the use of them in this study will allow us to place the descriptive findings (e.g., at baseline) in a national context. By collecting data in a detailed manner regarding substance use over these three reference periods, this evaluation will provide timely and policy-relevant research results to policymakers seeking to craft effective solutions to the serious problems associated with substance use among high school students across the country.

12. Total Annual Cost Burden to Respondents

Exhibit 3 presents our estimates of respondent burden for the requested data collection.

Student Surveys

A sample of 200 students in each of 23 treatment schools, and 22 control schools will be surveyed on 5 occasions. In addition, 200 students from each of the 12 external schools will be surveyed during the first 3 survey administrations. In sum, approximately 11,400 students will be surveyed in spring 2007, fall 2007, and spring 2008 and 9,000 students will be surveyed in spring 2009 and spring 2010. The survey will be administered in groups of students by external survey administrators and is expected to take approximately 30 minutes to complete. We assume the estimated hourly wage for high-school respondents is zero as they will complete the survey during regular school hours.

School Records Form

Schoolwide records will be collected on 5 occasions in the 45 treatment and control study schools; one form will be completed for each school at each round of data collection. Schoolwide records will not be collected in the nonparticipating external schools. We assume the time needed to complete the form is approximately 60 minutes. Therefore, the total respondent time for collecting schoolwide records each year equals 45 hours.

Drug Testing Records Form

The study team will collect drug testing records from 23 treatment schools (and 22 control schools starting in fall 2008) each time a mandatory random drug-test is implemented. Drug testing may begin in treatment schools in spring 2007 following the administration of the baseline survey. Grantees are currently planning testing schedules that vary by grantee, ranging from weekly to 3 times per year. Specifically, testing is planned to be conducted weekly by 1 grantee (2 schools), 1-2 times per month by 1 grantee (7 schools), monthly by 1 grantee (12 schools), 2 or 3 times per season (i.e., fall, winter, and spring by 2 grantees (9 schools), 4 times per year by 2 grantees (10 schools, and 3 times per year by 1 grantee (3 schools). One grantee with 2 schools has not decided on their schedule. One form will be completed for each school at each round of drug testing,. We assume the time needed to complete the form is approximately 15 minutes.

Staff Interviews

Up to 3 school staff in each of the 45 participating treatment and control schools will be interviewed at or near the date of the first 3 rounds of student surveys (for a total of 405 interviews). Interviews will not be conducted in the nonparticipating external schools. Each interview is expected to take approximately 60 minutes to complete. Therefore, the total respondent time for conducting staff interviews each year equals 135 hours annually.

Exhibit 3
Respondent Burden Estimates For Data Collection

Informant	Number of Responses	Number of Rounds	Average Time Per Response (Hours)	Total Respondent Time (Hours)	Estimated Hourly Wage (Dollars)	Estimated Lost Burden to Respondents (Dollars)
High school students
Spring 2007	11,400	1	.5 (30 min)	5,700	--^a	--
Fall 2007	11,400	1	.5 (30 min)	5,700	--^a	--
Spring 2008	11,400	1	.5 (30 min)	5,700	--^a	--
Spring 2009	9,000	1	.5 (30 min)	4,500	--^a	--
Spring 2010	9,000	1	.5 (30 min)	4,500	--^a	--
Total				26,100	--^a	--
School records form
Spring 2007	45	1	1 (60 min)	45	$22.81^b	$1,026.45
Spring 2008	45	1	1 (60 min)	45	$22.81^b	$1,026.45
Spring 2009	45	1	1 (60 min)	45	$22.81^b	$1,026.45
Spring 2010	45	1	1 (60 min)	45	$22.81^b	$1,026.45
Total				180	$22.81^b	$4,105.80
Drug testing records form
2006–2007	23	3^c,d	.25 (15 min)	17.25	$22.81^b	$393.47
2007–2008	23	9^c	.25 (15 min)	51.75	$22.81^b	$1,180.42
2008–2009	45	9^c,e	.25 (15 min)	101.25	$22.81^b	$2,309.51
2009–2010	45	9^c,e	.25 (15 min)	101.25	$22.81^b	$2,309.51
Total				271.50	$22.81^b	$6,192.91
School Staff Interviews
Spring 2007	135	1	1 (60 min)	135	$22.81^b	$3,079.35
Fall 2007	135	1	1 (60 min)	135	$22.81^b	$3,079.35
Spring 2008	135	1	1 (60 min)	135	$22.81^b	$3,079.35
Total				405	$22.81^b	$9,238.05
Total				26,956.50		$19,536.76

^aIt is assumed that the estimated hourly wage for high-school respondents will be zero because they will be completing the survey during regular school hours. ^b Hourly wage calculated as average salary paid divided by mean number of work hours in a work year averaged across professional, central office and school-building level staff. Data source: Educational Research Service, National Survey of Salaries and Wages in Public Schools, 2004–2005. ^cGrantees are currently planning testing schedules that vary by grantee, ranging from weekly to 3 times per year. Number of rounds represent the average number of drug testing episodes to occur across all grantees. ^dDrug testing episodes in 2006-2007 is limited to March, April, and May. ^eDrug testing will occur in treatment and control schools.

The annualized burden for 2007 is11,758 hours with 11,603 responses.

13. Estimate of the Total Annual Burden to Respondents or Recordkeepers

There are no direct costs to participants. The only costs to participating students are the opportunity costs of student’s time required to complete the survey. While we have assumed that financial cost is zero to these respondents because the surveys and forms will be completed during regular school hours, we recognize that students will each miss approximately 30 minutes from their classes to complete the survey. There will also be the cost of staff time required to complete the drug testing forms, and schoolwide record forms and staff interviews, which we calculate as $19,537 (see Exhibit 3); this is also not a direct cost to staff. The Impact Evaluation of Mandatory-Random Student Drug Testing does not place any capital equipment, start-up, or record maintenance requirements on respondents.

14. Estimate of Annualized Cost to the Federal Government

The estimated cost to the federal government of conducting the Impact Evaluation of Mandatory-Random Student Drug Testing is based on the budget awarded by the government. The estimated cost to the Federal government for the impact study is $4,174,863 or an average annual cost of $1,043,716 towards collection, analysis and reporting of the data described herein. There is an additional cost to the Federal government for conducting the study of contamination (collecting data from external schools) which is $854,190 or an average annual cost of $213,548. These costs include personnel benefits, overhead, supplies, and indirect costs.

15. Program Changes or Adjustments

This request is for this information collection with a program change of 11,758 hours.

16. Plans for Tabulation, Analysis, and Publication of Results

This section describes the analysis, tabulation, and publication of results focusing on the main study questions outlined in the introduction. We first address plans for tabulation related to the impact analysis (section a); we then address plans for tabulation related to the implementation analysis (section b); we then address the publication plans and time schedule (section c).

a. Impacts of Mandatory-Random Student Drug Testing

Random assignment of schools to treatment and control groups should ensure no systematic differences between the treatment and control groups. To verify that the treatment and control groups are similar, we will conduct two sets of tests. First, we will conduct statistical tests (using baseline student survey data) to assess the similarity of the baseline characteristics of the students in the treatment and control schools. Second, we will examine school records to determine the similarity of the baseline characteristics of the treatment and control schools.

After conducting the baseline analyses noted above, we will estimate the impacts of school-based mandatory-random drug testing on student self-reported drug use. We will do this both for the full sample of activity participants and nonparticipants and also separately for these 2 groups. A simple unbiased estimator of the program impact is the difference between average treatment and control group outcomes. This difference will be based on the average self-reported level of drug use among students in schools randomly assigned to the treatment group and among students in schools randomly assigned to the control group. A t-test of the difference will be used to assess the likelihood that the difference in student drug use was due to chance or to the drug testing policy. As described in section B, the study’s sample size is sufficient to allow detection of policy-relevant impacts with a high probability.

Building on the simple difference-in-means estimator, we will estimate a regression model that uses information collected about baseline student drug use. Incorporating baseline information on student drug use allows for the calculation of more precise estimates of the impacts of MRDT than simply examining the difference between the 2 groups after testing is implemented, because the baseline measure explains a large proportion of the variation in the follow-up measure.

To account for the nesting of students within schools, we will use a 2-level hierarchical linear model (HLM) to estimate regression-adjusted impacts. The Level One model relates students’ post-intervention drug use to their baseline drug use, personal characteristics, a constant term, and a residual term. The Level Two model relates the school-level average drug use to grantee indicator variables and grantee-specific treatment indicators (allowing the effect of treatment to vary by grantee), and a residual term. The overall impact of the intervention is the simple average of the impacts for each grantee. The simple average of grantee-specific impacts answers the question “How effective is the average grantee in the study at affecting student outcomes?” We will also calculate a weighted average, which will represent the effect on the average student.

Level One: Students (i) within schools (j):

Level Two: Schools (j):

Where:	y =	Student outcome at follow up (from student surveys)
	X =	Baseline student characteristics
	y₀ =	Baseline measure of the outcome (from student surveys)
	SITE_k=	Grantee indicator variable (1 if student is in grantee k, and 0 otherwise)
	T =	Treatment status indicator (1 if treatment, 0 if control)
	Z =	Baseline school characteristics (from grant applications or other data sources)
	β, γ, δ,  =	Coefficients to be estimated
	θ, e =	Random (and mean zero) school-level and individual-level error components

Additionally, we will conduct subgroup analyses, where subgroups are defined by student characteristics. This will occur by comparing the average self-reported drug use for each subgroup in the treatment schools with the average for comparable students in the control schools. Because subgroups will be defined using data collected before random assignment, the study’s experimental design supports unbiased estimates for these subgroups.

Calculating Contamination Effects. To calculate differences in drug use between students in the control schools and students in external schools, we will use a regression approach similar to that described above. Specifically, we will calculate the average difference in drug use between students in the control schools and students in the external schools, adjusting for baseline differences and taking into account the nesting of students in schools.

b. Implementation Analysis

The implementation analysis involves 2 components. The first component involves describing the intervention that each grantee intends to implement, based on information about school characteristics and program design from each grantee’s application. The second component involves describing the implementation of the MRDT programs and any other substance use prevention policies, programs, or practices being implemented in treatment and control schools. This component will be based on staff interview data and other secondary data sources (e.g., school policies).

c. Publication Plans and Time Schedule

The primary product of the evaluation will be a major report. This report will cover data collection from schools included in the study during the 2006–2007 and 2007–2008 school years. A draft will be submitted in spring 2009. The final report will be completed in summer 2009. The findings will be based on all data collected during the study. An annual performance reporting memorandum will be submitted in the summer of 2007, 2008, 2009, and 2010. In addition, a memorandum will be prepared for ED that displays the trajectory of descriptive statistical results for the external schools over the 3 waves of data collection, with a discussion of its similarity (or lack thereof) to the trajectory of results from the control schools.

Exhibit 4
Schedule of Key Activities

Activity

Schedule

Study design and materials preparation

Data collection

Interim baseline data analysis

Study report

Memorandum on external schools

Annual performance reporting

September 2006–January 2007

Spring 2007–May 2010

March 2007–July 2007

June 2008–February 2009

June 2008–October 2008

June 2007–July 2010

17. Approval Not to Display the Expiration Date for OMB Approval

Approval Not to Display the Expiration Date for OMB approval is not requested.

18. Exception to the Certification Statement

No exceptions to the certification statement are requested or required.

References

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society (Series B), 57, 289–300.

Ellickson, P.L. & Hawes, J.A. (1989, February). An assessment of active versus passive methods for obtaining parental consent. Evaluation Review, 13(1), 45–55.

Goldberg, L., et al., (2003). Drug testing athletes to prevent substance abuse: Background and pilot study results of the SATURN (Student Athlete Testing Using Random Notification) study. Journal of Adolescent Health, 32, 16–25.

Ji, P.Y., Pokorny, S.B., & Jason, L.A. (2004). Factors influencing middle and high schools’ active parental consent return rates. Evaluation Review, 28, 578–591.

Johnston, L.D., O’Malley, P.M., Bachman, J.G., & Schulenberg, J.E. (2006). Monitoring the Future national results on adolescent drug use: Overview of key findings, 2005. (NIH Publication No. 06-5882). Bethesda, MD: National Institute of Drug Abuse.

Leakey, T., Lunde, K.B., Koga, K., & Glanz, K. (2004). Written parental consent and the use of incentives in a youth smoking prevention trial: A case study from Project SPLASH. American Journal of Evaluation, 25(4), 509–523.

MacGregor, E. & McNamara, J. (1995). Comparison of return procedures involving mailed versus student-delivered parental consent forms. Psychological Reports, 77, 1113–1115.

McMorris, B.J., Clements, J., Evans-Whipp, T., Gangnes, D., Bond, L., Toumbourou, J.W., & Catalano, R.F. (2004, February). A comparison of methods to obtain active parental consent for an international student survey. Evaluation Review, 28(1), 64–83.

Murray, D.M., Varnell, S.P., & Biltstein, J.L. (2004). Design and analysis of group-randomized trials: A review of recent methodological developments. American Journal of Public Health, 94, 423–432.

National Institute on Drug Abuse. (2003). Monitoring the future: National results on adolescent drug use. Overview of key findings. National Institutes of Health, U.S. Department of Health and Human Services.

1Activities subject to drug testing may differ across grantees (e.g., athletes only or students involved in all extracurricular activities). Thus the definition of “participant” may vary across grantees. For a grantee with 2 schools including a treatment school that is drug testing all athletes, baseline survey responses about participation in sports will be used to define likely participants and nonparticipants in both the treatment and control schools.

2Because it is unlikely that the intervention would cause students to move to another school district, we will not follow students who leave the school district. We will, however, track sampled students who transfer between schools within a district.

3In fall 2007, we will ask all students retrospective questions regarding their activity participation during 2006–2007. If we find that students’ answers to these retrospective questions are reliable, then we could use these questions to identify likely participants among the refresh sample. This would allow us to include the refresh sample in the analysis of impacts on the drug use of likely participants.

4This method requires that random assignment to condition be made within block, not across all 4 schools within district.

File Type	application/msword
File Title	RMC Basic Report & Proposal Template
Author	Lori Sievert
Last Modified By	Sheila.Carey
File Modified	2007-04-26
File Created	2007-04-26