Att_1850-0814 v.2 Part B SS revised 4-15-09

In order to obtain the sample for the study, first schools will be sampled and then a subgroup of students within those schools will complete the outcomes measures. The following explains the sampling methods for both these levels of sampling.

Schools. The sampling frame of schools will be constructed using publicly available information from the Common Core of Data regarding poverty (based on the percentage of students in each school eligible for free and reduced meals), enrollment size, locale, and race/ethnicity. In order to address the analytical objectives of this study and manage the many operational challenges, the sampling frame will be limited to regular (public, non-charter, and non-magnet) schools with grades 6-8 with a 6^th-grade population of at least 250 students. We require at least 250 students in each school in order to obtain a sufficiently large high-risk subsample. School districts with fewer than three eligible schools will be excluded because they will yield insufficient matched pairs of schools to include in the study. Moreover, schools in the sampling frame will be selected to be urban, high poverty, and/or high minority schools, factors that are strongly correlated with school violence. Because data on actual rates of school violence are often not collected and, when they are, are notoriously of poor validity, the aforementioned characteristics will be used as proxies for identifying schools with elevated levels of violence and aggression.

Working from the sampling frame, districts will first be screened to determine their eligibility and identify any approval process required to contact sampled schools. Districts will be deemed ineligible to participate if there is a current or planned district mandate for all middle schools to implement curriculum based or whole-school violence prevention programs that are similar to RiPP or Best Behavior. Upon receipt of district approval to contact schools, schools will be mailed study materials and called to invite participation into the study. Criteria such as 6^th grade enrollment, percentage of minority students, percentage of students eligible for free and reduced lunch will be requested to verify school level eligibility. Schools will be excluded if they are currently implementing or planning to implement programs similar to the RiPP and or Best Behavior programs or if future redistricting plans will negatively impact the 6^th grade enrollment levels for the ensuing academic year. Schools must have a class schedule that can accommodate sixteen 50- minute lessons for all 6^th, 7^th, and 8^th graders and identify placement for the RiPP curriculum in their normal academic day. Schools will agree to be randomly selected into the intervention or control group. If selected for the control group, schools must also agree to refrain from implementing similar violence prevention programs throughout the three years of the study. Within each school district, matched pairs of schools will be created based on levels of poverty and ethnicity mix. One school from within each pair will be randomly assigned to each of the two experimental conditions.

Students. We will survey sixth graders in the 2006-2007 school year, seventh graders in the 2007-2008 school year, and eighth graders in the 2008-2009 school year. Exhibit 9 shows the number of students we anticipate will participate in each round of the student survey. We will obtain consent from parents of sixth-grade students in fall 2006. We anticipate that 9,720 sixth-grade students (80% of the sixth-grade population) will receive parental consent and will be able to participate in the baseline survey. In spring 2007, we will obtain parental consent for sixth grade students who have moved into the schools since the baseline survey and will again survey sixth graders at that time. In spring 2008 we will survey seventh-grade students who completed the baseline the previous year and who remain in the school; their prior parental consent will still apply. We estimate that of the 9,720 students in the baseline survey, 7,760 (80%) will remain in the study schools and complete the survey in spring 2008. In spring 2009 we will obtain consent for eighth graders who have transferred into the school since the baseline survey (and for whom the baseline consent would not apply) and will survey eighth graders at that time. We anticipate that 9,720 students (80% of the population) will participate in the spring 2009 survey.

Exhibit 9. Expected Number of Students, for All Students and the High-Risk Subset

	Total sample	Per school	Comments
All Students
Available	12,160	304
Fall 2006	9720	243	Census, assuming 80% parental consent
Spring 2007	9720	243	Census, assuming 80% parental consent
Spring 2008	7760	194	Students remaining from fall 2006 survey, assuming 20% attrition

Spring 2009	9720	243	Census, assuming 80% parental consent
Subset of High-Risk Students
Fall 2006	2560	64	Identified from fall 2006 survey
Spring 2007	2360	59	Assuming 8% attrition of identified group (within school year)
Spring 2008	1960	49	Assuming 16% attrition of remaining group (one year later)
Spring 2009	1640	41	Assuming 18% attrition of remaining group (one year later)

We will identify a subset of students at high risk for violent and aggressive behavior, based on responses to the baseline violence and aggression student self-report in fall 2006. We anticipate identifying on average 64 high-risk students per school. Throughout the survey schedule described for all students, we will follow the high-risk students that remain enrolled in the participating districts. The number of students in this high-risk group will decrease over time to the extent that some students leave study schools. We estimate that of the 64 high-risk students per school at baseline, 59 will participate in the spring 2007 survey, 49 in the spring 2008 survey, and 41 in the spring 2009 survey. Consequently, a total of 1,640 high-risk such students are anticipated across the 40 schools at the final survey with approximately 820 high-risk students in each of the control and treatment groups, respectively.

Teachers. The teacher survey will be administered to a random sample of 24 teachers (stratified by grade) at each of the middle schools participating in the study. Teachers will complete the survey in spring of 2007, 2008, and 2009, with a new random sample of teachers selected each year.

2. Statistical Methods for Sample Selection and Degree of Accuracy Needed

Statistical power indicates the likelihood that an intervention effect will be judged to be statistically significant, given the assumptions of the specified model. In other words, power is the likelihood of observing a statistically significant difference where such a difference exists. An under-powered study then, is one in which the investigators risk failing to notice a significant intervention effect. Previous small-scale, less rigorous evaluations of the RiPP intervention have reported significant effects for suspensions, violent behaviors, self-reported frequency of physical aggression, drug use, peer provocation, non-physical aggression among boys, attitudes for violence in boys, delinquent behaviors, and victimization (Farrell, Valois, Meyer, & Tidwell, 2003, Farrell, Meyer, Sullivan, & Kung, 2003; Farrell, Meyer, & White, 2001; Farrell, Valois, & Meyer, 2002). The current study is designed to detect an effect size of 0.20, given the effect sizes found in previous studies of RiPP and the fact that the inclusion of the Best Behavior program is anticipated to increase program effects. In addition, experts with whom we consulted regard a minimum detectable effect (MDE) of 0.20 as reasonable given the estimated monetary and opportunity costs of a school-based intervention combining curricular and whole-school approaches.

In order to determine the sample size required for an MDE of 0.20, we assumed an ICC of 0.045 based on a conservative review of previous research measuring self-reported violent behavior in students (Janega, Murray,Varnell, Blitstein, Birnbaum, & Lytle, 2004). We also assume an alpha-level of 0.05 and a two-tailed significance test. Given these assumptions, randomly assigning 40 schools to either treatment or control conditions and surveying 243 students at each school will provide sufficient power for testing the null hypothesis regarding treatment effectiveness on schools (Exhibit 10). These estimates are based on the mixed-effect model as appropriate for an analysis that includes random effects at the individual and school level. Statistical power could be improved even further by including covariates in the models such as baseline data.

Exhibit 10. 80% Statistical Power as a Function of Minimal Detectable Difference
(Number of Students Per School = 243)

Minimal Detectable Difference	Number of Schools Required for Each Study Condition
0.24	14
0.22	17
0.20	20
0.18	25
0.16	32

Note: Calculations assume = .05 and a two-tailed test.

Note: Calculations are based on ICC = 0.045

Exhibit 11 provides a power table for the nested cohort analysis that will be used to assess the sub-sample of high risk youth. Using assumptions previously noted regarding attrition rates across the three years of the program, we anticipate approximately 41 students from each school will provide data for this analysis. Although fewer students provide data for the cohort analysis than the cross-sectional analysis, this potential limitation is offset by a smaller anticipated ICC and a gain in precision both of which are associated with taking replicate measures on individuals. In this case, a sample size of 40 schools will allow us to detect effects of only 0.20 or larger.

Exhibit 11. 80% Statistical Power as a Function of Minimal Detectable Difference
(Number of High Risk Students Per School = 41)

Minimal Detectable Difference	Number of Schools Required for Each Study Condition
0.24	15
0.22	17
0.20	20
0.19	21
0.18	26
0.16	33

Note: Calculations assume = .05 and a two-tailed test.

Note: Calculations are based on ICC = 0.019

As in Exhibit 10, statistical power could be improved even further by including covariates in the models such as baseline data. In summary, Exhibits 10 and 11 demonstrate that a total of 40 schools are required to detect the target minimal detectable effects.

3. Methods to Maximize Response Rates

Different issues concern response rates for student surveys versus other types of surveys, and our approaches vary accordingly. For the student survey, response rates at baseline will be largely determined by three factors: parental consent, student absence at survey administration, and student willingness/ability to complete the questionnaire. Active parental consent is required, and we will work closely with each school to ensure that parents who do not return the consent form are sent reminder notices with replacement forms. We will provide a $25 incentive to classroom teachers or the school survey coordinator to encourage high rates of return for the parental consent forms needed for the student survey. Follow-up procedures will be repeated, as needed, to achieve the target active parental consent of 80%. Regarding student absence, we will track attendance at survey administration for all students for whom we have parental consent, keeping a list of absentees. We will schedule one make-up session in each school for students who missed the regular administration. Regarding students being able and willing to complete the questionnaire, we will pilot-test the questionnaire with nine students to determine that they are able to complete the questionnaire in 45 minutes or less and that the items are not overly difficult and do not make respondents uncomfortable. During survey administration, we will bolster students’ belief in the confidentiality of the survey by using two-part identification labels with peel-away portions that leave only a bar-coded label with no identifiers on the survey. The surveys will be administered by trained RTI staff members and we will ask teachers to leave the classroom during administration. Upon completing the survey, students will seal their response sheets in a large envelope.

At the follow-up student survey administrations, participation will also be affected by students transferring out of study schools. We will trace students and attempt to administer the survey in their new locations, as long as the student remains within the school district.

For participants other than students, we will facilitate participation in the interviews by being flexible in scheduling interviews. For the teacher survey, we will monitor data collection returns and contact respondents who have not yet returned their completed questionnaire to encourage their participations. Teachers who complete the teacher survey will be given a small gift card valued at $30.

So far, parental consent rates have ranged from 55% to 90% while student response rates have ranged from 86% to 99%. Teacher response rates have ranged from 89% to 97%.

4. Tests of Procedures and Methods to be Undertaken

We will pilot test the student survey with fewer than nine students in grades 6, 7, and 8. We will be certain to include 6^th graders because they are relatively more likely to be challenged by completing the survey and will therefore provide the most diagnostic information. We will recruit students through personal contacts and, if necessary, through middle schools in the vicinity of RTI offices in Research Triangle Park, NC. We will provide pilot test participants a small incentive for participating. We will instruct participants that their participation is voluntary and they may skip any questions that they do not choose to complete. We will also assure them that their responses are anonymous and confidential and will instruct them not to put their names on the survey.

We will ask pilot test participants to complete the survey and will monitor the time required. We will ask them to flag items that are unclear or difficult, to facilitate later discussion. When all participants have completed the survey we will lead a group discussion of their reactions to the survey, with a particular focus on those items that they found unclear or difficult. As appropriate we will ask their input on improving the items. We will also invite participants to provide written comments if they like.

5. Individuals Consulted on Statistical Aspects and Individuals Analyzing Data

Name, Degree	Title	Telephone
Mansour Fahimi, Ph.D.	Senior Research Statistician	301-230-4675
Jonathan L. Blitstein, Ph.D.	Health Research Methodologist	919-541-7313
Jason Williams, Ph.D.	Health Research Analyst	919-541-6734

C. REFERENCES

Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook, 2006-07 Edition, Teachers—Preschool, Kindergarten, Elementary, Middle, and Secondary, on the Internet at http://www.bls.gov/oco/ocos069.htm (visited February 14, 2006).

Cheong, J., MacKinnon, D.P., & Khoo, S.-T. (2003). Investigation of mediational processes using parallel process latent growth curve modeling. Structural Equation Modeling, 10(2), 238-262.

Farrell, A. D., Meyer, A. L., & Sullivan, T.N., & Kung, E.M. (2003). Evaluation of the Responding in Peaceful and Positive Ways (RIPP) Seventh Grade Violence Prevention Curriculum. Journal of Child and Family Studies, Vol. 12, No. 1, 101-120.

Farrell, A. D., Valois, R. F., Meyer, A. L., & Tidwell, R. P. (2003). Impact of the RIPP violence prevention program on rural middle school students. The Journal of Primary Prevention , 24(2), 143-167.

Farrell, A. D., Meyer, A. L., & White, K.S. (2001). Evaluation of Responding in Peaceful and Positive Ways (RIPP): A School-Based Prevention Program fro Reducing Violence Among Urban Adolescents. Journal of Clinical Child Psychology, Vol. 30, No. 4, 451-463.

Farrell, A. D., Valois, R. F., & Meyer, A. L. (2002). Evaluation of the RIPP-6 Violence Prevention Program at a Rural Middle School. American Journal of Health Education , 33(3), 167-172.

Gold, M.R., J.E. Siegel, L.B. Russell, M.C. Weinstein (Eds). (1996). Cost-Effectiveness in Health and Medicine. New York, NY: Oxford University Press.

Indicators of School Crime and Safety (2003). National Center for Education Statistics; Bureau of Justice Statistics, Department of Justice. “School related” is defined as occurring at, or on the way to or from school.

Janega, J. B., Murray, D. M., Varnell, S. P., Blitstein, J. L., Birnbaum, A. S., & Lytle, L. A. (2004) Assessing the most powerful analysis method for schools intervention studies with alcohol, tobacco, and other drug outcomes. Addictive Behaviors, 29(3), 595-606.

Murray, D. M., (1998).Design and Analysis of Group Randomized Trials. New York: Oxford University Press.

Polsky, D., H.A. Glick, R. Willke, and S. Dunmeyer. (1997). Confidence intervals for cost-effectiveness ratios: A comparison of four methods. Health Economics 6(3): 243-252.

Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin Company.

Snedecor, G. W., & Cochran, W. M. (1989). Statistical Methods (8th ed.). Ames, IA: The Iowa State University Press.

Varnell, S., Murray, D. M., Janega, J. B., & Blitstein, J. L. (2004). Design and analysis of group-randomized trials: A review of recent practices. American Journal of Public Health, 94(3), 393-399.

Zarkin, G.A., L.J. Dunlap, and G. Homsi. (In Press). The substance abuse services cost analysis program (SASCAP): a new method for estimating drug treatment services costs. Evaluation and Program Planning.

Exhibit 12 –

Statute Authoring the Evaluation

SEC. 4121. FEDERAL ACTIVITIES.

(a) PROGRAM AUTHORIZED- From funds made available to carry out this subpart under section 4003(2), the Secretary, in consultation with the Secretary of Health and Human Services, the Director of the Office of National Drug Control Policy, and the Attorney General, shall carry out programs to prevent the illegal use of drugs and violence among, and promote safety and discipline for, students. The Secretary shall carry out such programs directly, or through grants, contracts, or cooperative agreements with public and private entities and individuals, or through agreements with other Federal agencies, and shall coordinate such programs with other appropriate Federal activities. Such programs may include —

(1) the development and demonstration of innovative strategies for the training of school personnel, parents, and members of the community for drug and violence prevention activities based on State and local needs;

(2) the development, demonstration, scientifically based evaluation, and dissemination of innovative and high quality drug and violence prevention programs and activities, based on State and local needs, which may include —

(A) alternative education models, either established within a school or separate and apart from an existing school, that are designed to promote drug and violence prevention, reduce disruptive behavior, reduce the need for repeat suspensions and expulsions, enable students to meet challenging State academic standards, and enable students to return to the regular classroom as soon as possible;

(B) community service and service-learning projects, designed to rebuild safe and healthy neighborhoods and increase students' sense of individual responsibility;

(C) video-based projects developed by noncommercial telecommunications entities that provide young people with models for conflict resolution and responsible decisionmaking; and

(D) child abuse education and prevention programs for elementary and secondary students;

(3) the provision of information on drug abuse education and prevention to the Secretary of Health and Human Services for dissemination;

(4) the provision of information on violence prevention and education and school safety to the Department of Justice for dissemination;

(5) technical assistance to chief executive officers, State agencies, local educational agencies, and other recipients of funding under this part to build capacity to develop and implement high-quality, effective drug and violence prevention programs consistent with the principles of effectiveness in section 4115(a);

(6) assistance to school systems that have particularly severe drug and violence problems, including hiring drug prevention and school safety coordinators, or assistance to support appropriate response efforts to crisis situations;

(7) the development of education and training programs, curricula, instructional materials, and professional training and development for preventing and reducing the incidence of crimes and conflicts motivated by hate in localities most directly affected by hate crimes;

(8) activities in communities designated as empowerment zones or enterprise communities that will connect schools to community-wide efforts to reduce drug and violence problems; and

(9) other activities in accordance with the purpose of this part, based on State and local needs.

(b) PEER REVIEW- The Secretary shall use a peer review process in reviewing applications for funds under this section.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Paperwork Reduction Act Submission
Author	Authorised User
File Modified	0000-00-00
File Created	2021-02-03

Att_1850-0814 v.2 Part B SS revised 4-15-09