An Evaluation of the SOS Program:
Final Report of Findings from the 2001-2002 School Year
Prepared by
Robert H. Aseltine, Jr., PhD
Associate Professor
University of Connecticut Health Center
August 22, 2002
Executive Summary
This report presents process and outcome data from an evaluation of the safety, efficacy, and feasibility of implementing the Signs Of Suicide prevention program using data collected from 376 schools participating in the program during the 2001-2002 school year. Information about the program was obtained from individual site coordinators, typically guidance counselors, psychologists, social workers and nurses, who completed structured questionnaires immediately following implementation of the program, 30 days after implementation, and again 3 months after implementation. In general, the program and its materials were very well received. Site coordinators rated the core components of the program very favorably, and gave particularly positive ratings to the “Friends for Life” video. In terms of program efficacy, the vast majority of site coordinators reported that the program was effective in increasing help-seeking, in improving communication among students, parents, and teachers, and bringing students in need of help to the school’s attention. Of particular importance is the nearly 150 percent increase in help-seeking among students who participated in the program: site coordinators reported that the number of students seeking counseling for depression or suicidal ideation increased from an average of 3.9 per month over the past year to 9.6 in the 30 days following the program’s implementation. These increases in help-seeking were maintained 3 months following implementation of the program. Smaller but statistically significant increases (+68%) in the number of students seeking help on behalf of emotionally troubled friends were also observed. In addition, the program appears to have an excellent safety profile, as the vast majority of high schools reported no adverse reactions among students exposed to the SOS program.
Suicide among young people is one of the most serious public health problems facing the United States. According to the National Center for Health Statistics, the suicide rate for youth and young adults aged 15-24 has tripled since 1950, and suicide is now the third leading cause of death in this age group.1 Although it is difficult to obtain reliable estimates because of the accompanying stigma, there are estimates that the incidence rate of suicide attempts among adolescents may be as high as 10% over a 6 month period. 2,3
To address this problem, Screening for Mental Health, Inc., a non-profit organization in Wellesley MA, developed SOS: Signs of Suicide, a school-based suicide prevention targeting high school students. SOS educates students to understand that suicide is directly related to mental illness, typically depression, and that it is not a normal reaction to stress or emotional upset. 4-8 This approach stands in direct contrast to other programs that seek to destigmatize and therefore normalize suicide by separating it from mental illness. Promoting the understanding that suicide is a feature of mental illness and is in fact a part of the diagnostic criteria for major depressive disorder is a crucial component of the SOS program.
The basic message of the program is to teach high school students to respond to the signs of suicide as a mental health emergency, much as one would react to a heart attack as a health emergency. The program focuses on teaching youths to recognize the signs of suicide and depression in themselves and others and the specific action steps needed to respond to those signs. The goal is to make the action step -- ACT -- as instinctual a response as the Heimlich maneuver and as familiar an acronym as “CPR.” ACT stands for Acknowledge, Care, and Tell. First, ACKNOWLEDGE the signs of suicide that others display and take them seriously. Next, let that person know you CARE about him or her and that you want to help. Then, TELL a responsible adult.
The SOS program’s main teaching materials consist of a video and a discussion guide. The video includes dramatizations depicting the right and wrong ways to react to someone who is depressed and suicidal as well as interviews with real people whose lives have been touched by suicide. Following the video students are asked to complete the Columbia Depression Scale, a brief screening instrument for depression. 9 The screening form is scored by the students themselves; a score of 16+ on the CDS is considered a strong indicator of depressive disorder and such students are encouraged to seek help immediately. There are also educational flyers and posters, and an additional screening form that is sent home to help parents determine whether their child is exhibiting symptoms of depression and suicidal ideation. Schools participating in the program receive a kit of materials containing the video, discussion guide, screening forms and other educational and promotional items. They also receive the Procedure Manual that describes methods of implementing the program and discusses some of the issues involved (i.e., parental notification, anonymous versus identified screening, and referrals).
This report presents process and outcome data from an evaluation of the SOS program during the 2001-2002 school year. Specifically, this report attempts to address four basic questions: 1) how was the program presented to students, i.e., in what settings, under what conditions, etc.; 2) how did teachers rate the quality and effectiveness of various elements of the program, and what were their perceptions of students’ and teachers’ reactions to it; 3) did the program have any adverse or negative effects on depressed or suicidal youths, and 4) was the program successful in encouraging help-seeking among depressed youth?
Thus, the estimated response rates for the three components of the evaluation fall in an acceptable range and lend confidence that the data presented below are broadly representative of experiences with the SOS program during the 2001-2002 year.
Demographic Information on Program Schools
Table 1 presents demographic information on the 1006 schools registering for the program in 2001-2002. These schools are overwhelmingly public schools (85%) containing approximately 925 students on average. The majority of students in these schools are White (64%), and program schools are slightly more likely to be located in rural and suburban as opposed to urban areas. Approximately one third of the students in participating schools qualify for free or reduced school lunches. The demographic characteristics of the subset of schools from which we have Postscreen Surveys are similar to those presented in Table 1. Moreover, the racial demographic profile of these schools is very similar to national figures obtained from the U.S. Census and other data sources.
Table 1 about here
Table 2 presents process data on the manner in which the SOS program was implemented in the 376 participating schools. As this table indicates, the presentation of the Friends for Life: Preventing Teen Suicide video and the implementation of the depression screenings were very similar. Approximately twenty percent of the participating schools presented the video and conducted screenings with the entire student body. This number is lower than we observed last year (when approximately 30% presented the program to the entire student body) due mainly to the fact that a number of schools were now conducting this for the second year and had fewer unexposed students to treat. Among those schools offering the program in select classes, the vast majority conducted the program in health (75%) classes. Schools were more likely to involve freshmen and sophomores as opposed to juniors and seniors in the program, again partly due to the implementation of the program in the previous year among some schools.
Table 2 about here
In terms of the settings in which the video presentation and screenings were conducted, these data indicate that the classroom is the preferred venue. The locations aggregated in the “other” category typically included the school library, the student lounge, and the cafeteria.
The program required relatively little time to implement. The average amount of time to implement across the 376 schools with completed summary forms was approximately 2.5 days, but almost 40% of schools reported that they completed the program in one day. Although most schools did not have any difficulties in implementation, some did report some relatively minor issues. For the most part these involved problems surrounding the coordination of the program given staffing levels in the school. A couple of program coordinators did encounter some resistance from administrators, some of which was tied to policies governing the use of mental health screening instruments. (Additional data relevant to these issues are presented in Tables 3 and 5 and will be discussed below).
Ratings of the SOS Program
Data from Summary Forms indicate that the SOS program made a very favorable impression on school personnel. Site coordinators were asked to rate each component of the program on a scale of 1 to 4, where 4 represents “very effective” and 1 “not at all effective.” Ratings of the various components of the SOS program are presented in Table 3 (top panel). These ratings are similar to last year’s ratings of the program, but are slightly more positive. The average ratings of all program components range between 3 (somewhat effective) and 4 (very effective), with the “Friends for life: Preventing Teen Suicide” video (M = 3.74), the accompanying educational materials (M = 3.63), and the Procedure Manual (M = 3.62) garnering the highest ratings. The enthusiasm of school personnel for the video is clearly seen in the last column of this table, which presents the proportion of the sample reporting that a particular element of the program was “very or somewhat effective.” Ninety eight percent of the sample rated the video as very “very or somewhat effective.” Although the average ratings for all the elements of the program were acceptable, both the student and parent depression screening forms and the accompanying scoring and interpretation sheets tended to garner the lowest overall ratings among all the elements of the program, with an average in the “somewhat effective” range. Information collected in the open-ended questions on the Summary Form indicate that a few of the schools felt that the screening instrument was not an accurate indicator of depression (e.g., would yield too many false positives) and that the wording of questions was somewhat confusing. In response to these criticisms, Screening for Mental Health has adopted a new screening instrument for the 2002-2003 program. This instrument, the Brief Screen for Adolescent Depression, is based on the latest version of the widely used Diagnostic Interview Schedule for Children.
Table 3 about here
The Summary Form also contained a series of more specific questions which asked the site coordinators to rate various aspects of the Friends for Life: Preventing Teen Suicide video on a scale of 1 to 4, where 4 represents “excellent” and 1 “poor.” These data are summarized in the second panel of Table 3 and were quite favorable. Site coordinators gave their highest ratings to the “real life stories about depression and suicide,” (M=3.76) as virtually all site coordinators described this aspect of the video presentation as “excellent” or “good.” Somewhat lower though clearly positive ratings were given to the dramatizations presented in the video (M=3.29), which 85% of the sample described as “excellent” or “good.” Overall, 96% of the sample rated the overall quality of the video to be “excellent” or “good” (M=3.56).
The bottom panel in Table 3 presents site coordinators’ perceptions of the reactions of teachers, students, and parents to the program. These perceptions ranged from “very positive” (5) to “very negative” (1) along a five-point scale. The most positive reactions to the program were observed among teachers and school personnel (M = 4.46), with 90% percent reporting that the responses of their school colleagues were “very” or “somewhat positive.” The average rating for students’ responses was also in the “very” to “somewhat positive” range (M = 4.19), while parents were perceived to be between “somewhat positive” and “neither positive or negative” (M = 3.73). Thirty-eight percent of parents were described as “neither positive or negative,” although this should not be interpreted as parental ambivalence toward the program. Several site coordinators explained anecdotally that they were not able to characterize parents’ reactions as either positive or negative because they received virtually no feedback from parents concerning the program.
Two of the central messages of the SOS video relate to the importance of seeking help when experiencing the emotional and behavioral signs of suicide, and of intervening on behalf of troubled friends. In these preliminary data the principal measure of efficacy is school personnel’s reports of help-seeking behavior as defined by the number of youths visiting the school counselor’s or nurse’s office to talk about their own feelings of depression/suicidality or problems they perceive in their friends. To assess changes in help-seeking behavior that can be attributed to the SOS program, reports of the numbers of youths seeking help in the 30 days following the program can be compared to reports of the average numbers of students seeking help per month over the past year, both of which were obtained in the Postscreen Survey. These data are presented in Table 4 and are very favorable. The average number of visits per month to the school counselor’s or nurse’s office among youth experiencing emotional problems was 3.91 for the previous year, but increased to 9.59 in the 30 days following the program. This represents an increase in help seeking of nearly 150% that can plausibly be attributed to the SOS program. The 95% confidence intervals presented at the bottom of Table 4 indicate that the increase in help seeking following exposure to the program is statistically significant.
Table 4 about here
Fewer students (approximately 2.25 per month on average) seek help on behalf of a troubled friend, yet there is also an increase in this behavior following the program. The bottom row of this table indicates a 70% increase in the number of visits on behalf of friends following the program as compared to the year prior, an increase that achieves statistical significance at the .05 level. Of those who visit the school counselor’s/nurse’s office for counseling, roughly three quarters are female, and for those seeking help for themselves approximately one fifth are accompanied by a friend.
Evaluating Costs and Benefits of the Program
In contrast to our analysis of help-seeking behavior before and after implementation, this portion of the analysis focuses on more subjective indicators of program efficacy. In addition, we are interested in determining whether there were any adverse experiences associated with the program. Two potential “costs” that have been raised in conjunction with school-based prevention programs in general and suicide prevention programs in particular involve the burden of implementation and the safety of the program. Regarding the latter, there are some concerns that programs raising the difficult issues surrounding depression and suicide may have adverse effects on emotionally troubled youths, ultimately increasing the likelihood of self-destructive behavior.
To obtain subjective indicators of program efficacy, site coordinators were asked in the Postscreen Survey how true it was that the SOS program increased help-seeking behavior, improved communication about depression and suicide among students, parents, and teachers, and brought students in need to the school’s attention. Responses ranged from “very true” (4) to “not at all true” (1) on a four-point scale. Results are presented in the top panel of Table 5. Very similar results were obtained across the various indicators of efficacy, as the mean responses to all of these questions were in the 2.9 - 3.6 range (i.e., the modal response category ranged between “somewhat true” to “very true”). The right hand column in this table indicates that the vast majority of site coordinators felt that it was “somewhat” or “very true” that the SOS program increased help-seeking, improved communication among teachers, students and their parents, and brought students in need to the school’s attention.
Table 5 about here
Regarding any costs or negative experiences associated with the program, site coordinators were also asked whether the program created difficulties because of the large numbers of students seeking help, whether the program took too much time to implement, and whether school administrators and teachers were supportive of the program. Although a few schools did report some difficulties handling the demands placed on their counseling staffs by students identified in the screening as at risk, the vast majority of schools had no such problems. Only 11% of site coordinators said that it was “somewhat true” that the program created difficulties because of student help-seeking, and only 4% said it was “very true.” Few also felt that the program took too much time to implement (only 3% said this was “very true”), and resistance from school administrators was encountered at a small fraction of schools (only 3% said this was “very true”).
In terms of the safety of the program, our results are even more definitive. The average level of agreement with the statement that “the program had an adverse effect on depressed or suicidal youth” is “not at all true” (M = 1.21). While two site coordinators described that statement as “very” or “somewhat true,” follow up interviews indicated that they were reporting as “adverse” effects the fact that program exposed a lack of internal resources in the school to deal with the students who were depressed. Both schools contacts, however, were extremely positive about the program itself and its ability to help students in need.
Finally, site coordinators were given the opportunity in the Postscreen Survey to provide an overall rating of the helpfulness of the SOS program for their students. These data are presented at the bottom of Table 5 and are quite favorable: 92% of site coordinators described the program as “very” or “somewhat helpful.”
To ascertain the longer term impact of the SOS program, schools participating in the 2001-2002 program were also asked to complete an abbreviated Postscreen Survey 3 months following implementation. Results from the 3 month Postscreen Survey, presented in Table 6, clearly indicate that the positive effects of the program observed 1 month following implementation were maintained over time. Comparing the results in Panel A of Table 6 with those presented in Table 5, we see that site coordinators give virtually identical ratings in the 3 month follow up as they did 1 month following implementation on ratings of the program’s impact on help-seeking, improved communication, bringing students in need to the school’s attention, and in overall helpfulness. Similarly, adverse events associated with the program were virtually nonexistent. The average level of agreement with the statement that “the program had an adverse effect on depressed or suicidal youth” is “not at all true” (M = 1.30), and fewer than 2% of site coordinators described this statement as somewhat or very true.
Table 6 about here
Finally, site coordinators reports of the numbers of students seeking help for emotional problems 3 months following exposure to the program are presented in the Panel B of Table 6. The average number of youths per month who visited the school counselor’s or nurse’s office to talk about depression/suicidality 3 months following the program was 9.74, which is virtually identical to that observed 30 days following the program (see Table 4). The number of students seeking help on behalf of a troubled friend 3 months following the program is somewhat lower than that observed 30 days following implementation, yet still represents a 25% increase over the average per month over the past year. The 95% confidence intervals presented in the right hand column of this table indicate that the number of students seeking counseling for depression/suicidality at 3 months is significantly higher than the average over the past year.
Also, of those who visit the school counselor’s/nurse’s office for counseling 3 months following the program, roughly three quarters are female, and for those seeking help for themselves approximately one quarter are accompanied by a friend (data not shown). These numbers are virtually identical to those observed in the month following implementation of the program.
In the final section of this report we present a profile of the mental health of students participating in the SOS: Signs of Suicide prevention program during the 2001-2002 school year. The data presented below were obtained from the National Depression Screening Day questionnaires completed by students as part of the SOS program. The report is based on a total of 35,651 students in 225 high schools who returned screening forms by March 1, 2002.
Table 7 about here
The overall prevalence of depression and suicidal behavior is presented in Table 7. Scores on the Columbia Depression Scale indicate that 14.8% of students in schools participating in this program have been depressed during the past year. Almost 17% report that there has been a time during the past year “…when you thought seriously about killing yourself.” The past year prevalence of suicide attempts is 5%, while the lifetime prevalence of suicide attempts is 10.7%. As expected, the association between the items in the CDS measuring depressed mood and those assessing suicidal behavior is striking (data not shown), as levels of depressed mood are almost twice as high among those with a history of suicidal ideation and suicidal behavior. These data support the SOS program’s emphasis on mental illness in general and depression in particular as appropriate targets in the prevention of adolescent suicide.
Demographic differences in depression and suicidal thoughts and behavior are also presented in this table. There are moderate differences in levels of depression and suicidality by grade, with older youths slightly more likely to experience depression. The association between grade and suicidal ideation and behavior appears to be nonlinear however, with rates generally peaking in grades 10 & 11 before falling slightly by grade 12. Differences in depression and suicidality by race and ethnicity are more pronounced. The lowest levels of depression and suicidal behavior are observed among Blacks, Asians, and Whites, while the highest levels are observed among American Indians, those with a multiracial background, and those in the ‘other’ race category. Rates of depression, suicidal ideation, and suicidal behavior among American Indian students are particularly striking. More than one fifth of students in this racial/ethnic category have been depressed during the past year and almost a quarter report a lifetime suicide attempt, with 14 percent reporting an attempt during the past year. Finally, sex differences in depression and suicidal behavior observed in these data mirror trends reported nationally. Girls are much more likely to be depressed than are boys (19% versus 10%) and are more likely to report a prior suicide attempt (14.4% versus 6.7% lifetime prevalence).
Recent literature has identified alcohol use as a major risk factor for suicidal behavior among teens.10,11 To examine this association, the screening form also included 2 questions on drinking behavior. Students were asked whether there had been a time in the past year when they “had 5 or more alcoholic drinks in a row” (yes/no), and whether they had “used alcohol when [you] were feeling down” (yes/no). The crosstabulations presented in Table 8 indicate that episodes of binge drinking (5+ drinks in a row) and use of alcohol when feeling down are both strongly associated with suicidal behavior. In fact, the strength of this association is startling: those reporting that they have used alcohol when feeling down are almost 6 time more likely to report a suicide attempt during the past year and over 4 times more likely to report a lifetime attempt than are those who have not used alcohol when feeling down. Similarly, those reporting an episode of bingeing in the past 12 months are almost 3 times more likely to have attempted suicide in the past year and 4.5 times more likely to have ever attempted suicide than those who have not had an episode of binge drinking.
Table 8 about here
The second year of the SOS Suicide Prevention Program has been quite successful. The program experienced phenomenal growth, with over 1000 high schools expressing interest in the program and (as estimated in the Methods section) approximately 600 of these fully implementing the program during the 2001-2002 school year. Many of the schools that had implemented the program in its first year continued to do so, with a number of these reporting that they had integrated the program into their standard health curriculum. In addition, the favorable reactions to the program that were observed in its first year were maintained despite this rapid growth. Overall ratings of the program are excellent, and key components of the program, such as the “Friends for Life: Preventing Teen Suicide” video, have been received enthusiastically. Site coordinators report that teachers, students and parents have responded positively and report few difficulties with regards to implementation. Most importantly, these results indicate that the program is efficacious – that is, that it actually increases help-seeking behavior and brings students in need of help to the school’s attention. The number of students who sought counseling for depression and/or suicidal ideation is indeed impressive, more than doubling in the 30 days following implementation of the program relative to the monthly average over the past year. Significant increases in the number of students seeking counseling on behalf of a depressed friend in the 30 days following the program were also observed. Moreover, the increases in help-seeking among depressed and suicidal youth did not appear to dissipate in the 3 months following exposure to the program.
Despite the relative increase in the numbers of students seeking counseling, schools reported little undue burden on their staffs as a result of the program. This point cannot be overemphasized, as it indicates that the mental health screening component of the SOS program is not overburdening school resources and draw mental health services away from truly needy students. Furthermore, these data highlight another of the attractive features of the SOS program: the program incorporates mental health screening as one of its central components, yet it can be implemented by health educators on a school-wide basis with relative ease. Other suicide prevention programs that include a mental health screening component can require a great deal of time and resources to implement and are therefore restricted to small groups of students.12
References
Moscicki E. K. (1999). Epidemiology of suicide. In: Jacobs D. G. (ed.). Guide to Suicide Assessment and Intervention (pp. 40-51). San Francisco: San Francisco, 1999.
Joffe, R.T., Offord, D. R. & Boyle, M. H. (1988). Ontario child health study: suicidal behavior in youth age 12-16 years. American Journal of Psychiatry 145(11):1420-1423.
Kann L, Kinchen SA, Williams BI, Ross JG, Lowry R, Grunbaum JA, Kolbe, LJ. Center for Disease Control and Prevention. Youth risk behavioral surveillance—United States. In: CDC Surveillance Summaries. June 9, 2000;49:No. 22-5.
Jacobs, D. G., Brewer, M., & Klein-Benheim, M. (1999). Suicide assessment: an overview and recommended protocol. In: Jacobs DG (ed.). Guide to Suicide Assessment and Intervention (pp. 3-39). San Francisco: Jossey-Bass.
Brent DA, Kolko DJ. The assessment and treatment of children and adolescents at risk for suicide. In: Blumenthal SJ, Kupfer DJ, eds. Suicide Over the Life Cycle: Risk Factors, Assessment, and Treatment of Suicidal Patients. Washington, DC: American Psychiatric Press; 1990:253-302.
Lewinson PM, Rohde P, Seeley JR. Psychosocial risk factors for future adolescent suicide attempts. Journal of Consulting and Clinical Psychology. 1994; 62:297-305.
Andrews JA, Lewinsohn PM. Suicidal attempts among older adolescents: prevalence and co-occurrence with psychiatric disorders. Journal of Consulting and Clinical Psychology. 1992; 4:655-662.
Velez CN, Cohen P. Suicidal behavior and ideation in a community sample of children: maternal and youth reports. J Am Acad Child Adolesc Psychiatry. 1988; 27:349-356.
Shaffer, D., Garland, A., Vieland, V., Underwood, M., & Busner, C. (1991). The impact of curriculum-based suicide prevention programs for teenagers. Journal of the American Academy of Child and Adolescent Psychiatry 30(4):588-596.
Brent, D., Perper, J., Moritz, G., et al. (1993). Psychiatric risk factors for adolescent suicide: A case-control study. Journal of the American Academy of Child and Adolescent Psychiatry, 32(3):521-529.
Andrews, J., & Lewinsohn, P. (1992). Suicidal Attempts among older adolescents: Prevalence and co-occurrence with psychiatric disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 31(4):655-662.
Shaffer D., & Craft, L. (1999). Methods of adolescent suicide prevention. Journal of Clinical Psychiatry, 60(supp. 2), 70-4.
| File Type | application/msword |
| Author | Robert Aseltine |
| Last Modified By | LBUCHANAN |
| File Modified | 2011-02-02 |
| File Created | 2011-02-02 |