OMB CARES 2nd Package_Appendix B - F

LIST OF APPENDICES (B – F)

Appendix B: Exhibits for Supporting Statement

A2-1: Data Collection Timeline for Instruments in OMB Package

B1-1: Average Minimum Detectable Effects for Priority Student

Outcomes

B1-2: Average Minimum Detectable Effects for Priority Class

and Teacher Outcomes

B1-3: Estimated Sample Sizes for 4-Year Olds

B1-4: Average Minimum Detectable Effects for Priority Student

Outcomes (3 year olds)

B1-5: Estimated Sample Sizes for 3-Year Olds

B1-6: Survey/Assessment/Interview Sample Sizes

Appendix C.1: Supporting Documentation for Baseline Lead Teacher Self-Report

Survey

Appendix C.2: Supporting Documentation for Follow-up Lead Teacher Self-Report

Survey

Appendix C.3: Supporting Documentation for Teacher Report on Individual

Children

Appendix C.4: Supporting Documentation for Baseline Parent Survey

Appendix C.5: Supporting Documentation for Follow-up Parent Survey

Appendix C.6: Supporting Documentation for Coach Interview Guide

Appendix C.7: Supporting Documentation for Teacher Interview Guide

Appendix C.8: Supporting Documentation for Center Director Interview Guide

Appendix C.9: Supporting Documentation for Center Staff Interview Guide

Appendix C.10: Supporting Documentation for Grantee/Delegate Agency Director

Interview Guide

Appendix C.11: Supporting Documentation for Trainer Interview Guide

Appendix C.12: Supporting Documentation for Direct Child Assessments

Appendix D.1: Head Start CARES Teacher Consent Form

Appendix D.2: Head Start CARES Parent Consent Form

Appendix E: References

Appendix F: Federal Register Comment & Response

Exhibit B1-6

Survey/Assessment/Interview Sample Sizes

Survey Efforts/Sites	Core Study	3-year old add-on	Total
Baseline Lead Teacher Self-Report Survey	360	no additional sample	360
Follow-up Lead Teacher Self-Report Survey	360	no additional sample	360
Teacher Report on Individual Children	2,880	768	3,648
Baseline Parent Survey	2,880	768	3,648
Follow-up Parent Survey	2,880	768	3,648
Direct Child Assessment	2,880	n/a	2,880
Site Visit: Coach Interview Guide	60	no additional sample	60
Site Visit: Teacher Interview Guide	360	no additional sample	360
Site Visit: Center Director Interview Guide	60	no additional sample	60
Site Visit: Center Staff Interview Guide	180	no additional sample	180
Site Visit: Grantee/Delegate Agency Director Interview Guide	20	no additional sample	20
Trainer Interview	60	no additional sample	60

QUESTION-BY-QUESTION JUSTIFICATION OF BASELINE TEACHER SELF-REPORT SURVEY

Question #	Constructs/Items	Justification
A1-A23	Teacher age, gender, DOB, ethnicity/race, country of origin, language(s), teaching experience, educational attainment, credentialing	This demographic information helps to describe the respondent population.
A24-A26	Teacher salary, hours per week, primary income earner status	This economic information helps to capture characteristics of teachers, as well as structural characteristics of the classroom.
A27-A31	Teacher marital status, household children	These teacher background characteristics will help to provide predictive information about home stressors that may moderate impacts.
Section B	Student enrollment, assignment of teachers, average number of absent and late children, number of students today, other teachers/teaching assistants	These questions address structural qualities of Head Start classrooms as moderators of the effects of the implemented program on children’s emotional and behavioral adjustment (Jones, Brown & Aber, 2008; Raver et al., under review). Structural features that will be assessed include class size, teacher-to-student ratio, and teacher turnover.
C1-C10	Emotion-Related Parenting Styles Self-Test (adapted for teachers) (Hakim-Larson, Parker, Lee, Goodwin, & Voelker, 2006)	In order to assess emotional socialization of teachers to see if there is a mismatch with parent report of the same items, the Emotion Coaching and Dismiss/Disapprove subscales from the shortened Emotion-Related Parenting Styles Self Test measure are included and the wording for a teacher self-report will be adapted.
D1-D22	Maslach Burnout Inventory – Educators Survey (Maslach, Jackson, & Leiter, 1996)	Recent results from two efficacy trials (Chicago School Readiness Project and 4Rs) suggest that a limited set of psychosocial characteristics (such as teachers’ own feelings of job overload) was significantly predictive of teachers’ reports of more emotional and behavioral difficulty for the children in their classroom. For assessment of teacher burnout, we propose using the Maslach Burnout Inventory- a 22-item instrument frequently used with teachers. The instrument measures three dimensions of teacher burnout: emotional exhaustion, depersonalization, and loss of personal accomplishment (Burke & Greenglass, 1995). We consider this an important predictor of teacher reported outcomes, as well as teacher implementation of the program models.
E1a-E1f	K-6 Kessler Psychological Distress Scale (Kessler, Andrews, Colpe, et al., 2002)	This measure will be used to measure teacher mental health. The K-6 is a six-item truncated scale embedded within a ten-item screening scale of psychological distress developed for the redesigned U.S. National Health Interview Survey. In clinical trials, the K-6 was favored for its brevity and high level of precision for discriminating DSM-IV cases from non-cases, and is thus being used in annual government surveys in the United Start and Canada, as well as in the WHO World Mental Health Surveys. The K-6 has been used extensively with ethnically diverse samples of low-income families (Kling, Leibman, & Katz, 2007). Teacher depression is likely to be affected by these intervention program models, given the strong association between problem behavior among children and teacher burnout and stress found in prior research.
F1-F20	Organizational Readiness for Change (Lehman, Greener & Simpson, 2002)	Teacher personality traits such as openness to change (Durlak & DuPre, 2008; Han & Weiss, 2005; Flaspohler et al., 2008) and agreeableness and extraversion (Lochman, in press) predict the nature and extent of teacher implementation. Included is the 20-item TCU Organizational Readiness for Change (ORC) measure includes four subscales: Adaptability, Cohesion, Autonomy, and Change.
Section G	Views on social emotional development	Literature review by Han & Weiss (2005) shows that the perceived need for an intervention--including value placed on outcomes sought--predicts intervention fidelity. Included are items developed by the CARES research team.
Section H	Past training and professional development	The implementation study needs to examine the impact of CARES controlling for other social emotional-related professional development CARES teachers may have received. Program model must be different enough from practice as usual to benefit "end-users" (e.g., children) (Joyce & Showers, 2002). There also needs to be an understanding of what the “service contrast” is with the control groups. Included are items developed by the CARES research team.
I1-I2	Adapted Wehby Teacher-Consultant Alliance Scale (Domitrovich, Bradshaw, & Poduska, 2008)	Research suggests that program implementation might be strongest in settings where teachers and their assistants have a good functional and interpersonal relationship. These items are selected and adapted from the Wehby Teacher-Consultant Alliance Scale, adapted to reference the teacher-teaching assistant relationship.

QUESTION-BY-QUESTION JUSTIFICATION OF FOLLOW-UP TEACHER SELF-REPORT SURVEY

Question #	Constructs/Items	Justification
Section A	Teacher name, DOB	This demographic information will be used for matching purposes only.
Section B	Student enrollment, assignment of teachers, average number of absent and late children, number of students today, other teachers/teaching assistants	These questions address structural qualities of Head Start classrooms as moderators of the effects of the implemented program on children’s emotional and behavioral adjustment (Jones, Brown & Aber, 2008; Raver et al., under review). Structural features that will be assessed include class size, teacher-to-student ratio, and teacher turnover.
C1-C10	Emotion-Related Parenting Styles Self-Test (adapted for teachers) (Hakim-Larson, Parker, Lee, Goodwin, & Voelker, 2006)	In order to assess emotional socialization of teachers, included is the Emotion Coaching and Dismiss/Disapprove subscales from the shortened Emotion-Related Parenting Styles Self Test measure and adapting the wording for a teacher self-report. We will collect this information at follow-up from teachers, as it may be related these program models.
D1-D22	Maslach Burnout Inventory – Educators Survey (Maslach, Jackson, & Leiter, 1996)	Recent results from two efficacy trials (Chicago School Readiness Project and 4Rs) suggest that a limited set of psychosocial characteristics (such as teachers’ own feelings of job overload) was significantly predictive of teachers’ reports of more emotional and behavioral difficulty for the children in their classroom. For assessment of teacher burnout, included is the Maslach Burnout Inventory- a 22-item instrument frequently used with teachers. The instrument measures three dimensions of teacher burnout: emotional exhaustion, depersonalization, and loss of personal accomplishment (Burke & Greenglass, 1995).
E1a-E1f	K-6 Kessler Psychological Distress Scale (Kessler, Andrews, & Colpe, 2002)	This measure will be used to measure teacher mental health. The K-6 is a six-item truncated scale embedded within a ten-item screening scale of psychological distress developed for the redesigned U.S. National Health Interview Survey. In clinical trials, the K-6 was favored for its brevity and high level of precision for discriminating DSM-IV cases from non-cases, and is thus being used in annual government surveys in the United Start and Canada, as well as in the WHO World Mental Health Surveys. The K-6 has been used extensively with ethnically diverse samples of low-income families (Kling, Leibman & Katz, 2007). Teacher depression is likely to be affected by these intervention program models, given the strong association between problem behavior among children and teacher burnout and stress found in prior research.
Section F	Views on social emotional development	Literature review by Han & Weiss (2005) shows that the perceived need for an intervention--including value placed on outcomes sought--predicts intervention fidelity. Items were developed by the CARES research team.
Section G	Social emotional-related classroom practices	From an implementation perspective, it is hypothesized that program teachers who implement their assigned program model with greater fidelity will demonstrate more frequent and higher quality social-emotional teaching practices than program teachers who implement the program model with lower fidelity. These items are teachers' quantitative ratings of their social emotional-related classroom practices. Items were developed by the CARES research team.
H1-H5	Working with your coach	Coaching is more effective when coaches have positive relationships with their practitioner and are engaged in participatory planning (Joyce & Showers, 2002). Greater coach engagement, buy-in, confidence, and motivation predicts greater intervention fidelity (Durlak & DuPre, 2008; Fixsen et al., 2005; Flaspohler et al., 2008; Han & Weiss, 2005; Lochman et al, in press; Schoenwald & Hoagwood, 2001). Therefore, all program teachers will be asked to provide quantitative ratings of coaches' engagement in coaching, the teacher-coach relationship and quantitative ratings of coaches' fidelity behavior. Some items come from the PATHS to PAX study; others were adapted or developed by the CARES research team.
I1-I8	Perceptions of Program Model	"Learning" (Kirkpatrick, 1994), or an understanding core principles of the intervention (Durlak & DuPre, 2008; Flaspohler et al., 2008), is necessary if teachers are to implement the program model with fidelity. Research also shows that teachers’ views of treatment acceptability (e.g., program “fit” with their beliefs about how to support social emotional development in children, ease of implementation, its perceived usefulness, and satisfaction with the program implemented), and their confidence and motivation to implement program models in the classroom are key predictors of implementation fidelity (Flaspohler et al., 2008; Han & Weiss, 2005; Lochman, 2000; Schoenwald and Hoagwood, 2001). Therefore, all program teachers will be asked to provide quantitative ratings of buy-in, satisfaction, and confidence. Items were adapted or developed by the CARES research team.
J1-J6	Organizational Readiness for Change (Lehman, Greener, & Simpson, 2002)- cohesion subscale	To assess whether staff cohesion changed as a result of the program model, teachers will be asked to answer items from the 6-item "Cohesion" subscale of the Organizational Readiness for Change scale.
Section K	Past training and professional development	The implementation study needs to examine the impact of CARES understanding the other social emotional-related professional development CARES teachers may have received. Program model must be different enough from practice as usual to benefit "end-users" (e.g., children) (Joyce & Showers, 2002). There also needs to be an understanding of what the “service contrast” is with the control groups. Items were developed by the CARES research team.
L1-L6	Adapted Wehby Teacher-Consultant Alliance Scale (Domitrovich, Bradshaw, & Poduska, 2008)	To assess whether teacher and assistant teachers’ interpersonal relationships changed as a result of the intervention, items were selected and adapted from the Wehby Teacher-Consultant Alliance Scale, adapted to reference the teacher-teaching assistant relationship.
M1-M2	Supervisor Monitoring and Support	Research suggests that program implementation might be strongest in settings with sufficient support in the program model from leadership (Fixsen et al., 2005;Flaspohler et al., 2008; Han & Weiss, 2005; Lochman et al, in press), including efforts to assess teacher performance then provide regular, systematic feedback (Fixsen et al., 2005; Flaspohler et al., 2008; Han & Weiss, 2005; Schoenwald and Hoagwood, 2001). Therefore, program teachers will be asked items developed by the CARES research team, based on research by Durlak & DuPre (2008), Flaspohler et al. (2008), Glisson et al. (2008), Han & Weiss (2005).

QUESTION-BY-QUESTION JUSTIFICATION OF TEACHER REPORT ON INDIVIDUAL CHILDREN

Question #	Constructs/Items	Justification
A1-A39	Social Skills Rating Scale, SSRS- Social Skills Scale & Academic Competence Scale (FUP only), Teacher-Preschool version (Gresham & Elliott, 1990)	The Social Skills Rating Scale - Social Skills Scale (SSRS) will be valuable in helping us to understand whether the intervention affects children’s social problem-solving with peers. Subscales include Cooperation, Assertion, and Self-Control. We recommend using the SSRS because of its comprehensiveness in tapping children’s social skills (Merrell, Streeter, & Boelter, 2001), its superior psychometric properties (Merrell & Gimpel, 1998), and its widespread use by other preschool studies, such as FACES and the ECLS-K. The Social Skills Rating Scale – Academic Competence Scale (SSRS) will be administered to teachers at Kindergarten follow-up to assess children’s academic progress relative to other children in the classroom. This scale also has widespread use and has been either used or adapted in FACES and the Head Start Impact Study.
B1-B15	Student-Teacher Relationship Scale, STRS (Pianta, 2001)	Teachers’ perceptions of conflict, closeness, dependency, and overall quality of the relationship with individual children will be assessed with the Student-Teacher Relationship Scale. The STRS is a 28-item teacher-report instrument that uses a 5-point Likert-type rating scale to assess teacher’s perceptions of his or her relationship with a child, a child’s interactive behavior with the teacher, and a teacher’s beliefs about the student’s feelings towards the teacher. Teachers rate the 28 items in this scale in terms of how applicable each statement was to their relationship with a particular child. Three subscales comprise this measure: Conflict, Dependence, and Closeness. It has been used or adapted in the Head Start Impact Study, NICHD SECC, and ECLS-K.
C1-C32	Behavior Problems Index (Zill & Peterson, 1986)	The Behavior Problems Index (BPI) will be used to assess behavioral problems by teachers. The BPI measures the frequency, range, and type of childhood behavior problems for children age 4 and older and has been used extensively with three year olds as well (Zill & Peterson, 1986). Many items included in the BPI were drawn from the Child Behavioral Checklist (Achenbach, 1991) and other child behavior scales (Graham & Rutter, 1968). It consists of 32 items describing behavior problems. It is widely used for several reasons including its ability to measure a broad developmental range, its simplicity in reading and understanding, and its short length. This measure was used in FACES as well as number of efficacy trials.
D1-D37	Cooper-Farran Behavior Rating Scales, CFBRS (Cooper & Farran, 1991)	The Cooper-Farran Behavioral Rating Scales (CFBRS) is an instrument designed to give educational practitioners and researchers a reliable, valid, quantitative assessment of children’s behaviors at the time they are adjusting to the social-cognitive demands of kindergarten and the early school years. This measure has been used extensively with preschool-aged and kindergarten-aged children, and has shown good predictive validity for children’s later academic outcomes (McLelland, Morrison, & Holmes, 2000).
E1-E21	Academic Rating Scale, ARS (Perry & Meisels, 1996)	Included are items asking teachers to report on children’s early literacy, math, and general knowledge skills using the Academic Rating Scale.
F1-F9	Parent-Teacher Involvement Questionnaire (Bierman, Greenberg & CPPRG, 1996)	Included is the Parent-Teacher Involvement Scale (PTI), a 9-item measure developed to assess facets of primary caregiver and teacher involvement. The PTI measure assessed the frequency of contact that occurred between primary caregivers and teachers, the quality of primary caregiver-school involvement, and the primary caregiver's degree of academic engagement with their children. This measure has had success in other prevention studies such as Head Start REDI and Fast Track.

QUESTION-BY-QUESTION JUSTIFICATION OF BASELINE PARENT SURVEY

Question #	Constructs/Items	Justification
A1-A49	Parent-child relationship, DOB, ethnicity/race, country of origin, language(s), child gender, child DOB, child ethnicity/race, child country of origin, child care experiences, housing/household composition, marital status, educational attainment, employment, income, public assistance	To understand what factors at home might predict or moderate the effects of the implemented program on children’s outcomes, included are items asking primary caregivers (most likely mothers) to answer questions regarding demographic information, including racial and ethnic background, family structure, household composition, marital status, levels of educational attainment, employment levels, public assistance, prior child care arrangements, and use of additional child care arrangements. Items are taken from previous intervention studies such as the Chicago School Readiness Project and Foundations of Learning, as well as national evaluations of Head Start such as the Families and Children Experiences Survey (FACES) 2006 surveys.
B1a-B1f	K-6 Kessler Psychological Distress Scale (Kessler, Andrews, Colpe, et al., 2002)	Primary caregivers will also complete measures of maternal depressive symptoms on the K-6 Kessler 6-item Psychological Distress Scale, which is a measure of generalized distress developed using national samples. In this regard, parents’ mental health appears to be especially critical considering its relation to parental perspectives on children’s behavior and to children’s behavior more generally across contexts.
C1-C36	Parenting Stress Index (PSI; Abidin, 1983).	Parenting stress will be assessed using the Parenting Stress Index. As with the mental health problem scale, this measure is likely to be related to children’s behavior and therefore an important covariate in our models.
D1-D15	Parent-Teacher Involvement Questionnaire (PTI; Bierman, Greenberg, & CPPRG, 1996)	The Parent-Teacher Involvement Scale (primary caregiver version) is a 15-item measure developed to assess facets of primary caregiver and teacher involvement. The PTI measure assessed the frequency of contact that occurred between primary caregivers and teachers, the quality of primary caregiver-school involvement, and the primary caregiver's degree of academic engagement with their children. This measure has had success in other prevention studies such as Head Start REDI and Fast Track and therefore, we propose using this measure, as a measure of baseline parental involvement.
E1-E10	Emotion-Related Parenting Styles Self-Test	To assess primary caregivers’ own preferred emotional styles and emotion socialization practices with their children, included are items from the Emotion-Related Parenting Styles Self Test (Hakim-Larson, Parker, Lee, Goodwin, & Voelker, 2006), which is a modification and psychometric evaluation of Gottman’s (1997) scale of the same name. Included are the Emotion Coaching and Dismiss/Disapprove subscales from the shortened Emotion-Related Parenting Styles Self Test measure. This measure will help us understand whether children’s social emotional development changes as a result of parent’s emotion socialization practices.
Section F	Financial resources, housing, and connection to social institutions	Since this population will be low-income and from diverse backgrounds, recency of immigration will be assessed at baseline. Given that we cannot ask directly about caregiver citizenship status, it was recommended that we collect data on things that may be part of the experience of being undocumented, items around household-level access to savings and checking accounts, drivers’ licenses, credit cards, as well as housing quality.
G1-G32	Behavior Problems Index (Zill & Peterson, 1986)	The Behavior Problems Index (BPI) will be used to assess behavioral problems by caregivers. The BPI measures the frequency, range, and type of childhood behavior problems for children age 4 and older and has been used extensively with three year olds as well (Zill & Peterson, 1986). Many items included in the BPI were drawn from the Child Behavioral Checklist (Achenbach, 1991) and other child behavior scales (Graham & Rutter, 1968). It consists of 32 items describing behavior problems. It is widely used for several reasons including its ability to measure a broad developmental range, its simplicity in reading and understanding, and its short length. This measure was used in FACES as well as number of efficacy trials.
H1-H39	Social Skills Rating Scale, SSRS- Social Skills Scale, Parent-Preschool version (Gresham & Elliot, 1990)	The Social Skills Rating Scale - Social Skills Scale (SSRS) will be valuable in helping us to understand whether the intervention affects children’s social problem-solving with peers. Subscales include Cooperation, Assertion, and Self-Control. We recommend using the SSRS because of its comprehensiveness in tapping children’s social skills (Merrell, Streeter, & Boelter, 2001), its superior psychometric properties (Merrell & Gimpel, 1998), and its widespread use by other preschool studies, such as FACES and the ECLS-K.

QUESTION-BY-QUESTION JUSTIFICATION OF FOLLOW-UP PARENT SURVEY

Question #	Constructs/Items	Justification
A1-A20	Parent-child relationship, marital status, educational attainment, employment, income, public assistance	A few brief items are included to assess changes in demographic characteristics at follow-up. These will include changes in marital status, poverty status, employment status, and reliance on public assistance. We do not expect these to be affected by the intervention models, but see these as important contextual factors to report for descriptive purposes, for this sample.
B1-B10	Emotion-Related Parenting Styles Self-Test	To assess primary caregivers’ own preferred emotional styles and emotion socialization practices with their children, included are items from the Emotion-Related Parenting Styles Self Test (Hakim-Larson, Parker, Lee, Goodwin, & Voelker, 2006), which is a modification and psychometric evaluation of Gottman’s (1997) scale of the same name. Included are the Emotion Coaching and Dismiss/Disapprove subscales from the shortened Emotion-Related Parenting Styles Self Test measure. This measure will assess changes to parents’ emotion socialization practices as a result of changes to children’s social emotional development and behavior in the home.
C1-C32	Behavior Problems Index (Zill & Peterson, 1986)	The Behavior Problems Index (BPI) will be used to assess behavioral problems by caregivers. The BPI measures the frequency, range, and type of childhood behavior problems for children age 4 and older and has been used extensively with three year olds as well (Zill & Peterson, 1986). Many items included in the BPI were drawn from the Child Behavioral Checklist (Achenbach, 1991) and other child behavior scales (Graham & Rutter, 1968). It consists of 32 items describing behavior problems. It is widely used for several reasons including its ability to measure a broad developmental range, its simplicity in reading and understanding, and its short length. This measure was used in FACES as well as number of efficacy trials.
D1-D39	Social Skills Rating Scale, SSRS- Social Skills Scale, Parent-Preschool version (Gresham & Elliott, 1990)	The Social Skills Rating Scale - Social Skills Scale (SSRS) will be valuable in helping us to understand whether the intervention affects children’s social problem-solving with peers. Subscales include Cooperation, Assertion, and Self-Control. We recommend using the SSRS because of its comprehensiveness in tapping children’s social skills (Merrell, Streeter, & Boelter, 2001), its superior psychometric properties (Merrell & Gimpel, 1998), and its widespread use by other preschool studies, such as FACES and the ECLS-K.
E1a-E1d	School performance	This measure allows parents to fill out a mock report card on key areas of children’s academic learning such as reading, writing, mathematics, and an overall impression of school performance.

QUESTION-BY-QUESTION JUSTIFICATION OF THE COACH INTERVIEW GUIDE

Question #	Constructs/Items	Justification
Section 1 1-7	Program model knowledge, skills, and strategies; teacher/asst engagement (in training, in coaching); teacher fidelity behavior; successes and challenges in implementing intervention in classroom; adaptations	"Learning transfer" (Kirkpatrick, 1994), or "practice adoption" (Dane & Schneider, 1998), or "practice outcomes" (Dunst et al., 2008) indicates that teachers have internalized what they have learned and are implementing the intervention with fidelity. Various aspects of intervention fidelity have been shown to be associated with larger program impacts on children's behavior and achievement (Spoth, Guyll, Trudeau, & Goldberg-Lillehoj, 2002; Ialongo et al., 1999; Dane & Schneider, 1998; Schoenwald et al., 2004). It will be important for replication purposes to understand what is going well, and the challenges teachers faced and how these challenges were addressed. Also, we need to understand how teachers may have adapted the program to fit their particular circumstances and whether this compromises or enhances fidelity (Flaspohler et al., 2008). Items were developed by the CARES research team.
Section 2 1-10	Content of coaching; methods of coaching; dynamics of coaching sessions; quality of coaching; successes and challenges in coaching; adaptations; teacher-coach relationship	Coaching is more effective when coaches teach new skills, strengthen practitioners' confidence, offer safety in sessions, devote time to specific (vs. generic) skills (Walker, Koroloff, & Shutte, 2002), offer support during stressful times (Schoenwald et al., 2004), focus on outcomes (Bond et al., 2001), and reinforce evidenced-based skill development and adaptation of skills to fit the personal styles of practitioners (Fixsen et al., 2005). Items were developed by the CARES research team.
Section 3 1a-7a	Content of mentoring, methods of mentoring, dynamics of mentoring, quality of mentoring, successes and challenges in mentoring and the coach-trainer relationship	Coaches need training and mentoring to provide specialized coaching to teachers (Fixsen et al., 2005; Schoenwald et al., 2004). Fidelity of coaching is affected by reluctance to seek help from mentor, self-reported feelings of inadequacy on the part of mentors (McCormick & Brennan, 2001). Items were developed by the CARES research team.
Section 4 1a-1b	Teacher-Asst relationship; peer coaching	Research suggests that program implementation might be strongest in settings where teachers and their assistants have a good functional and interpersonal relationship (Foundations of Learning). Items were developed by the CARES research team.
Section 5 1-9c	Staff cohesion; supervisor monitoring; priorities (priority of program, program fits with Center priorities); other factors affecting implementation (e.g., organizational issues)	Research suggests that program implementation might be strongest in settings where staff cohesion is strong (Flaspohler et al., 2008; Foundations of Learning). Research suggests that program implementation might be strongest in settings with sufficient support from leadership (Fixsen et al., 2005; Flaspohler et al., 2008; Han & Weiss, 2005; Lochman et al, in press). The priority that centers place on CARES, and how well CARES fits with other center priorities, may affect implementation success (Flaspohler et al., 2008). Items were developed by the CARES research team.
Section 6 1-3	Additional comments	Additional items concerning additional experiences by coach implementing program model, advice/suggestions for future coaches in program model, any additional general comments, concerns or suggestions. Items were developed by the CARES research team.

QUESTION-BY-QUESTION JUSTIFICATION OF THE TEACHER INTERVIEW GUIDE

Question #	Constructs/Items	Justification
Section 1 1 Section 2 1a-3a	View on children’s social emotional development; experiences with the program model	Literature review by Han & Weiss (2005) shows that the perceived need for an intervention--including value placed on outcomes sought--predicts intervention fidelity. Positive experiences with the program may lead to improved views toward SE development among CARES teachers and, thus, higher intervention fidelity. Organizational buy-in is a critical element of implementation success (Flaspohler et al., 2008). Items were developed by the CARES research team.
Section 3 1a-2	Adaptations in program model implementation	We need to understand how teachers may have adapted the program to fit their particular circumstances, and whether teachers understand the underlying principles of the program model so they can adapt as needed, and whether this compromises or enhances fidelity (Flaspohler et al., 2008). Items were developed by the CARES research team.
Section 4 1a-3a	Challenges in classroom implementation	It will be important for replication purposes to understand what is going well, and the challenges teachers faced and how these challenges were addressed. Items were developed by the CARES research team.
Section 5 1a-6c	Reflections on the coaching process; successes and challenges; quantitative ratings of coaches	Coaching is more effective when coaches teach new skills, strengthen practitioners' confidence, offer safety in sessions, devote time to specific (vs. generic) skills (Walker, Koroloff, & Shutte, 2002), offer support during stressful times (Schoenwald et al., 2004), focus on outcomes (Bond et al., 2001), and reinforce evidenced-based skill development and adaptation of skills to fit the personal styles of practitioners (Fixsen et al., 2005). Higher fidelity coaching predicts higher intervention fidelity (Fixsen et al., 2005; Schoenwald et al., 2004). Items were developed by the CARES research team.
Section 6 1-2e Section 7 1	Teacher & teacher assistant dynamic; informal peer coaching	Research suggests that program implementation might be strongest in settings where teachers and their assistants have a good functional and interpersonal relationship (Foundations of Learning). Items were developed by the CARES research team.
Section 8 1-9	Organizational setting; supervisor support; center priorities affecting implementation	Research suggests that program implementation might be strongest in settings with sufficient support from leadership (Fixsen et al., 2005; Flaspohler et al., 2008; Han & Weiss, 2005; Lochman et al, in press), including efforts to assess teacher performance then provide regular, systematic feedback (Fixsen et al., 2005; Flaspohler et al., 2008; Han & Weiss, 2005; Schoenwald and Hoagwood, 2001). The priority that centers place on CARES, and how well CARES fits with other center priorities, may affect implementation success (Flaspohler et al., 2008). Items were developed by the CARES research team.
Section 9 1-3	Additional comments	Additional items concerning additional experiences by teacher implementing program model, advice/suggestions for future teachers that will implement the program model, any additional general comments, concerns or suggestions. Items were developed by the CARES research team.

QUESTION-BY-QUESTION JUSTIFICATION OF THE CENTER DIRECTOR INTERVIEW GUIDE

Question #	Constructs/Items	Justification
Section 1 1-3	Center director background	These items ask the center director about length of employment at the particular center, specific job duties and responsibilities, and whether they have ever been a Head Start teacher or had another role within Head Start. Items were developed by the CARES research team.
Section 2 1a-10	Experiences with and support for the program model	Research suggests that program implementation might be strongest in settings with sufficient support from leadership (Fixsen et al., 2005;Flaspohler et al., 2008; Han & Weiss, 2005; Lochman et al., in press), including efforts to assess teacher performance then provide regular, systematic feedback (Fixsen et al., 2005; Flaspohler et al., 2008; Han & Weiss, 2005; Schoenwald and Hoagwood, 2001). Items were developed by the CARES research team.
Section 3 1-7d	Program model implementation	Along with support and supervision, the nature and extent of involvement of the center director and other center staff in implementing CARES may affect implementation success (Flaspohler et al., 2008). Items were developed by the CARES research team.
Section 4 1-2	Organizational structure and climate	Center directors have a unique perspective on what factors may have affected implementation of CARES professional development. Additionally, center directors have a unique perspective on how implementation of CARES may have affected center structure, operations, organizational climate, staff roles, and communication. Items were developed by the CARES research team.
Section 5 1-2	Views on children’s social emotional development	The literature review by Han & Weiss (2005) shows that the perceived need for an intervention—including value placed on the outcomes sought—predicts intervention fidelity. Flaspohler et al. (2008) also note the importance of the intervention's fit with one's values. Center directors have a unique perception of how much the organization as a whole values children's social emotional development (especially as related to how much they value academic development). Items were developed by the CARES research team.
Section 6 1-3	Additional comments	Additional items concerning additional experiences regarding implementation program model, advice/suggestions for center directors that will implement the social emotional program enhancements in the future, any additional general comments, concerns or suggestions. Items were developed by the CARES research team.

QUESTION-BY-QUESTION JUSTIFICATION OF THE CENTER STAFF INTERVIEW GUIDE

Question #	Constructs/Items	Justification
Section 1 1-4	Center staff background	These items ask center staff about length of employment at the particular grantee/delegate agency, specific job duties and responsibilities, and whether they have ever had another role within Head Start, and whether they work across multiple centers. Items were developed by the CARES research team.
Section 2 1a-3a	Organizational structure	Flaspohler et al. (2008) identifies numerous features of the broader community that may affect the dissemination of evidence-based practice (see Table 6, p. 190), including leadership, resources, connections among people and organizations, sense of community, norms and values, and community commitment. Items were developed by the CARES research team.
Section 3 1-8	Social emotional development and Head Start CARES	The literature review by Han & Weiss (2005) shows that the perceived need for an intervention—including value placed on the outcomes sought—predicts intervention fidelity. Flaspohler et al (2008) also note the importance of the intervention's fit with one's values. Center staff have a unique perception of how much the organization as a whole values children's social emotional development (especially as related to how much they value academic development). Items were developed by the CARES research team.
Section 4 1-4a	Effects of implementing program models	Along with support and supervision, the nature and extent of involvement of the center staff in implementing CARES may affect implementation success (Flaspohler et al., 2008). Items were developed by the CARES research team.
Section 5 1-4d	Assessment of program models	Organizational buy-in is a critical element of implementation success (Flaspohler et al., 2008). Items were developed by the CARES research team.
Section 6 1	Additional comments	An additional item concerning any additional general comments, concerns or suggestions. Items were developed by the CARES research team.

QUESTION-BY-QUESTION JUSTIFICATION OF THE GRANTEE/DELEGATE AGENCY DIRECTOR INTERVIEW GUIDE

Question #	Constructs/Items	Justification
Section 1 1a-9d	Reasons for participation; additional staff and resources provided; displacement of current services	Grantee directors have a unique perspective on what factors may have affected implementation of CARES professional development. Directors will be probed regarding reasons for participation, how staff and resources were supplemented in order to participate in CARES, and whether participation and services due to CARES displaced other services. Items were developed by the CARES research team.
Section 2 1a-1bi	Sustainability	These items ask grantee/delegate agency director whether they have any plans to continue use of the program models, and if so, what additional resource will be necessary, what types of modifications will be made for continued use. Items were developed by the CARES research team.
Section 3 1-3	Additional comments	Recommendations for future grantees/delegate agencies that will implement the social emotional program enhancements in the future, any additional general comments, concerns or suggestions. Items were developed by the CARES research team.

QUESTION-BY-QUESTION JUSTIFICATION OF THE TRAINER INTERVIEW GUIDE

Question #	Constructs/Items	Justification
Section 1 1-2	Background	These items provide background as to how many coaches and how many centers the trainer works with on the CARES project. Items were developed by the CARES research team.
Section 2 1-8 Section 3 1a-4	Trainer mentoring; relationship with coach	It will be important for replication purposes to understand what is going well, and the challenges coaches faced and how these challenges were addressed. Learning transfer (Kirkpatrick, 1994), or "practice adoption" (Dane & Schneider, 1998), indicates that coaches have internalized what they have learned and are implementing the intervention with fidelity. Items were developed by the CARES research team.
Section 4 1-6c	Teacher implementation	How teachers may have adapted the program to fit their particular circumstances, and whether teachers understand the underlying principles of the program model so they can adapt as needed, and whether this compromises or enhances fidelity is important to understand (Flaspohler et al., 2008). Various aspects of intervention fidelity have been shown to be associated with larger program impacts on children's behavior and achievement (Spoth, Guyll, Trudeau, & Goldberg-Lillehoj, 2002; Ialongo et al., 1999; Dane & Schneider, 1998; Schoenwald et al., 2004). Items were developed by the CARES research team.
Section 5 1-4a	Organizational factors	The priority that centers place on CARES, and how well CARES fits with other center priorities, may affect implementation success (Flaspohler et al., 2008). Trainers have a unique perspective on what factors may have affected implementation of CARES professional development. Items were developed by the CARES research team.
Section 6 1-2	Additional comments	Additional items concerning additional experiences mentoring within the program model, any additional general comments, concerns or suggestions. Items were developed by the CARES research team.

TASK-BY-TASK JUSTIFICATION OF THE DIRECT CHILD ASSESSMENT

Task #	Constructs/Items	Justification
1	Emotion Recognition Questionnaire (ERQ; Ribordy, Camras, Stafani, & Spacarelli, 1988)	The Emotion Recognition Questionnaire is included to assess children’s emotions identification. With the ERQ (as adapted by the Fast Track project), children listen to 16 stories describing characters with emotionally evocative situations, and identify the character’s feeling by pointing to pictures of happy, mad, sad, or scared faces. Measuring the child’s capacity for using emotion knowledge, not just having the knowledge, is important because it is part of the mediational chain.
2	Challenging Situations Task (CST; Denham & Bouril, 1994)	It is important to get an unbiased assessment of social problem solving and competence, therefore direct observation will supplement teacher reporting of these domains. The Challenging Situations Task is included to assess both emotions labeling and social problem skills. Children are presented with pictures of four peer scenarios (e.g., a peer knocking down blocks, being hit, entering a group, a peer taking a ball). The stories focus on peer entry and peer provocation, both challenging situations likely to elicit an affective response from young children. After each scenario, children are asked what they would do in the situation. The REDI trial found that this measure is sensitive to the teaching of social-emotional skills and thus was critical in that trial.
3	Head-to-Toes task (Cameron Ponitz et al., 2008)	The Head-to-Toes task taps a composite assessment of children’s ability to suppress a dominant response in order to carry out a subdominant response and draws on children’s inhibitory control, attention, and working memory.
4	Pencil Tap Task (Diamond & Taylor, 1996)	The Pencil Tap Task will be a measure of children’s executive function that taps working memory, attention, and inhibitory control. This task has been included in several recent efficacy trials with low-income preschool children (including REDI and CSRP) and has demonstrated high levels of predictive validity in a large preschool study (including low-income Head Start children) currently being conducted by Blair and Razza (2007). A proportion score – the number of correct responses divided by the total number of trials – is used as a measure of performance on the task.
5	Item-Selection/Attention Shifting task (Jacques & Zelazo, 2001)	The Item-Selection/Attention Shifting task is included to measure children’s attention and working memory. Children are presented with pictures of three items that vary along some combination of two or three dimensions, including size, shape, and color. The task requires children to identify two of the three objects that are similar along one dimension (i.e., shape) but then to shift cognitive set and identify two of the three objects that are similar along a second dimension (i.e., size).
6	Letter-Word and Applied Problems subscales of the Woodcock Johnson-III (WJ-III; McCrew & Woodcock, 2001).	Based on assessments of school readiness that have been successfully used in large-scale studies with national samples (Zaslow, Reidy, Moorehouse, Halle, Calkins, & Margie, 2001), included is the Letter-Word and Applied Problems subscales of the Woodcock Johnson-III (WJ-III) to assess children’s early academic skills and school readiness. We suggest the WJ-III because it has been used in large-scale studies, including the National Head Start Impact Study.
7	Expressive One-Word Picture Vocabulary Test (EOWPVT; Brownell, 2000)	The Expressive One-Word Picture Vocabulary Test (EOWPVT) which will assess children’s vocabulary.
N/A	Adapted Leiter-R Assessor Report (Smith-Donald, Raver, Hayes, & Richardson, 2007)	Upon completion of direct assessment of the child’s performance on cognitive, self-regulation, and executive function tasks, the assessor will fill out an additional brief report describing the levels of attention, emotion, and behavioral regulation demonstrated by the child during the assessment, using the Adapted Leiter-R Assessor Report. In the Smith-Donald et al. (2007) measure, tapping attention, impulse control, activity, sociability, and affect regulation,

Appendix D.1: Head Start CARES Parent Consent Form

PARENT PERMISSION FORM

Dear Parent or Primary Guardian, Fall 2009

We are working with your child’s Head Start center on an exciting new research project called Head Start CARES. This project aims to learn about ways to help young children succeed in preschool and later years. Your child’s classroom is part of the project and we hope that you will help us.

Success in preschool is often measured by things like how many words or numbers a child knows. Those skills are very important, but we are also interested in learning about your child’s social, emotional, and behavioral skills. Members of our research team will visit your child’s classroom on a few days to watch your child’s teacher “in action” during a typical school day. The purpose of these visits is to get a better understanding of your child’s classroom environment.

We ask your permission for the following:

1. We ask you to allow your child’s teacher to fill out a few questionnaires about your child’s social, emotional, behavioral and early academic progress. These questionnaires will be completed at the beginning and towards the end of this school year. We will also contact your child’s school next year to do the same the following spring.

2. We ask you to participate in the project by completing a questionnaire about you, your child, and your family at the beginning of the school year. Families are their children’s most important “teachers” and you have important information about your child. To show our appreciation, you will be paid $30 after you complete the questionnaire. We will also contact you next year to do the same the following spring and you will also be paid $30 for completing the questionnaire at that point.

3. We ask you to allow your child to participate in several short games that most children find fun, such as looking and pointing at different pictures or playing “opposites” games. These games will occur at the beginning and towards the end of this school year. If your child participates in the games, he or she will receive a toy or book as a gift. We will also contact you next year to complete the games with your child again the following spring and your child will receive a toy or book as a gift.

Is this voluntary?

Yes. For the project to be successful, we hope that as many children and parents participate as possible. There are no known risks associated with participating. However, we want to be clear: your participation and your child’s participation are completely voluntary, and you and your child are free to stop participating at any time without any negative consequences. If you do decide to stop participating, we may continue to use information that was collected about you during the period you were in the study. You and your child can also decide not to participate in any specific portion of the project listed above. You may refuse to answer any questions and/or your child may stop the games and still remain in the study.

Protecting your information

The data collected for this research project will be kept strictly confidential. What we learn from the classroom observations, the teacher reports, the parent questionnaires, or the child games will not be shared

Appendix D.1: Head Start CARES Parent Consent Form

with anyone outside the research staff of the Head Start CARES project. The information from this project will be used only for research purposes and program improvement purposes.

To make sure we keep your information as confidential as possible, all paper data will be kept locked up, and computerized files will be password-protected. The results of this project may be published in reports, but your child will never be identified individually in any published reports. There is one exception to the confidentiality offered in this project: If we are worried about your child’s safety, then we might have to share information about you and your child with an agency outside of the research team without your signed permission.

We will do everything we can to keep others from learning about your participation in the research. We expect to receive a Confidentiality Certificate from the U.S. government that adds special protection for the research information that identifies you. It will say that we do not have to identify you, even under a court order or subpoena. You should know, however, that we may tell someone if harm to you, harm to others, or child abuse becomes a concern. Also, the federal agency that pays for this study may see your information in an audit, but it too will protect your privacy. This Certificate does not mean the government approves or disapproves of our project.

Do you have any questions?

Please ask! If you agree to allow your child to participate in this research project, please sign below. If you have any questions about this project, please call Pamela Morris or Ximena Portilla at MDRC using the toll-free number (800) 221-3165.

I

your first name, last name

, _________________________________________, understand the nature of this research project and I

a

child’s first name, last name

gree to the following data collection about my child, _________________________________________:

(check one)

Teacher Report, Parent Survey, and Child Games: ₁ YES ₂ NO

_______________________________ ______/_______/_____ Child Gender: ₁ Male ₂ Female

Child First Name, Last Name Child Date of Birth

Name of Your Child’s Head Start: __________________________________________________________

Your Signature: ______________________________________________ Date: ____________________

Lastly, we would like your permission to videotape our observation of your child playing the games for training purposes and evaluating our work. Videotaping is voluntary and not a requirement to complete the activities listed above.

I agree to my child being videotaped during the child games for training and evaluation purposes.

(check one)

₁ YES ₂ NO

Your Signature: ______________________________________________ Date: ____________________

Appendix D.2: Head Start CARES Teacher Consent Form

TEACHER PERMISSION FORM

Dear Head Start Teacher, Fall 2009

Welcome to the Head Start CARES research project! The purpose of this project is to learn about ways that teachers and children succeed in early educational classrooms. This project gives us the chance to learn about ways teachers structure their students’ learning environments, and ways that teachers manage the multiple challenges they face in their Head Start classrooms.

As part of this project, we will conduct a few days of classroom observation in your classroom, where members of our research team will watch the classroom “in action” during a typical school day. By agreeing to be part of the project, your Head Start center has provided us with the permission to conduct these observations in your classroom so that we can get a better understanding of the Head Start environment and the kinds of activities the children do in a typical school day.

During the fall and spring of the school year, we will ask you to fill out questionnaires about the children enrolled in your classroom to be able to measure children’s academic and socioemotional progress over the course of the school year. Parents of these children will have agreed to this before you complete them. You will receive $15 per child for completing the questionnaires.

We ask your permission for the following:

1. In the fall and spring of this school year, we will ask you to complete a teacher survey lasting approximately 20 minutes, asking questions about yourself, your experiences being a teacher, and your thoughts about your classroom experiences. This information will be combined across all centers so that we can provide a general description of the teachers who are part of the Head Start CARES project. You will receive $15 for completing each survey.

Is this voluntary?

Yes. For the project to be successful, we hope that as many teachers participate as possible. However, we want to be clear: your participation with regard to the completion of the surveys is completely voluntary, and you are free to stop participating at any time. You may refuse to answer any questions and still remain in the study. You may also stop the interview at any time without penalty. There are no known risks associated with participation.

Protecting your information

The results of this research project are strictly confidential. The surveys you complete will not be shared with anyone outside the Head Start CARES research project, including the staff from Head Start. The information from this project will be used for research purposes and program improvement purposes only, and you can be assured that your performance as a teacher is not being evaluated. To make sure we keep your information as confidential as possible, all paper data will be stored in locked research facilities, and computerized files will be password-protected. The results of this project may be published in reports, but

Appendix D.2: Head Start CARES Teacher Consent Form

you will never be identified individually in any published reports. The information collected from you about individual children will also be kept confidential and will only be used for research purposes.

We will do everything we can to keep others from learning about your participation in the research. We expect to receive a Confidentiality Certificate from the U.S. government that adds special protection for the research information that identifies you. It will say that we do not have to identify you, even under a court order or subpoena. You should know, however, that we may tell someone if harm to you, harm to others, or child abuse becomes a concern. Also, the federal agency that pays for this study may see your information in an audit, but it too will protect your privacy. This Certificate does not mean the government approves or disapproves of our project.

Do you have any questions?

Please ask! If you agree to participate in this research project, please sign the consent form below. If you have any questions about this project, please call Pamela Morris or Ximena Portilla at MDRC using the toll-free number (800) 221-3165.

I

your first name, last name

, _________________________________________________, understand the nature of this research

project and agree to have the following data collection occur:

(check one)

Teacher Survey: ₁ YES ₂ NO

I understand that I am free to stop participating at any time, and that I will be paid $15 for completing each teacher survey about myself, and $15 per child for completing the reports on each child.

Name of Head Start Center: _______________________________________________________________

Your Signature: ______________________________________________ Date: ____________________

References Cited

Aber, J. L., Jones, S., & Cohen, J. (2000). The impact of poverty on the mental health and development of very young children. In C. H. Zeanah, Jr. (Ed.), Handbook of infant mental health (2nd ed., pp. 113-128). New York: Guilford Press.

Abidin, R. R. (1983). Professional Manual for the Parenting Stress Index (3rd Ed.). Odessa, FL: Psychological Assessment Resources.

Ackerman, B. P., Kogos, J., Youngstrom, E., Schoff, K., & Izard, C. (1999). Family instability and the problem behaviors of children from economically disadvantaged families. Developmental Psychology, 35, 258-268.

Alexander, K. L., Entwisle, D. R., & Kabbani, N. S. (2001).The dropout process in life course perspective: Early risk factors at home and school. Teachers College Record, 103, 760 – 822.

Alexander, K. L., Entwistle, D. R., & Dauber, S. L. (1993). First grade classroom behavior: Its short and long-term consequences for school performance. Child Development, 64, 801- 814.

Arnold, D. H., Brown, S., Meagher, S., Baker, C. N., Dobbs, J., & Doctoroff, G. L. (2006). Preschool-based programs for externalizing problems. Education and Treatment of Children, 29, 311-340.

Belsky, J., Vandell, D. L., Burchinal, M., Clarke-Stewart, A., McCartney, K., Owen, M. T., & the NICHD Early Child Care Research Network (2007). Are there long-term effects of early child care? Child Development, 78, 681-701.

Berlin, M., Mohadjer, L., Waksberg, J., Kolstad, A., Kirsch, I., Rock, D., & Yamamoto, K. (1992). An experiment in monetary incentives. Proceedings of the Survey Research Section of the American Statistical Association, 393-398.

Bierman, K. L., Greenberg, M. T., & Conduct Problems Prevention Research Group (1996). Social skill training in the Fast Track program. In R. DeV. Peters & R.J. McMahon (Eds.), Preventing childhood disorders, substance abuse and delinquency (pp. 65-89). Newbury Park, CA: Sage.

Blair, C., Granger, D., & Razza, R. P. (2005). Cortisol reactivity is positively related to executive function in preschool children attending Head Start. Child Development, 76, 554-567.

Brooks-Gunn, J., Duncan, G., & Aber, J. L. (Eds.). (1997). Neighborhood poverty: Context and consequences for children, Volume 1. New York: Russell Sage.

Brooks-Gunn, J., Klebanov, P. K., Liaw, F., & Spiker, D. (1993). Enhancing the development of low-birthweight, premature infants: Changes in cognition and behavior over the first three years. Child Development, 64, 736-753.

Brownell, R. (2000). Expressive One-Word Picture Vocabulary Test, 2000 Edition. Lutz, FL: Psychological Assessment Resources.

Campbell, S. B. (1995). Behavior problems in preschool children: A review of recent research. Journal of Child Psychology and Psychiatry, 36, 113-149.

Cannell, C. F., & Henson, R. (1974). Incentives, motives, and response bias. Annals of Economic and Social Measurement, 3, 307-317.

Chromy J. R., & Horvitz, D. G. (1978). The use of monetary incentives in National Assessment Household Surveys. Journal of the American Statistical Association, 73, 473-478.

Conduct Problems Prevention Research Group (1999). Initial impact of the Fast Track prevention trial for conduct problems: II. Classroom effects. Journal of Consulting and Clinical Psychology, 67, 648-657.

Consortium on the School-Based Promotion of Social Competence (1994). The school-based promotion of social competence: Theory, research, practice, and policy. In R. J. Haggerty, L. R. Sherrod, N. Garmezy, & M. Rutter (Eds.), Stress, risk, and resilience in children and adolescents: Processes, mechanisms, and interventions (pp. 268-316). New York: Cambridge University Press.

Cooper, D. H., & Farran, D. C. (1991). Behavioral risk factors in kindergarten. Early Childhood Research Quarterly, 3, 1-19.

Cooper, H. M., & Hedges, L. V. (1994). The handbook of research synthesis. New York: Russell Sage.

Dane, A.V., & Schneider, B.H. (1998). Program integrity in primary and early secondary

prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23-45.

Denham, S. A., Bouril, B., (1994) Preschoolers' affect and cognition about challenging peer

situations. Child Study Journal, Vol. 24( 1).

DeVellis, R. F. (1991). Scale development: Theory and applications. Newbury Park, CA: Sage

Publications.

Dodge, K. A., Pettit, G. S., & Bates, J. E. (1994). Socialization mediators of the relations between socioeconomic status and child conduct problems. Child Development, 65, 649- 660.

Durlak, J.A., and DuPre, E.P. (2008). Implementation matters: a review of research on the

influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3-4), 327-350.

Entwisle, D. R., & Hayduk, L. A. (1988). Lasting effects of elementary school. Sociology of Education, 61, 147-159.

Fantuzzo, J., Stoltzfus, J., Lutz, M. N., Hamlet, H., Balraj, V., Turner, C., & Mosca, S. (1999). An evaluation of the special needs referral process for low-income preschool children with emotional and behavioral problems. Early Childhood Research Quarterly, 14, 465- 482.

Farmer, E. M. Z., Stangl, D. K., Burns, B. J., Costello, E. J., & Angold, A. (1999). Use, persistence, and intensity: Patterns of care for children's mental health across one year. Community Mental Health Journal, 35, 31-46.

Fixsen, D.L., Naoom, S.F., Blasé, K.A., Friedman, R.M., & Wallace, F. (2005). Implementation

research: A synthesis of the literature. Tampa, FL: National Implementation Research Network at the Louis de la Parte Florida Mental Health Institute, University of South Florida.

Flaspohler, P., Duffy, J., Wandersman, A., Stillman, L., and Maras, M.A. (2008). Unpacking

prevention capacity: An intersection of research-to-practice models and community-centered models. American Journal of Community Psychology, 41, 182–196.

Garmezy, N. (1991). Resilience and vulnerability to adverse developmental outcomes associated with poverty. American Behavioral Scientist, 34, 416-430.

Gennetian, L., & Miller, C. (2000). Reforming welfare and rewarding work: Final report on the

Minnesota Family Investment Program, Volume 2: Effects on children. New York: MDRC.

Gennetian, L. A., Morris, P., Bos, J., & Bloom, H. S. (2005). Constructing instrumental variables from experimental data to explore how treatments produce effects. In H. Bloom (Ed.), Learning more from social experiments: Evolving analytic approaches. New York: Russell Sage.

Gilliam, W. (2005). Pre-kindergarteners left behind: Expulsion rates in state pre-kindergarten systems. New Haven, CT: Yale University Child Study Center.

Gilliom, M., Shaw, D., Beck, J. E., Schonberg, M. A., & Lukon, J. L. (2002). Anger regulation in disadvantaged preschool boys: Strategies, antecedents, and the development of self- control. Developmental Psychology, 38, 222-235.

Glisson, C., and Hemmelgarn, A. (1998). The effects of organizational climate and

interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse & Neglect, 22(5), 401–421.

Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K.E., Mayberg, S., &

Green, P. (2008). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health, 35, 98–113.

Greenberg, D., Meyer, R. H., & Wiseman, M. (1993). Prying the lid from the black box: Plotting evaluation strategy for welfare employment and training programs. Madison, WI: University of Wisconsin, Institute for Research on Poverty.

Gresham, F. M., & Elliott, S. N. (1990). Social Skills Rating System. Circle Pines, MN: American Guidance Service. Retrieved January 9, 2009 from: http://ags.pearsonassessments.com/group.asp?nGroupInfoID=a3400

Groves, R. M. (1989). Survey errors and survey costs. New York: John Wiley and Sons.

Hakim-Larson, J., Parker, A., Lee, C., Goodwin, J., & Voelker, S. (2006). Measuring parental meta-emotion: Psychometric properties of the emotion-related parenting styles self-test. Early Education and Development, 17, 229–251.

Hamre, B. K., & Pianta, R. C. (2001). Early teacher-child relationships and the trajectory of children’s school outcomes through eighth grade. Child Development, 72, 625-638.

Han, S.S., & Weiss, B. (2005). Sustainability of teacher implementation of school-based mental

health programs. Journal of Abnormal Child Psychology, 33(6), 665-679.

Hazard. 2002, May 14. “Proposal for Incentives in the 1999 SPD.”

URL: http://www.bls.census.gov/spd/workpaper/incent_jus.htm 1-41.

Ialongo, N.S., Werthamer, L., Kellam. S.G., Brown, C.H., Wang, S. & Lin, Y. (1999). Proximal

impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27 (5), 599-641.

James, T. L. (1997). Results of the wave 1 incentive experiment in the 1996 Survey of Income and Program Participation. Proceedings of the Survey Research Section of the American Statistical Association, 834-839.

Kaiser, A. P., Xinsheng, C., Hancock, T. B., & Foster, E. M. (2002). Teacher-reported behavior problems and language delays in boys and girls enrolled in Head Start. Behavioral Disorders, 28, 23-29.

Kessler, R. C., Andrews, G., Colpe, L., Hiripi, E., Mroczek, D. K., Normand, S. T., Walters, E. E., & Zaslavsky, A. M. (2002). Short screening scales to monitor population prevalence and trends in non-specific psychological distress. Psychological Medicine, 32, 959-976.

Kochanska, G., Murray, K. T, & Harlan, E. T. (2000). Effortful control and early childhood: Continuity, change, antecedents, and implications for social development. Developmental Psychology, 36, 200-232.

Kirkpatrick, D.L. (1994). Evaluating Training Programs: The Four Levels. San Francisco, CA:

Berrett-Koehler.

Kulka, R. (1992). A brief review of the use of monetary incentives in federal statistical surveys. Paper presented at the Council on Professional Associations on Federal Statistics/OMB Symposium on Providing Incentives to Survey Respondents. Cambridge, MA: Harvard University.

Ladd, G. W., & Price, J. M. (1987). Predicting children’s social and school adjustment following the transition from preschool to kindergarten. Child Development, 58, 1168-1189.

Ladd, G. W., Birch, S. H., & Buhs, E. S. (1999). Children's social and scholastic lives in kindergarten: Related spheres of influence? Child Development, 70, 1373-1400.

Ladd, G. W., Buhs, E. S., & Troop, W. (2002). Children’s interpersonal skills and relationships in school settings: Adaptive significance and implications for school-based prevention and intervention programs. In P. K. Smith & C. H. Hart (Eds.), Blackwell handbook of childhood social development: Blackwell handbooks of developmental psychology (pp. 394-415). Malden, MA: Blackwell Publishing.

Ladd, G. W., Kochenderfer, B. J., & Coleman, C. (1997). Classroom peer acceptance, friendship, and victimization: Distinct relational systems that contribute uniquely to children’s school adjustment. Child Development, 68, 1181-1197.

Lavigne, J. V., Gibbons, R. D., Christoffel, K. K., Arend, R., Rosenbaum, D., Binns, H., Dawson, N., Sobel, H., & Isaacs, C. (1996). Prevalence rates and correlates of psychiatric disorders among preschool children. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 204-214.

Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness

for change. Journal of Substance Abuse Treatment, 22(4), 197-209.

Lee, V. E., & Burkham, D. T. (2002). Inequality at the starting gate: Social background differences in achievement as children begin school. Ann Arbor, MI: EPI.

Lochman, J. E. (2000). Parent and family skills training in targeted prevention programs for at-

risk youth. The Journal of Primary Prevention, 21(2), 253-265.

Lochman, Powell, Boxmeyer, Qu, Wells, and Windle (In Press). Dissemination of the Coping

Power Program: Importance of intensity of counselor training.

Mack, S., Huggins, V., Keathley, D., & Sudukchi, M. (1998). Do monetary incentives improve response rates in the survey of income and program participation? Proceedings of the Survey Research Section of the American Statistical Association, 529-534.

Magnuson, K. A., Ruhm, C., & Waldfogel, J. (2007). Does prekindergarten improve school preparation and performance? Economics of Education Review, 26, 33-51.

Maslach, C., & Jackson, S. E. (1981). The measurement of experienced burnout. Journal of Occupational Behaviour, 2, 99-113.

Mather, N., & Woodcock, R. W. (2001a). Examiner’s Manual: Woodcock-Johnson III Tests of Cognitive Abilities. Itasca, IL: Riverside Publishing.

Mather, N., & Woodcock, R. W. (2001b). Examiner’s Manual: Woodcock-Johnson III Tests of Achievement. Itasca, IL: Riverside Publishing.

McCabe, L. A., Hernandez, M., Lara, S. L., & Brooks-Gunn, J. (2000) Assessing preschoolers’ self-regulation in homes and classrooms: Lessons from the field. Behavioral Disorders, 26, 53-69.

McGrew, K. S., & Woodcock, R. W. (2001). Technical Manual: Woodcock-Johnson III. Itasca, IL: Riverside Publishing.

McLelland, M. M., Morrison, F. J., & Holmes, D. H. (2000). Children at risk for early academic problems: The role of learning-related social skills. Early Childhood Research Quarterly, 15, 307-329.

Miller, H. W., Kennedy, J., & Bryant, E. E. (1972). A study of the effect of remuneration upon response in a health and nutrition examination survey. Proceedings of the Social Statistics Section of the American Statistical Association, 370-375.

Moffit, R. (2004). The Three-City Study Incentive Experiment: Results from the first two waves. Retrieved January 9, 2009 from: http://www.jhu.edu/~welfare

National Education Goals Panel (1996). The national education goals report: Building a nation of learners. Washington, DC: U.S. Government Printing Office.

O’Neil, R., Welsh, M., Parke, R. D., Wang, S., & Strand, C. (1997). A longitudinal assessment of the academic correlates of early peer acceptance and rejection. Journal of Clinical Child Psychology, 26, 290-303.

Perry, N. E., & Meisels, S. J. (1996). Teacher judgments of students academic performance (National Center for Education Statistics Working Paper #96-08). Washington, DC: U.S. Department of Education, OERI.

Pianta, R. C. (2001). STRS Student-teacher Relationship Scale: Professional Manual. Lutz, FL: Psychological Assessment Resources.

Pollak, S. D., Cichetti, D., Hornung, K., & Reed, A. (2000). Recognizing Emotion in Faces:

Developmental Effects of Child Abuse and Neglect. Developmental Psychology, 36(5), 679-688.

Pollak, S. D. (2003). Experience-dependent affective learning and risk for psychopathology in

children. Annals of the New York Academy of Sciences, 1008, 102–111.

Qi, C. H., & Kaiser, A. P. (2003). Behavior problems of preschool children from low-income families. Topics in Early Childhood Special Education, 23, 188–216.

Raver, C. C. (2002). Emotions matter: Making the case for the role of young children’s emotional development for early school readiness. Social Policy Report, 16, 3-18. Ann Arbor, MI: Society for Research in Child Development.

Raver, C. C., Garner, P. W., & Smith-Donald, R. (2006). The roles of emotion regulation and emotion knowledge for children’s academic readiness: Are the links causal? In R. C. Pianta & K. L. Snow. (Eds.), Kindergarten transition and early school success. Baltimore, MD: Brookes Publishing.

Reynell, J. K., & Gruber, C. P. (1990). Reynell Development Language Scales: U.S. Edition. Los Angeles, CA: Western Psychological Services.

Rimm-Kaufman, S., Pianta, R. C., & Cox, M. (2000). Teachers’ judgments of problems in the transition to school. Early Childhood Research Quarterly, 15, 147-166.

Singer, E., van Hoewyk, J., & Maher, M. P. (1998). Does the payment of incentives create expectation effects? Public Opinion Quarterly, 62, 152-164.

Spouse, J. (2001). Bridging theory and practice in the supervisory relationship: A sociocultural

perspective. Journal of Advanced Nursing, 33(4), 512-522.

U.S. Census Bureau, Survey of Program Dynamics (2002). Proposal for incentives in the 1999 SPD. Retrieved January 9, 2009 from http://www.census.gov/spd/workpaper/incent_jus.htm

U.S. Department of Health and Human Services, Administration for Children and Families (2005). Head Start Impact Study: First year findings. Retrieved January 9, 2009 from: http://www.acf.hhs.gov/programs/opre/hs/impact_study/reports/first_yr_finds/first_yr_fin ds.pdf

Zill, N., & Peterson, J. (1986). Behavior Problems Index. Washington, DC: Child Trends, Inc.

Federal Register Comment:

IMPROVING MENTAL HEALTH IN HEAD START

It appears that, at last, we may have an administration that understands the significant of the issues facing our most vulnerable and valuable resources in our country – our children. I am encouraged to think of what new changes may be ahead for Head Start. There are undoubtedly many areas that are being looked at for enhancing Head Start so that children can be ready for success in school and for life.

Without coming across as presumptuous, I would like to suggest an area of child development that may go unnoticed. It center around Mental Health. Based upon my experiences at Head Start I do not believe there is a clear understanding of what constitutes child Mental Health. I witnessed this many times for over a decade. This includes my discussions with the previous Commission of Head Start (concerning the NRS system), various directors of Head Start, and numerous management personnel in Head Start. The following have been some of my many struggles in Head Start:

• The NRS - an academic approach to helping children with Mental Health concerns.

• The use of academic curriculum methodologies that prohibit hugs because they are disruptive in the classrooms.

• Brain restructuring approach such as brain-gyms to solve children’s need for quality relationships.

• Therapeutic approaches that believe that parent involvement can be replaced with child therapy.

Quality relationships between primary care providers at home and at child care facilities are essential for the social and emotional development of children. There are no exceptions or replacements for this critical need. Even though there is some small references to this the Head Start Performance Standard, it does not seem to be understood by those in leadership in Head Start. Perhaps it needs to be made clearer in the Performance Standards.

The following are my suggestions for improving the Mental Health area of Head Start:

• Require qualified leadership in Head Start. There has been a valid attempt by Head Start to educate teachers (i.e. CDS, AA, BA). But this is a bottom up approach. The focus of directors in HS seems to be in clerical areas and not in child development. This is due to a lack of knowledge of child develop. The result has been a dumbing down effect because the Leaders who lead, and the Policy Makers who make policies, do not understand why children are falling into the pipeline to failure. There is a clear and critical need for qualified leadership in the ECE field – ESPECIALLY HS.

• Require parent participation. This requirement is similar to Welfare Reform under the Clinton administration. It requires participants to buy in to their own success. It can be monitored and tract the same way Inkind is monitored and tract. If a parent read to his/her child as little as 20 minutes, 6 nights a week, they will accumulate approximately 10% of Head Start Inkind requirements. There are many other ways parents can become more fully involved in the child’s success and their success. It should be required in order to receive tax payers’ financial support.

• Require Head Start programs to focus on relationship based trainings for parents and teachers. (i.e. Watch, Wait, and Wonder, The Incredible Years ,and Brazelton Touchpoints).

Joe Michael Poynter,
Children’s Defense Funds, Fellow

Joempoynter@MSN.com 970-563-0142

Response:

We received one comment to our federal register notice. In this comment, the author argues for the importance of attention to children’s mental health in Head Start settings. Notably, the programs we are testing as part of the Head Start CARES project will allow us to address whether and how a set of social-emotional program enhancements can make a difference for children when implemented on a national scale. While not focusing exclusively on children’s mental health, these programs target the emotional and social competencies that underlie children’s mental health outcomes. That is, they target children’s emotional competence (children’s their ability to manage their emotions, to understand the feelings and emotions of others, and to take another’s perspective), and their social competence (their ability to enter into social relationships and manage their interactions with peers and form friendships). And, internalizing and externalizing behavior problems, early indicators of mental health problems, will be key outcomes that will be assessed as part of our measurement plan. Therefore, this project will help to provide important information to the field on whether these social-emotional strategies are effective in improving social-emotional as well as academic outcomes for children in Head Start settings.