RESEARCH QUESTIONS
|
CONSTRUCTS CAPTURED BY EACH QUESTION
|
SURVEY ITEMS/OPERATIONAL DEFINITIONS
|
Child and Family Outcomes
|
|
|
How
does Project LAUNCH affect specific
individual-level child and family outcomes?
|
|
|
Are
children in LAUNCH communities in better
social-emotional health
than those in comparison communities?
|
relationships
Initiative
Self-regulation
Behavioral concerns
Social awareness
Self-management
Goal direction
Relationship skills
|
Parent Survey
(NOTE:
Listed below are the relevant items and subscales from the
DECA/DESSA instruments, with the specific items and subscales to
vary by the age of the child in the version of the Parent Survey
used for each respondent.)
Attachment/relationships
(DECA-I)
Show
pleasure when interacting with adults
Accept
comfort from a familiar adult
Smile
at familiar adults
Act
happy
Respond
positively to adult attention
Smile
back at a familiar adult
Calm
down with help from a familiar adult
Make
eye contact with others
Act
in a good mood
Seek
comfort from familiar adults
Show
affection for a familiar adult
Enjoy
being cuddled
Enjoy
interacting with others
Respond
when spoken to
Initiative
(DECA-I)
Try
to do new things
Imitate
actions of others
Keep
trying when unsuccessful
Show
interest in what others were doing
Notice
changes in surroundings
Adjust
her/his energy level to the type of play
Act
happy when praised
Explore
surroundings
Express
her/his dislikes
Reach
for a familiar adult
Respond
to her/his name
React
to another child's cry
Keep
trying to obtain a toy
Act
in a way that make others smile or show interest
Easily
go from one activity to another
Seek
attention when a familiar adult was with another child
Enjoy
being around other children
Look
to familiar adult when exploring her/his surroundings
Attachment/relationships
(DECA-T)
Show
affection for a familiar adult
Easily
go from one activity to another
Act
happy with familiar adults
Show
pleasure when interacting with adults
Smile
back at a familiar adult
Seek
comfort from familiar adults
Express
a variety of emotions (e.g., happy, sad, mad)
Reach
for a familiar adult
Makes
needs known to a familiar adult
Accept
comfort from a familiar adult
Act
happy when praised
Make
eye contact with others
Makes
others aware of her/his needs
Show
interest in her/his surroundings
Respond
to her/his name
Respond
when spoken to
Enjoy
being cuddled
Enjoy
interacting with others
Initiative
(DECA-T)
Show
concern for other children
Try
to comfort others
Play
make-believe
Try
to clean up after herself/himself
Show
preference for a particular playmate
React
to another child’s cry
Ask
to do new things
Play
with other children
Participate
in group activities
Try
to do things for herself/himself
Follow
simple directions
Self-regulation
(DECA-T)
Handle
frustration well
Accept
another choice when the first choice was not available
Adjust
to changes in routine
Calm
herself/himself
Easily
follow a daily routine
Have
regular sleeping pattern
Easily
go from one activity to another
Initiative (9
items) (DECA-P2)
Show
confidence in his/her abilities (for instance, say “I can
do it!”)?
Keep
trying when unsuccessful (show persistence)?
Try
different ways to solve a problem?
Show
an interest in learning new things?
Make
decisions for himself/herself?
Choose
to do a task that was hard for him/her?
Remember
important information?
Try
or ask to try new things or activities?
Start
or organize play with other children?
Self-regulation
(9 items) (DECA-P2)
Listen
to or respect others?
Control
his/her anger?
Handle
frustration well?
Accept
another choice when his/her first choice was not available?
Cooperate
with others?
Calm
himself/herself down?
Play
well with others?
Share
with other children?
Show
patience?
Attachment/relationships
(9 items) (DECA-P2)
Act
in a way that made adults smile or show interest in him/her?
Seem
happy or excited to see his/her parent or guardian?
Trust
familiar adults and believe what they say?
Seek
help from children/adults when necessary?
Appear
happy when playing with others?
Look
forward to activities at home or school (for instance, birthdays
or trips)?
Show
a preference for a certain adult, teacher, or parent?
Show
affection for familiar adults?
Ask
adults to play with or read to him/her?
Behavioral
concerns (11 items) (DECA-P2)
Seem
sad or unemotional at a happy occasion?
Have
a temper tantrum?
Seem
uninterested in other children or adults?
Use
obscene gestures or offensive language?
Destroy
or damage property?
Fight
with other children?
Become
upset or cry easily?
Hurt
others with actions or words?
Get
easily distracted?
Touch
children or adults in a way that you thought was inappropriate?
Have
a short attention span (difficulty concentrating)?
Social
awareness (DESSA)
Cope well with insults and mean
comments?
Get along with different types
of people?
Act respectfully in a game or
competition?
Respect another person’s
opinion?
Contribute to group efforts?
Resolve a disagreement
Share with others?
Cooperate with peers or
siblings?
Forgive somebody who hurt or
upset her/him?
Self-management
(DESSA)
Pay
attention?
Wait
for her/his turn?
Focus
on a task despite a problem or distraction?
Act
comfortable in a new situation?
Perform
the steps of a task in order?
Think
before he/she acted?
Pass
up something he/she wanted, or do something he/she did not like,
to get something better in the future?
Accept
another choice when his/her first choice was unavailable?
Stay
calm when faced with a challenge?
Adjust
well to changes in plans?
Adjust
well when going from one setting to another?
Goal-direction
(DESSA)
Keep
trying when unsuccessful?
Take
steps to achieve goals?
Try
to do her/his best?
Seek
out additional knowledge or information?
Take
an active role in learning?
Do
things independently?
Ask
to take on additional work or responsibilities?
Show
creativity in completing a task?
Seek
out challenging tasks?
Work
hard on projects?
Relationship
skills (DESSA)
Compliment
or congratulate somebody?
Do
something nice for somebody?
Show
appreciation of others?
Greet
a person in a polite way?
Attract
positive attention from peers?
Express
concern for another person?
Attract
positive attention from adults?
Make
a suggestion or request in a polite way?
Offer
to help somebody?
Respond
to another person’s feelings?
|
Do
families in LAUNCH communities exhibit stronger
parent-child relationships
than those in comparison communities?
|
Parent-Child Relationship
Attachment
Discipline practices
Involvement
Parenting confidence
|
Parent Survey
PRQ-P
Attachment
When
my child is upset, I can calm him/her.
I
know what my child is thinking.
I
can sense my child's moods.
I
know when my child will become upset.
I
know when my child wants to be left alone.
I
enjoy spending time with my child.
I
know what to say to calm down my child.
I
know what my child is feeling.
When
upset, my child comes to me for comfort.
My
child enjoys spending time with me.
I
know how my child will react in most situations.
Discipline
practices
I
punish my child if he/she talks back to an adult.
I
punish my child if he/she shows disrespect to an adult
I
punish my child when he/she misbehaves.
I
punish my child if he/she destroys someone else's things.
It’s
my responsibility as a parent to punish all my child’s
misbehavior
I
insist that my child follow the rules of the house.
I
punish my child so he/she learns the proper respect for others.
It
is important for a child to follow family rules
Children
should do what parents tell them to do.
Involvement
My
child and I plan things to do together.
My
child and I go on outings together.
I
teach my child how to play new games.
My
child and I do arts and crafts together.
My
child and I take walks together.
My
child and I play games together.
My
child and I work on projects together.
My
child and I do things together outdoors.
Parenting
confidence
I
am confident in my parenting ability.
I
make good parenting decisions.
It
is easy for me to make decisions about what my child should do.
I
have the energy that I need to cope with my child.
I
remain calm when dealing with my child's behavior
My
child knows the house rules.
I
am in control of my household.
Relational
frustration
My
child is hard for me to handle.
During
the last year, my child has been difficult to take care of.
I
lose my temper with my child.
My
child tests my limits.
I
lose my patience with my child.
I
overreact when my child misbehaves.
It's
hard being a parent.
I
make a lot of mistakes when dealing with my child.
My
child complains about how I treat him/her.
|
Do
parents in LAUNCH communities have higher
reported levels of social support
than do parents in comparison communities?
|
|
Parent Survey
Social Support (FSSQ)
|
Do
parents in LAUNCH communities have lower
reported levels of depression
than do parents in comparison communities?
|
|
Parent Survey
CES-D-10
|
Do
families in LAUNCH communities engage more
frequently in specific family activities
(e.g., playing games, reading books, arts and crafts) than those
in comparison communities?
|
|
Parent Survey
Home Environment items (ECLS-K
and ECLS-B)
|
Are
children in LAUNCH communities more
likely to be “school ready”
than those in comparison communities measured on the following
domains?
physical
health and well-being,
social
competence,
emotional
maturity,
language
and cognitive development, and
communication
skills and general knowledge
in relation to developmental benchmarks
|
Physical
readiness for school day
Physical
independence
Gross
and fine motor skills
Basic
and advanced literacy
Interest
literacy/numeracy and memory
Basic
numeracy
Overall
social competence
Responsibility
and respect
Readiness
to explore new things
Behavior:
Communication and general knowledge
|
EDI
Section A – Physical
Well-Being
Since
the start of school in the fall, has this child sometimes (more
than once) arrived:
Would
you say that this child:
How
would you rate this child’s:
Proficiency
at holding a pen, crayons, or a brush
Ability
to manipulate objects
Ability
to climb stairs
Level
of energy throughout the school day
Overall
physical development
Section B – Language and
Cognitive Skills
Section C – Social and
Emotional Development
Section D – Special
Concerns
Does
the student have a difficulty that influences his/her ability to
do school work in a regular classroom?
If
the child has received a diagnosis or identification by a doctor
or psychological professional, please indicate.
Is
the child receiving any school-based support(s) (e.g. educational
assistant, equipment)?
Is
the child currently receiving further assessment?
Is
the child currently on a wait list to receive further assessment?
Do
you feel that this child needs further assessment?
|
Are
children in LAUNCH communities less
likely to be suspended and/or expelled
from school than those in comparison communities?
|
|
School survey
During
the last full school year, how many children were suspended for
behavioral reasons in each age group or grade level?
During
the last full school year, how many children were expelled (or
involuntarily unenrolled) for behavioral reasons in each age
group or grade level?
|
[SECONDARY
RESEARCH QUESTION] Are children in LAUNCH communities in better
physical health than
those in comparison communities after program implementation*?
[*
Note: Although program implementation has occurred at different
points in time across LAUNCH communities, we will be able to
capture this “dose/duration” dimension in our
analysis—and address this directly in Q4 below.]
|
Child health
Physical independence
|
Parent Survey
Q14.
Does your child currently need or use medicine prescribed by a
doctor, other than vitamins?
Q15.
Does your child need or use more medical care, mental health or
educational services than is usual for most children of the same
age?
Q16.
Is your child limited or prevented in any way in your child’s
ability to do the things most children of the same age can do?
Q17.
Does your child need or get special therapy, such as physical,
occupational, or speech therapy?
Q21.
In general, how would you describe your child's health?
|
[SECONDARY
RESEARCH QUESTION] Do children in LAUNCH communities use
more health-care services
such as primary care, mental-health services, preventive
care/screening, dental care, and vision care than those in
comparison communities?
|
|
Parent survey
Q20.
During the past 12 months, how many times did your child see a
doctor, nurse, or other health care provider for preventive
medical care such as a physical exam or well-child checkup?
Q21.
During the past 12 months, did your child see a dentist for any
kind of dental care, including check-ups, dental cleanings,
x-rays, or filling cavities?
Q22.
Has your child ever has your child had his/her vision tested with
pictures, shapes, or letters?
|
[SECONDARY
RESEARCH QUESTION] Do children in LAUNCH communities have better
access to specific health-care services
such as primary care, mental-health services, dental care, and
vision care than those in comparison communities?
|
|
Parent survey
Medical
care
Dental
care
Vision
care
Mental
health services
|
Does
Project LAUNCH’s impact on the individual-level outcomes
in Question 1 vary across the interventions supporting each of
the program’s core
strategies? These
include:
|
Program implementation – core strategies
|
|
home
visiting
|
|
Direct Services survey
|
mental
health consultation in school and ECE settings
|
|
Direct Services survey
Did
you implement any mental health consultation in school and ECE
settings activities during the current reporting period?
Please
provide a brief description of this activity
What
type of activity is this?
How
many times did this activity occur in the past 6 months?
Who
directly participates in this activity?
How
many of these individuals participated in the activity over the
last 6 months?
Who
is intended to benefit from this activity?
If
this activity was intended to help children, what specific age
range of child?
Where
is the activity implemented?
Which
of the following evidence-based or promising practice mental
health consultation models did you implement?
Which
of the following evidence-based or promising practice
social-emotional curricula did you implement? (May be used with
or without MHC)
|
integrating
behavioral health into primary health care
|
|
Direct services survey
|
screening
|
|
Direct Services survey (under integrating behavioral health into
primary care)
|
family
strengthening
|
|
Direct Services survey
Did
you implement any family strengthening activities during the
current reporting period?
Please
provide a brief description of this activity.
What
type of activity is this?
How
many times did this activity occur in the past 6 months?
Who
directly participates in this activity?
How
many of these individuals participated in the activity over the
last 6 months?
Who
is intended to benefit from this activity?
If
this activity was intended to help children, what specific age
range of child?
Where
is the activity implemented?
Which
family strengthening frameworks did you use?
Which
of the following evidence-based or promising practice family
strengthening programs did you implement with parents?
Which
of the following therapeutic interventions did you use?
|
Does
the impact of LAUNCH vary by family
and community socioeconomic factors?
|
Demographic/socioeconomic characteristics
|
|
family
education level?
|
|
Parent survey
|
race/ethnicity?
|
|
Parent survey
Is
your child of Hispanic, Latino/a, or Spanish origin?
What
is your child’s race? (One
or more categories may be selected)
|
income?
|
|
Parent survey
|
Target
children’s age
|
|
Parent survey
|
Do
families in LAUNCH communities fare better on the outcomes
included in Question 1 over time (i.e., as a function of time
since program implementation)?
|
Impact of duration of exposure (i.e., time since implementation)
|
N/A
|
Systems-Level
Outcomes
|
|
|
Are
LAUNCH coalition-building
activities (e.g., wellness
councils) associated with other systems outcomes (such as
advocacy activities, funding levels, and others)?
|
Coalition-building activities
|
Systems Change Assessment Survey
Did
you implement any coalition-building activities during the
current reporting period?
Please
list all of the coalition-building activities you have
implemented during the current reporting period and select the
outcomes these activities were working towards. Please click on
“Add activity” to add new activities to the table and
then select the outcome(s) that the activities were working
towards.
Comparison Sites Interview Protocol
Did
you implement any coalition-building activities related to early
childhood health during the past ___ months?
Please
describe the coalition-building activities you implemented.
For
each activity you described in the question above, what type of
outcome were you working toward by implementing the activity?
|
Are
LAUNCH public information
campaigns associated with
other systems outcomes (such as advocacy activities, funding
levels, and others)?
|
Public information campaigns
|
Systems Change Assessment Survey
Did
you implement any public information campaigns during the current
reporting period?
Please
list all of the public information campaigns you have implemented
during the current reporting period and select the outcomes these
campaigns were working towards. Please click on “Add
activity” to add new activities to the table below and then
select the outcome(s) that the campaigns were working towards.
Comparison Sites Interview Protocol
Did
you implement any public information campaigns related to early
childhood health during the past ___months?
Please
describe the public information campaigns you implemented.
For
each public information campaign you described above, what type
of outcome were you working toward by implementing the campaign?
|
Are
LAUNCH advocacy activities
associated with other
systems outcomes (such as public information campaigns, funding
levels, and others)?
|
Advocacy activities
|
Systems Change Assessment Survey
Did
you implement any advocacy activities during the current
reporting period?
Please
list all of the advocacy activities you have implemented during
the current reporting period and select the outcomes these
activities were working towards. Please click on “Add
activity” to add new activities to the table and then
select the outcome(s) that the activities were working towards.
Comparison Sites Interview Protocol
Did
you implement any advocacy activities related to early childhood
health during the past __ months?
Please
describe the advocacy activities related to early childhood
health and wellness that you implemented.
For
each advocacy activity you described above, what type of outcome
were you working toward?
|
Are
LAUNCH activities that are
designed to increase funding
associated with other systems outcomes (such as public
information campaigns, coalition building, and others)?
|
Sustainability
|
Systems Change Assessment Survey
Did
you implement any activities to build funding during the current
reporting period?
Please
list all of the activities you have implemented to build funding
during the current reporting period and select the outcomes these
activities were working towards. Please click on “Add
activity” to add new activities to the table and then
select the outcome(s) that the activities were working towards.
Comparison Sites Interview Protocol
Did
you implement any activities to build funding during the past ___
months?
Please
describe the activities to build funding that you implemented.
For
each activity to build funding that you described above, what
type of outcome were you working toward?
|
Does
the impact of LAUNCH vary by community
socioeconomic characteristics?
|
Demographic/socio-economic characteristics at the community or
jurisdictional level
|
Source: American Community Survey
|
family
education level
|
|
|
race/ethnicity
|
|
|
income?
|
|
|
What
are the facilitators
that contribute to improving systems-level outcomes?
|
Facilitators of systems outcomes
|
|
Are
LAUNCH communities in areas with co-occurring
child and family programs
more likely than communities without them to achieve improved
systems outcomes?
|
|
Direct Services Survey
Home Visiting
Mental Health Consultation
Integrating Behavioral and
Primary Health Care
Family Strengthening
Other Direct Service Activities
|
Are
LAUNCH communities with schools that emphasize child and family
well-being (either through formal programs or other otherwise)
more likely than communities without them to achieve improved
systems outcomes?
|
Child and family wellbeing program
|
School Survey
|
What
are the barriers
that impede improvement of systems-level outcomes?
|
Barriers to systems outcomes
|
|
Are
LAUNCH communities in states with larger
uninsured populations
less likely to achieve improved systems outcomes?
|
|
Percent of state population with no health insurance.
|
b. Are LAUNCH communities in states with specific fiscal
challenges less likely to achieve improved systems outcomes?
(These may include, for instance, large unemployed populations,
divided government with respect to party control of executive and
legislative branches, and budgetary issues, among others.)
|
|
We will use secondary data sources to explore issues such as:
large
unemployed populations (Source: Bureau of Labor Statistics);
divided
government with respect to party control of executive and
legislative branches (Source: National Conference of State
Legislatures); and
budgetary
issues (Source: National Association of State Budget Officers)
|
What
are the economic costs of LAUNCH Program Implementation and how
do these compare to the economic benefits related to improvements
in child and family outcomes?
|
Economic returns on LAUNCH program investment
|
|
How
much does LAUNCH cost to implement overall and per child in the
designated target population—with the target population
identified as a combination of geography (defined ZIP codes for
inclusion) and demographics (children living beneath a
designated income level)?
|
|
Direct Services Survey
Home Visiting
Mental Health Consultation
Integrating Behavioral and
Primary Health Care
Family Strengthening
Other Direct Service Activities
|
What
percentage of resources on average do grantees spend on each
LAUNCH strategy?
|
|
Direct Services survey
Home Visiting
How
much (in dollars) was spent on this activity in the current
reporting period?
What
percentage of the current grant year’s local LAUNCH funding
was spent on this activity in the current reporting period?
What
percentage of the current grant year’s state LAUNCH funding
was spent on this activity in the current reporting period?
Mental Health Consultation
How
much (in dollars) was spent on this activity in the current
reporting period?
What
percentage of the current grant year’s local LAUNCH funding
was spent on this activity in the current reporting period?
What
percentage of the current grant year’s state LAUNCH funding
was spent on this activity in the current reporting period?
Integrating Behavioral and
Primary Health Care
How
much (in dollars) was spent on this activity in the current
reporting period?
What
percentage of the current grant year’s local LAUNCH funding
was spent on this activity in the current reporting period?
What
percentage of the current grant year’s state LAUNCH funding
was spent on this activity in the current reporting period?
Family Strengthening
How
much (in dollars) was spent on this activity in the current
reporting period?
What
percentage of the current grant year’s local LAUNCH funding
was spent on this activity in the current reporting period?
What
percentage of the current grant year’s state LAUNCH funding
was spent on this activity in the current reporting period?
Other Direct Service Activities
How
much (in dollars) was spent on this activity in the current
reporting period?
What
percentage of the current grant year’s local LAUNCH funding
was spent on this activity in the current reporting period?
What
percentage of the current grant year’s state LAUNCH funding
was spent on this activity in the current reporting period?
|
If
LAUNCH resulted in changes in child and family outcomes, what
are published estimates of the dollar value of these
improvements?
|
|
See survey items listed under 12)a (activity cost)
|
What
is the total cost of LAUNCH per child in the designated target
population after accounting for these cost offsets?
What
is the cost per child served by LAUNCH?
What
is the cost per child who saw developmental improvement?
If
the cost offsets exceed the implementation costs of LAUNCH,
what is the return on investment (ROI) of LAUNCH with ROI
defined as:
|
|
See survey items listed under 1) a and b (developmental
improvement)
See survey items listed under 12)a
(activity cost)
The
MSE team’s current plan is to
conduct a Social Return on Investment (SROI) analysis consistent
with the principles of the Global Social Venture Competition
established by Columbia University’s Business School. This
framework involves the following steps: (1) defining the social
value proposition of the LAUNCH program; (2) articulating the
means to quantify the costs and, more important, the three to five
main expected benefits of LAUNCH; and (3) monetizing the intended
social value of these benefits and costs. In applying this
framework, we will combine documented evidence on the effects of
improvements in different childhood outcomes on long term
outcomes. As part of our analysis, we will consider the extent to
which changes in socioemotional health have been demonstrated to
translate into higher graduation rates and educational attainment,
higher earned income, increased productivity for parents during
childhood, and decreased stress, among other outcomes. The value
placed on these outcomes will be informed by published literature
on, for examples, the relationship between childhood behavior
problems and dropping out of school (Robst,
J., & Weinberg, C., 2010) and
the relationship between educational attainment and adult incomes
(Blanden, J., Gregg, P., and Macmillan, L., 2007). We estimate the
incremental change in net benefits and the incremental change in
net costs compared to what costs and benefits would be if the
intervention had not taken place. The ratio of costs to benefits
is the number of dollars in benefit returned for each dollar in
cost invested. If the number is greater than 1.0, then the net
benefit exceeds costs.
There
are three primary challenges to conducting an SROI: (1) measuring
implementation costs; (2) measuring incremental changes in
benefits; and (3) converting non-monetary benefits into their
monetary equivalents. For the LAUNCH evaluation, we propose
to estimate costs using an activity-based costing (ABC) approach
that involves collecting data on categories of activities, and the
labor hours, facilities, supplies, capital, and other costs
required to carry out that activity.
We will collect data from
each LAUNCH program using a web-based data collection instrument
and through interviews with program staff conducted during site
visits. We will assess changes in incremental net benefits
of the program based on the quasi-experimental data analysis
indicated above, programmatic LAUNCH records, and educated
assumptions when necessary. We will assign monetary costs to
these benefits based on collaborative methods outlined by the
Social Return on Investment Network, through contingent valuation
and willingness-to-pay studies when available, and through market
equivalent prices of the benefits, when available.
|