Download:
docx |
pdf
SFTXT evaluation planning matrix
Attachment 1
Evaluation Planning Matrix
Evaluation questions
|
Category
|
Construct
|
Data sources
|
|
Methods: Recruitment
|
Eligibility criteria
|
|
|
Methods: Recruitment
|
Non-response bias
|
|
Of those
enrolled, how many complete each of the follow-up surveys? Are
participants who drop out different than participants who do not?
Are there particular questions participants do not answer? What
proportion of incentives are claimed? Are follow-up strategies
effective in converting non-responders to responders?
|
Methods: Attrition
|
Attrition
|
Baseline
survey
Quit-date
survey
12week
survey
24
week survey
Incentive claims
|
|
Intervention
|
Intervention components
|
Communication
with NCI/MMG
“Smokefree
txt Theory Report”
SMS
message mapping document
“Smokefree
Teen A Theory-Informed Design” document
|
How
engaged were participants with SMS content?
How
many times did they reply to messages?
How
many key word messages did they send?
How
many times did they click on a link in the text message?
#
of undeliverable messages
#
of declined messages (STOP)
When
did they opt out of the program?
|
Exposure/
Engagement
|
Engagement with SMS
|
|
Do
those who received the full program of text messages in (arm 3)
recall receiving more messages than those in arms 1 (assessment
only) and 2 (assessment + countdown messages)?
Do those in arm 2
recall receiving more messages than those in arm 1?
|
Exposure/
Engagement
|
Recall of text messaging
|
1
week survey
End of treatment
survey
|
Evaluation Planning Matrix (continued)
Evaluation questions
|
Category
|
Construct
|
Data sources
|
How
useful did respondents think Quittxt messages were in helping
them stay smokefree, overcome cravings, etc. ?
Were
any specific types of messages perceived as more useful than
others (quit date reminders; mood assessments; smokefree
reminders, etc.)?
Overall
how useful did participants perceive the program to be?
|
Exposure/
Engagement
|
Perceived utility
|
1
week survey
End of treatment
survey
|
|
Risk perception
|
Risk perceptions
|
|
|
Proximal outcomes
|
Forecasted future smoking
|
Baseline
End
of treatment survey
12 week survey
|
Evaluation Planning Matrix (continued)
|
Evaluation questions
|
Category
|
Construct
|
Data sources
|
Did
exposure to the full text program of text messaging (arm 3)
result in increased self-efficacy toward cessation initiation?
Did
exposure to assessment only messages plus countdown messages (arm
2) result in increased self-efficacy toward cessation initiation
than assessment messages only (arm 1)?
Did
those who were more engaged with SMS have higher self-efficacy
toward cessation initiation?
Did
exposure to the full program of text messaging (arm 3) result in
increased self-efficacy toward cessation maintenance compared to
the other groups)?
Did
exposure to assessment only messages plus countdown messages (arm
2) result in increased self-efficacy toward cessation maintenance
than assessment messages only (arm 1)?
Did
those who were more engaged with the program have higher
self-efficacy toward cessation maintenance than those who were
less engaged?
|
Proximal outcomes
|
Self-efficacy
|
Baseline
(most items)
6
week (1 item)
12
week survey (1 item)
24
week survey (1 item)
|
Did
exposure to the intervention result in change in positive
expectancies for smoking?
Did
the full program of messages (arm 3) result in greater changes in
positive expectancies than assessment only messages (arm 1) or
assessment and countdown messages (arm 2)?
Did
assessment and countdown messages (arm 2) result in greater
changes in positive expectancies than assessment only messages
(arm 1)?
|
Proximal outcomes
|
Smoking expectancies (i.e.,
costs and benefits of smoking)
|
Baseline
End of treatment
|
Evaluation Planning Matrix (continued)
Evaluation questions
|
Construct
|
Data sources
|
Did
the full program of text messaging (arm 3) result in higher
motivation to quit than assessment only messages (arm 1) or
assessment and countdown messages (arm 2)?
Were
those who were more engaged with SMS more likely to develop
motivation?
Did
motivation/salience of cessation moderate the effects the
intervention?
|
Motivation/salience of
cessation
|
Baseline
6
week survey
12
week survey
24 week survey
|
Did
full program of text messaging (arm 3) result in more skills
development than assessment only messages (arm 1) or assessment
and countdown messages (arm 2)?
Did
assessment and countdown messages (arm 2) result in more skills
development than assessment only messages (arm 1)?
Were
those who were more engaged with SMS more likely to develop
skills?
Were
those who were more engaged with web and SMS more likely to
develop skills?
Did skills/abilities
moderate the effects the intervention?
|
Skills/abilities (manage
cravings; emotional coping responses; develop quit plan)
|
6
week survey
12
week survey
24 week survey
|
Did
those who received the full program of text messaging (arm 3)
have more extra-treatment social support than those who received
assessment only messages (arm 1) or assessment and countdown
messages (arm 2)?
Did
assessment and countdown messages (arm 2) result in more
extra-treatment social support than assessment only messages (arm
1)?
Were
those who were more engaged with SMS more likely to receive
extra-treatment social support?
Did extra-treatment
social support moderate the effects the intervention?
|
Extra-treatment social
support
|
Baseline
6
week survey
12
week survey
24
week survey
|
Evaluation Planning Matrix (continued)
Evaluation questions
|
Construct
|
Data sources
|
|
Current smoking behavior
(cessation)
|
6
week survey
12
week survey
24 week survey
|
|
Reduction in rate/ frequency
of smoking (# of non-smoking days, # of cpd)
|
Baseline
1
week survey
6
week survey
12
week survey
24
week survey
Text message
responses
|
|
Abstinence
|
Baseline
1
week survey
6
week survey
12
week survey
24
week survey
Text message
responses
|
|
Relapse (return to regular
smoking after an established period of abstinence)
Treatment
Failure (failure to establish an initial period of abstinence)
|
1 week survey
6 week survey
12 week survey
24
week survey
|
|
Environment (household
smoking ban; access in the home)
|
|
Evaluation Planning Matrix (continued)
|
|
|
Evaluation questions
|
Construct
|
Data sources
|
|
|
|
|
Socio-demographics (age,
gender, race/ethnicity, SES)
|
|
|
Psychosocial factors (mental
health, negative mood regulation, perceived stress)
|
Baseline
survey
6 week survey
|
|
Smoking history (age at
first cigarette, quit attempts, etc.)
|
|
|
Current smoking patterns
|
|
|
Technology use patterns
|
|
|
Withdrawal symptoms,
cravings
|
|
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
Author | bkelly |
File Modified | 0000-00-00 |
File Created | 2021-01-29 |