Attach1_SFTXT Planning Matrix_12062012

Attach1_SFTXT Planning Matrix_12062012.docx

The National Cancer Institute (NCI) SmokefreeTXT (Text Message) Program Evaluation

Attach1_SFTXT Planning Matrix_12062012

OMB: 0925-0676

Document [docx]
Download: docx | pdf














SFTXT evaluation planning matrix

Attachment 1




Evaluation Planning Matrix

Evaluation questions

Category

Construct

Data sources

  • Were the eligibility criteria a barrier to participation among those indicating interest?

Methods: Recruitment

Eligibility criteria

  • Screener

  • Did those enrolled in the study differ from those who were not enrolled?

Methods: Recruitment

Non-response bias

  • Exit survey

  • Of those enrolled, how many complete each of the follow-up surveys? Are participants who drop out different than participants who do not? Are there particular questions participants do not answer? What proportion of incentives are claimed? Are follow-up strategies effective in converting non-responders to responders?

Methods: Attrition

Attrition

  • Baseline survey

  • Quit-date survey

  • 12week survey

  • 24 week survey

  • Incentive claims

  • What are the key intervention strategies/methods?

Intervention

Intervention components

  • Communication with NCI/MMG

  • Smokefree txt Theory Report”

  • SMS message mapping document

  • Smokefree Teen A Theory-Informed Design” document

  • How engaged were participants with SMS content?

  • How many times did they reply to messages?

  • How many key word messages did they send?

  • How many times did they click on a link in the text message?

  • # of undeliverable messages

  • # of declined messages (STOP)

  • When did they opt out of the program?


Exposure/

Engagement

Engagement with SMS

  • SMS metrics



  • Do those who received the full program of text messages in (arm 3) recall receiving more messages than those in arms 1 (assessment only) and 2 (assessment + countdown messages)?

  • Do those in arm 2 recall receiving more messages than those in arm 1?

Exposure/

Engagement

Recall of text messaging

  • 1 week survey

  • End of treatment survey


Evaluation Planning Matrix (continued)

Evaluation questions

Category

Construct

Data sources

  • How useful did respondents think Quittxt messages were in helping them stay smokefree, overcome cravings, etc. ?

  • Were any specific types of messages perceived as more useful than others (quit date reminders; mood assessments; smokefree reminders, etc.)?

  • Overall how useful did participants perceive the program to be?


Exposure/

Engagement

Perceived utility

  • 1 week survey

  • End of treatment survey

  • Does risk perception moderate the effects of the intervention?

Risk perception

Risk perceptions

  • Baseline


  • Are those who received the full program of text messaging (arm 3) less likely to predict that they will be smoking one year from now than those who only received assessment messages (arm 1) or assessment and countdown messages (arm 2)?

  • Are those exposed to assessment only messages plus countdown messages (arm 2) less likely to predict that they will be smoking one year from now than those who received assessment messages only (arm 1)?

Proximal outcomes

Forecasted future smoking

  • Baseline

  • End of treatment survey

  • 12 week survey




Evaluation Planning Matrix (continued)


Evaluation questions

Category

Construct

Data sources

  • Did exposure to the full text program of text messaging (arm 3) result in increased self-efficacy toward cessation initiation?

  • Did exposure to assessment only messages plus countdown messages (arm 2) result in increased self-efficacy toward cessation initiation than assessment messages only (arm 1)?

  • Did those who were more engaged with SMS have higher self-efficacy toward cessation initiation?

  • Did exposure to the full program of text messaging (arm 3) result in increased self-efficacy toward cessation maintenance compared to the other groups)?

  • Did exposure to assessment only messages plus countdown messages (arm 2) result in increased self-efficacy toward cessation maintenance than assessment messages only (arm 1)?

  • Did those who were more engaged with the program have higher self-efficacy toward cessation maintenance than those who were less engaged?

Proximal outcomes

Self-efficacy



  • Baseline (most items)

  • 6 week (1 item)

  • 12 week survey (1 item)

  • 24 week survey (1 item)





  • Did exposure to the intervention result in change in positive expectancies for smoking?

  • Did the full program of messages (arm 3) result in greater changes in positive expectancies than assessment only messages (arm 1) or assessment and countdown messages (arm 2)?

  • Did assessment and countdown messages (arm 2) result in greater changes in positive expectancies than assessment only messages (arm 1)?

Proximal outcomes

Smoking expectancies (i.e., costs and benefits of smoking)

  • Baseline

  • End of treatment



Evaluation Planning Matrix (continued)

Evaluation questions

Construct

Data sources

  • Did the full program of text messaging (arm 3) result in higher motivation to quit than assessment only messages (arm 1) or assessment and countdown messages (arm 2)?

  • Were those who were more engaged with SMS more likely to develop motivation?

  • Did motivation/salience of cessation moderate the effects the intervention?

Motivation/salience of cessation

  • Baseline

  • 6 week survey

  • 12 week survey

  • 24 week survey

  • Did full program of text messaging (arm 3) result in more skills development than assessment only messages (arm 1) or assessment and countdown messages (arm 2)?

  • Did assessment and countdown messages (arm 2) result in more skills development than assessment only messages (arm 1)?

  • Were those who were more engaged with SMS more likely to develop skills?

  • Were those who were more engaged with web and SMS more likely to develop skills?

  • Did skills/abilities moderate the effects the intervention?

Skills/abilities (manage cravings; emotional coping responses; develop quit plan)

  • 6 week survey

  • 12 week survey

  • 24 week survey

  • Did those who received the full program of text messaging (arm 3) have more extra-treatment social support than those who received assessment only messages (arm 1) or assessment and countdown messages (arm 2)?

  • Did assessment and countdown messages (arm 2) result in more extra-treatment social support than assessment only messages (arm 1)?

  • Were those who were more engaged with SMS more likely to receive extra-treatment social support?

  • Did extra-treatment social support moderate the effects the intervention?

Extra-treatment social support

  • Baseline

  • 6 week survey

  • 12 week survey

  • 24 week survey




Evaluation Planning Matrix (continued)

Evaluation questions

Construct

Data sources

  • Are those who received the full program of text messaging (arm 3) more likely to quit than those who received assessment only messages (arm 1) or assessment and countdown messages (arm 2)?

  • Are those exposed to assessment only messages plus countdown messages (arm 2) more likely to quit than those who received assessment messages only (arm 1)?


Current smoking behavior (cessation)

  • 1 week survey

  • 6 week survey

  • 12 week survey

  • 24 week survey

  • Are those who received the full program of text messaging (arm 3) more likely to reduce the number of cigarettes they smoke than those who received assessment only messages (arm 1) or assessment and countdown messages (arm 2)?

  • Are those exposed to assessment only messages plus countdown messages (arm 2) more likely to reduce the number of cigarettes they smoke than those who received assessment messages only (arm 1)?

Reduction in rate/ frequency of smoking (# of non-smoking days, # of cpd)




  • Baseline

  • 1 week survey

  • 6 week survey

  • 12 week survey

  • 24 week survey

  • Text message responses

  • Are those who received the full program of text messaging (arm 3) more likely to stay quit than those who received assessment only messages (arm 1) or assessment and countdown messages (arm 2)?

  • Are those exposed to assessment only messages plus countdown messages (arm 2) more likely to stay quit than those who received assessment messages only (arm 1)?

Abstinence

  • Baseline

  • 1 week survey

  • 6 week survey

  • 12 week survey

  • 24 week survey

  • Text message responses



  • Are those who received the full program of text messaging (arm 3) less likely to relapse than those who received assessment only messages (arm 1) or assessment and countdown messages (arm 2)?

  • Are those exposed to assessment only messages plus countdown messages (arm 2) less likely to relapse than those who received assessment messages only (arm 1)?

Relapse (return to regular smoking after an established period of abstinence)


Treatment Failure (failure to establish an initial period of abstinence)

  • 1 week survey

  • 6 week survey

  • 12 week survey

  • 24 week survey

  • Did environmental factors moderate the effects of the intervention?

Environment (household smoking ban; access in the home)

  • Baseline survey

Evaluation Planning Matrix (continued)



Evaluation questions

Construct

Data sources




  • Did socio-demographics moderate the effects of the intervention?

Socio-demographics (age, gender, race/ethnicity, SES)

  • Baseline survey

  • Did psychosocial factors moderate the effects of the intervention?

Psychosocial factors (mental health, negative mood regulation, perceived stress)

  • Baseline survey

  • 6 week survey

  • Did smoking history moderate the effects of the intervention?

Smoking history (age at first cigarette, quit attempts, etc.)

  • Baseline survey

  • Did smoking patterns at baseline moderate the effects of the intervention?

Current smoking patterns

  • Baseline

  • Did use of technology moderate the effects of the intervention?

Technology use patterns

  • Baseline survey

  • Quit Process Measures

Withdrawal symptoms, cravings

  • 1 week survey


1


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
Authorbkelly
File Modified0000-00-00
File Created2021-01-29

© 2024 OMB.report | Privacy Policy