Supporting Statement B For:
Evaluation of Cancer Control Leadership Forums at the
Center for Global Health (CGH) (NCI)
August 15, 2014
Brenda Kostelecky
Center for Global Health
National Cancer Institute
9609 Medical Center Dr.
RM 3W276
Rockville MD, 20850
Telephone: 240-276-5585
Email: brenda.kostelecky@nih.gov
Table of Contents
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 1
B.1 Respondent Universe and Sampling Methods 1
B.2 Procedures for the Collection of Information 2
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 3
B.4 Test of Procedures or Methods to be Undertaken 4
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 4
List of Attachments
Attachment 1: Evaluation Surveys
1A. Leadership Forum 3 month Pre-Assessment
1B. Leadership Forum 3 month
1C. Leadership Forum 6 month
1D. Leadership Forum 12 month
1E. Leadership Forum 24 month
Attachment 2: Privacy Act Memo
Attachment 3: Office of Human Subjects Research Protection Exemption (OHSRP)
Attachment 4: Leadership Forum Invitations
A total of 54 participants (18/year)1 will be invited to evaluate the Cancer Control Leadership Forums. These team leads have been identified through their participation as a country team lead in the Cancer Control Leadership Forums.
Table 1. Cancer Control Leadership Forum
CGH Workshop |
Workshops /Year |
Respondents /Workshop |
Number of Surveys/ Workshop |
Survey Intervals |
Cancer Control Leadership Forum |
3 |
6 |
3 months pre-, and 3, 6, 12, 24 months post-workshop |
Since the purpose of these evaluations is to understand whether or not these Forums been successful in meeting their goals, the country team leads will be invited to evaluate the Forum that they participated in. No statistical sampling methodology will be used. This is because much of the data that will be collected is qualitative (responses to open-ended questions), and the number of respondents in the evaluation is too low in order to draw statistical conclusions.
The Leadership Forums are primarily designed to assist in the development and implementation of national cancer control plans. For each Forum, CGH works with a six person in-country team to design, implement and follow-through with the action items and cancer control activities identified as priorities during the workshop. This country team consists of key members of the in-country Ministry of Health who are responsible for cancer control and prevention. Thus, this team is primarily responsible for carrying out the action plan designed during the workshop. The Leadership Forums will distribute the link to a pre-workshop assessment via email, which can be completed and submitted electronically by the country team leads 3 months prior to the workshop in order to gather baseline data. Then evaluations will be qualitative interviews held with the six country team leads at each of the four collection points (3, 6, 12, and 24 months post-workshop). By choosing to use qualitative interviews (and not surveys) with only the country team leads for this evaluation, CGH expects to get richer data regarding the outcomes of the workshop since they are the most invested in the workshop and are responsible for the actual outcomes.
Out of the 18 workshop participants invited to participate in these evaluations, we expect 16 to agree to participate. The overall response rate is expected to be approximately 90% (16/18).
The members of the country teams will receive an email from their Country Team Facilitator at each designated collection point (3, 6, 12 and 24 months post-workshop with the exception of the Pre-Forum Country Team Assessment) asking them to set up a time for the interview (Attachment 4). The country team leads will respond via email to set-up the interview. Although these interviews will coincide with the technical assistance that CGH is providing to the country teams, and the responses will be used to tailor the assistance to each member, each team member can choose whether or not to participate in the evaluation. For those who choose not to participate in the evaluation, NCI will provide technical assistance for all areas that they request. If the country team members do not respond to set up an interview, then the Leadership Forum Lead will send a follow-up email requesting an interview (Attachment 4). Trained interviewers will then complete the interviews with the country team leads and transcribe the interviews for analysis.
For the Pre-Forum Country Team Assessment, 3 months prior to the workshop, the Leadership Forum Lead will send an email inviting each Country Team Lead to complete an online survey that assess the status of cancer control planning and implementation in their country. If they do not respond within 1 week, the Leadership Forum Lead will send another email inviting them to complete the assessment (Attachment 4). The invitation will also include a link to the evaluation (Attachment 1A) that allows for easy completion and electronic submission. All documents that the respondent receives will be written in plain and clear language. Participants who choose to participate will choose click the provided link and complete the evaluation and then choose “submit” (Attachment 1A). All the participants that will be invited to complete the evaluation are fluent in English, thus it will only be distributed in English.
B.2.1. Quality Control
The program leads will review all returned evaluations and interview transcripts. If quality issues arise during the interviews, then the program leads will work with the interviewers to ensure that they are collecting and recording complete and accurate responses. The program leads will also monitor response rates.
The Evaluation of the Center for Global Health's Workshops expects to achieve a response rate of 90%, determined by the actual number of respondents divided by the total number expected (16/18). The initial email inviting the country team leads to participate will be sent by the Country Team Facilitator. The Leadership Forum Lead will follow up non-response to the initial contact (defined as within 7 days of the initial email) by emailing the country team leads again with the same letter of invitation (Attachment 4). Having the invitations come directly from the Leadership Forum Country Facilitators and Leads rather than a central mailbox should increase response, since these program leads are integral to the planning and implementation of these Forums and should be either familiar or known to participants. Additionally, the information provided by pre-assessment will be used to tailor the Forum to the needs of each individual country team, and the post Forum interviews will be used to tailor the technical assistance provided by CGH to country teams, thus the country team leads should be very motivated to participate.
Three country team leads from a past Forum was chosen and these three participants were asked to complete the evaluations in April, 2014 (one completed the baseline assessment, one the 3 months post assessment, and one the 12 months post assessment). The main goal of this pre-test was to validate the appropriateness of the questions. We wanted to ensure that the respondents were able to provide the data without considerable burden and that the sample questions were understandable. The intent and purpose of the evaluation was also discussed with the pre-tested respondents. Participant suggestions to improve clarity were incorporated into the survey design.
Various members of CGH’s staff were critical in developing the research plan, the conceptual framework, survey questions, and sampling strategies underlying evaluation of the survey, particularly: Dr. Sudha Sivaram, Ms. Cathy Muha, Ms. Rebecca Minneman, Ms. Catherine Hidalgo, and Ms. Renicha McCree. Many of the same individuals will be involved with analysis once the data are collected. Additionally, NCI’s Office of Science Planning and Assessment were consulted in the process of designing these evaluations.
1 There will be 6 respondents per Forum and 3 Forums per year, totaling 18 respondents annually. These 18 respondents will be asked to complete all five of the information collections.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | TABLE OF CONTENTS |
| Author | Vivian Horovitch-Kelley |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |