Supporting Statement A For:
Evaluation of Cancer Control Leadership Forums at the
Center for Global Health (CGH) (NCI)
November 12, 2014
Brenda Kostelecky
Center for Global Health
National Cancer Institute
9609 Medical Center Dr.
RM 3W276
Rockville MD, 20850
Telephone: 240-276-5585
Email: brenda.kostelecky@nih.gov
A. Justification 1
A.1 Circumstances Making the Collection of Information Necessary 1
A.2 Purpose and Use of the Information 4
A.3 Use of Improved Information Technology and Burden Reduction 6
A.4 Efforts to Identify Duplication and Use of Similar Information 7
A.5 Impact on Small Businesses or Other Small Entities 7
A.6 Consequences of Collecting the Information Less Frequently 7
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 9
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 9
A.9 Explanation of Any Payment or Gift to Respondents 9
A.10 Assurance of Confidentiality Provided to Respondents 10
A.11 Justification for Sensitive Questions 10
A.12 Estimates of Annualized Burden Hours and Costs 10
A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 12
A.14 Annualized Cost to the Federal Government 13
A.15 Explanation for Program Changes or Adjustments 13
A.16 Plans for Tabulation and Publication and Project Time Schedule 13
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 15
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 15
Attachment 1: Evaluation Surveys
1A. Leadership Forum 3 month Pre-Assessment
1B. Leadership Forum 3 month
1C. Leadership Forum 6 month
1D. Leadership Forum 12 month
1E. Leadership Forum 24 month
Attachment 2: Privacy Act Memo
Attachment 3: Office of Human Subjects Research Protection Exemption (OHSRP)
Attachment 4: Leadership Forum Invitations
This submission is a request for OMB to approve the Evaluation of the Cancer Control Leadership Forums at the Center for Global Health (CGH) for three years. These workshops are organized and funded by the National Cancer Institute's CGH in conjunction with various partners ranging from foreign Ministries of Health and research institutions, to international non-governmental organizations (NGOs) and U.S. academic institutions. The goal of the U.S. National Cancer Institute (NCI) Cancer Control Leadership Forums is to increase the capacity of participating countries to initiate or enhance cancer control planning and implementation in their respective countries. The Forums are an opportunity for countries to exchange experiences and ideas about creating and implementing comprehensive cancer control plans. The proposed evaluation requests information about the outcomes of the forums including 1) status of cancer control planning and implementation in each participating country, 2) outcomes related to the action plans (e.g. developing written materials, completion of action items, resources and support acquired), 3) successes and challenges related to the action plans, and 4) new cancer control partnerships and networks. Baseline information regarding the status of cancer control planning and implementation will be collected 3 months prior to the Forums in order to inform the development of each Forum. The evaluation information will be collected 3-24 months after each forum and is needed to evaluate the effectiveness of these workshops in order to inform future programming and funding decisions.
The National Cancer Institute's (NCI) Center for Global Health (CGH) has been mandated to "develop an appropriate research strategy to help incorporate cancer control into global health programs; foster relevant research activities throughout the NCI’s own extramural and intramural divisions; and work closely with the many potential collaborators who have displayed an interest in shared objectives.” 1 Incorporating cancer control is a key component of the mandate that necessitates this information collection. Cancer control has been broadly defined by the NCI as a set of activities that bring discoveries in cancer research to the population level. In global health, this gap of tailoring the evidence in cancer prevention and screening and delivering them to the populations and communities in need is stark and needs to be filled. As such, while NCI is a research agency and will not provide cancer care or orchestrate prevention campaigns directly, we will work with a wide range of partners in providing information and training about cancer control. This work will include activities “on topics that are highly relevant to the pragmatic aspects of treatment and prevention in developing countries: the geographic and cultural patterns of disease, the organization and function of health care systems, and the monitoring of the effectiveness of cancer control strategies. We also recognize that many kinds of cancers appear at different rates in different parts of the world for different reasons— and it is important to explain these differences in order to reduce the cancer burden in all countries. Finally, long-standing improvements in the control of cancer throughout the world will require the training of medical and scientific personnel who have vested interests in improving health in their own countries."2
With this in mind, CGH has developed the Cancer Control Leadership Forums to increase the capacity of participating countries to initiate or enhance cancer control planning and implementation in their respective countries. National Cancer Control Plans comprise an important part of a country's non-communicable disease (NCD) plan and can help countries meet NCD targets outlined in the WHO Global NCD Action Plan. The Forum is an opportunity for countries to exchange experiences and ideas about creating and implementing comprehensive cancer control plans.
The evaluations of the Forums are authorized by Section 410 of the Public Health Service Act (42 USC § 285), which authorizes collection of this information, as outlined in Special Authorities of the Director – Sec. 413. [285a-2]. Section 413 authorizes the NCI Director to collect and disseminate (including through publications) to clinicians and the general public information on cancer research, diagnosis, prevention and treatment.
Each Forum consists of 4-6 countries from a region that have been invited to participate. The program leads identify a Country Team Lead for each country. Together the Country Team Leads work with program leads to develop a country team which is comprised of 6-8 in-country partners ranging from representatives of: 1) Government agencies involved in cancer and non-communicable disease efforts, 2) Non-governmental organizations and other private organizations with an interest in cancer, 3) Cancer institutes, and 4) Academia. While all members of each of the 4-6 country teams attend the Forum, the Country Team Leads will be the only ones who are asked to evaluate the Forum.
The participation of the in-country partners on the country teams increase the relevance, utility and sensitivity of these workshops, while maximizing the benefits for all parties without requiring additional U.S. investment. Yet, despite the significant partner contributions, CGH plays a key role in developing the content of, organizing and funding these workshops and makes significant investments of time, energy, and resources in order to support these workshops.
The expected outcomes for each Forum country team are: 1) Increased awareness of the importance of national cancer control planning and implementation, 2) Enhanced understanding of how to develop and implement a national cancer control plan, and 3) Development of a written action plan with specific tasks to initiate or enhance each country’s national cancer control planning and implementation efforts. The action plan developed at the Forum differs from the countries’ National Cancer Control Plans. The action plan is designed to prioritize target areas for progress toward the ultimate goal of developing, implementing or strengthening countries’ National Cancer Control Plans. Each country team will develop their own action plan that aims at accomplishing those outlined priorities within 12-months of the Forum delivery.
Purpose of the Information
The Forums consist of 2½ day meetings with large group presentations and interactive discussions, along with country team facilitated discussions and action planning sessions. Presentations will include practical guidance and actual examples of how other countries have developed and implemented cancer control plans. Presentations and discussions will be supplemented with content specific tools and resources.
Table 1 summarizes the number of respondents, number of surveys per workshop, number of workshops per year and the survey intervals.
Table 1. Cancer Control Leadership Forum
CGH Workshop |
Workshops /Year |
Respondents /Workshop |
Number of Surveys/ Workshop |
Survey Intervals |
Cancer Control Leadership Forum |
3 |
6 |
5 |
3 months pre-, and 3, 6, 12, 24 months post-workshop |
The current proposal is to administer a survey (Attachments 1A) to help inform the
development of each Forum and 4 phone interviews (Attachments 1B-1E) to evaluate the effectiveness of the Forums in order to inform future programming and funding decisions. The country team leads, will be invited to complete the survey and participate in the phone interviews. The proposed evaluation requests information about the outcomes of the forums including:
Status of cancer control planning and implementation in each participating country,
Outcomes related to the action plans (e.g. developing written materials, completion of action items, resources and support acquired),
Successes and challenges related to the action plans, and
New cancer control partnerships and networks.
Baseline information regarding the status of cancer control planning and implementation will be collected 3 months prior to the Forums in order to inform the development of each Forum. The post-Forum evaluations (above outcomes #2, 3 and 4) will be collected 3 to 24 months after each forum and will be used to assess abilities of countries to implement cancer control programs, inform content and delivery of future forums and to systematically evaluate CGH’s contribution. Although CGH and NCI do not currently have any formal evaluation components for the Leadership Forums, program performance will be assessed as part of deliberations within CGH on the continuation and expansion of these Forums. This evaluation will aid in the analysis of program effectiveness and efficiency in achieving its objectives. For example, it may aid in the identification of areas that could benefit from increased efficiencies or shared activities, as well as inform other aspects of future programs (including types of activities, participants, focus areas, etc.).
The assessment consists of information that is either already known to the participant or is known to the participant’s country team members, which has not previously been gathered and submitted to the program office or made public.
Review and Use of Submitted Information
Completion of the proposed evaluation is of great importance in building and sustaining international partnerships in cancer research and control. The evaluation is intended to provide information on how the Forums improved/facilitated the participants' work in the areas of: cancer control planning and implementation, and development of partnerships and dissemination of information. This evaluation will help us to identify the accomplishments of the participants that are either completely or partially due to their participation in the Forums. This will allow program staff to have a complete understanding of effectiveness of the Forums. Additionally, the evaluation will distinguish the achievements of U.S. participants from their international collaborators, allowing both the U.S. and international governments to understand the value that each program brings to their country. Finally, it can serve as evidence to inform decisions by CGH, NCI, and other institutes across the NIH, and international governments, as to whether they should contribute to similar programs in the future.
Evaluation of outcomes is necessary to ensure that participants are utilizing the resources and skills gained from these Forums and that the goals of the Forums are being met. Evidence that program goals aren’t being met (e.g., lack of new partnerships, not completing action items, etc.) may be used by program staff to initiate discussions with country team leads on how they can support the country team in the implementation of their country plan, and how they can adjust the workshop agendas, speakers, activities, participants etc. to improve performance. Completing the evaluation three to twenty-four months after each workshop allows participants time to apply the resources and information provided by the workshops into their cancer control efforts, allowing CGH to measure the effectiveness and utility of these Forums, while still allowing any adjustments to be made relatively quickly in order to prevent serious shortcomings in future workshops. Strong performance by participants is used to inform best practices and identify areas that could benefit from shared activities.
For the baseline assessment that will be completed electronically (Attachment 1A) a link to the assessment will be sent via email to the country team leads. No automated or dedicated IT system will be used for these reports.
For those evaluations that will be completed via telephone (Attachments 1B-1E) the participants will be contacted via telephone and their answers will be recorded on an internal NIH drive. No automated or dedicated IT system will be used for these reports.
A consultation with the NCI Privacy Act Coordinator will be conducted to determine whether a Privacy Impact Assessment (PIA) is necessary for this project. If so, then a PIA will be completed.
The evaluation proposed here will not duplicate any existing information collection, since this information is not currently collected. Each collection is tailored to the time point at which the data is being collected (Attachments 1A-1E). This evaluation will elicit feedback on the outcomes of the Forums as well as outcomes such as capacity building that is measured across all CGH workshops. This will provide CGH the ability to quantify particular outcomes across workshops, as well as ensure workshop specific feedback is obtained.
None of the information to be collected is publicly available and it cannot be gathered from other sources. The program leads for the Forums have confirmed that they do not currently collect any of these data. Additionally, CGH leadership, partner organizations and collaborators have been consulted and also confirmed that none of this data is currently being collected. Since the participants of these Forums are usually not NCI or NIH grantees, the OMB No. 0925-0002, Expiration Date 8/31/2015, post-award grantee progress reports does not apply to them. For any participants who incidentally have NCI or NIH grants, these progress reports will only collect information on the grant, not on the workshop in which the grantee participated in, thus no duplicate information will be collected.
No small businesses or other small entities will be involved in this information collection.
The Cancer Control Leadership Forums will have a total of 5 data collections per workshop. The first collection will be a baseline data assessment, 3 months prior to the workshop, allowing the program leads to determine country, stakeholder, and participant needs in order to ensure that the workshop is as effective and relevant as possible. This will be completed by emailing the country team a link to the evaluation and having them complete it and submit it electronically. After the workshop is held, CGH provides on-going technical assistance to the country team through regular calls 3, 6, and 12 months after the workshop. In conjunction with providing this technical assistance to the country team, the team will be invited to evaluate the workshop through in-depth interviews. Collecting information on the outcomes across three different time points allows program leads to monitor progress, assess barriers and facilitators to workshop goals, needs for technical assistance, and provide support in order to improve outcomes. This will allow program leads to identify trends as the collaborations formed at these workshops mature. For example, it is possible that the challenges identified in the first collection, three months after the workshop is completed, may be overcome by 12 months after the workshop, but it is still very important to identify those challenges in order to be able to tailor technical assistance for the country teams and minimize them in future workshops. Similarly, some of the same challenges may be identified at both time points, which will indicate the scope and importance of these challenges for awardees to program leads, which, in turn will help program leads to prioritize changes for future workshops. Finally, collecting information on the final outcomes of the workshop 24 months after the workshop is complete is important because it allows the program leads to judge the effectiveness of the workshop and technical assistance that was provided with respect to the long term goals of the workshop which include the development of a written action plan with specific tasks to initiate or enhance each country’s national cancer control planning and implementation efforts. In the absence of the information provided by these evaluations, the program officers will not be able to judge the effectiveness of the programs and make informed decisions regarding funding future programs.
None of the special circumstances relating to the guidelines of 5 CFR 1320.5 applies to this information collection, and the proposed guidelines fully comply with 5 CFR 1320.5.
The 60-Day Federal Register Notice soliciting comments on this study prior to initial submission to OMB was published on July 15, 2014, Vol. 79, P. 41295. One public comment was received on July 17, 2014 in response to this Federal Register Notice; feedback about the comment was provided to the responder on October 24, 2014.
The Office of Science Planning and Assessment and various members of CGH’s staff have been consulted and provided feedback on all surveys. Additionally, all collaborators for the Forums have been informed of our intention to evaluate these programs. As workshops are organized, they often include new partners and collaborators that are not identified until the workshop's location and participants are determined (e.g. members of the Ministry of Health from host governments or academic institutions in country) and they will be informed of CGH's intention to evaluate each workshop before an agreement to host the workshop is reached.
There will be no payments or gifts to respondents.
Personally identifiable information (PII) will be collected in the form of names, titles and institutions. Information related to participants’ name, title and institution will be linked to information about their work, accomplishments and partnerships. Information will be collected by and seen only by members of the program offices. Personally identifiable information in reports will not be shared with anyone outside of NIH, and physical copies of reports will be kept in NIH secured storage areas. Electronic files will be kept on password protected government computers and secure NIH servers. Any future publications that arise from this evaluation will feature either an analysis of anonymized or aggregate data.
The data collection is covered by NIH Privacy Act Systems of Record Notice (SORN) #09-25-0036, “Extramural Awards and Chartered Advisory Committees (IMPAC 2), Contract Information (DCIS), and Cooperative Agreement Information, HHS/NIH” (Attachment 2). This SORN was published in Federal Register on 9/26/2002, Vol. 67, p. 60742.
Since this is not considered research, the Office of Human Subjects Research Protection (OHSRP) has reviewed this proposal and determined that it is exempt for IRB (Attachment 3).
There are no sensitive questions being asked in the survey.
The evaluation activities include an internet survey and guided interviews. There will be 6 respondents per Forum and 3 Forums per year, totaling 18 respondents annually. These 18 respondents will be asked to complete all five of the information collections. The estimated response time will range from 60 to 120 minutes, depending on the activity. The respondents will consist of the country team leads for each Forum, and they will be asked to respond a total of five times.
A total of 90 workshop participants will be asked to complete the workshop evaluations each year (Attachment 1A-1E) equaling a total of 270 potential participants over the course of the three-year information collection request. The estimated annual burden is 108 hours, which works out to be 324 burden hours over the course of the three-year information collection request (Table A.12-1). For the Pre-assessment (Attachment 1A), Country Team Leads will be asked to work with their country team members to gather the information in order to complete the assessment (Attachment 4), but ultimately the Country Team Leads will be the only ones who are sent the Pre-Assessment and they will be responsible for completing it.
Using the estimated value for each category of participants from the Bureau of Labor Statistics, the total cost to the respondents is $27,546.48 over the three-year information collection request, and this works out to be an annualized cost of $9,182.16 (Table A.12-2). The hourly wage rate is calculated based on the Bureau of Labor Statistics, http://www.bls.gov/oes/current/oes_nat.htm#11-0000 occupation "Chief Executive" occupation code 11-1011.
Table 12-1. Estimated Annualized Burden Hours
Type of Respondents |
Form Name |
Number of Respondents/ Year |
Number of Responses per Respondent |
Average Burden per Response (in hours) |
Total Annual Burden Hours |
Chief Executives |
3 Month Pre Workshop Form |
18 |
1 |
120/60 |
36 |
3 Month Post Workshop Interview |
18 |
1 |
1 |
18 |
|
6 Month Post Workshop Interview |
18 |
1 |
1 |
18 |
|
12 Month Post Workshop Interview |
18 |
1 |
1 |
18 |
|
24 Month Post Workshop Interview |
18 |
1 |
1 |
18 |
|
Totals |
|
|
|
|
108 |
Table 12-2. Annualized Cost to Respondents
Form Name |
Number of Respondents/ Year |
Number of Responses per Respondent |
Average Burden per Response (in hours) |
Total Annual Burden Hours |
Hourly Wage Rate |
Total Annual Respondent Cost |
3 Month Pre Workshop Interview |
18 |
1 |
1 |
36 |
$85.02 |
$3,060.72 |
3 Month Post Workshop Interview |
18 |
1 |
1 |
18 |
$85.02 |
$1,530.36 |
6 Month Post Workshop Interview |
18 |
1 |
1 |
18 |
$85.02 |
$1,530.36 |
12 Month Post Workshop Interview |
18 |
1 |
1 |
18 |
$85.02 |
$1,530.36 |
24 Month Post Workshop Interview |
18 |
1 |
1 |
18 |
$85.02 |
$1,530.36 |
Totals |
|
|
|
108 |
|
$9,182.16 |
There are no direct costs to respondents other than their time to participate.
The cost of this information collection to the federal government arises entirely from the labor of federal program staff spent on the development of the evaluation, the review of the responses, and the program evaluation. There are no contractors contributing time, energy or effort to this project.
We estimate that all work on the collection of information, as well as, the analysis and storage of evaluations will require the effort of .2 FTE of 1 program lead, per calendar year. The program official, at a GS13, Step 1 level, will solely lead this effort so that this data collection will result in an estimated cost of $17,984.80/year, for each of the three years we expect to gather the information. Therefore the annual cost to the Federal government is estimated to be 17,984.80/year, and the cost over the three years is estimated to be $53,954.40 (Table A.14-1).
Table A.14-1. Annual Cost to the Federal Government
|
Tasks |
Title |
Grade/ Step |
Staffing (Salary x % of time) |
Annual Cost |
NIH Personnel
|
R&D, Data Collection, Report, Data Analysis
|
Program Lead |
13/1 |
$89,924 x 20%
|
$17,984.80 |
Total |
|
|
|
|
$17,984.80 |
This is a new information collection.
All open-ended questions will provide qualitative data that will be analyzed for common themes and compared across Forums. The qualitative data will be read and annotated to identify core themes from which inductive and deductive codes will be developed and defined. The research team will code the data, which means that each time the data include information that one of the codes applies to, the information will be tagged by the research team using qualitative data analysis software (e.g., Maxqda or NVivo). For example, one of the codes that may be developed may be "cancer control planning." The definition of the code "cancer control planning" will be developed by the research team (e.g., any reference to the process of or outcomes associated with cancer control planning). Then, if one respondent discussed how they were able to create measurable outcomes for the national cancer control plan due to a partnership formed at a workshop, the research team could identify that as a "cancer control planning" and tag the relevant part of the response with the code "cancer control planning.” Once the data are coded, they will be searched by topical themes and a description encompassing the context, depth and breadth of core themes in the data will be developed. Themes will be compared by type of workshop to identify similarities and differences. Although the primary purpose of this information collection is to understand why these workshops have or have not been successful, data from these evaluations may be used for publications. These publications would not generalize findings to other programs.
The project time schedule (Table 16-1) represents a 3-month time frame which begins once clearance is received, a Forum is held, and the correct period of time has passed (ranging from 3-6 months depending on the workshop). This table would be repeated for every Forum through the three year information collection phase, so that each workshop is evaluated five times.
Table 16-1. Project Time Schedule
|
Months after Workshop and Waiting Period |
||
Month 1 |
Month 2 |
Month 3 |
|
Contact Participants |
|
|
|
Obtain responses from Participants |
|
|
|
Tabulation and analysis of responses |
|
|
|
Summarize results |
|
|
|
The Cancer Control Leadership Forums Evaluation Surveys will not require exemption from displaying the expiration date of OMB approval. Any surveys will prominently display the OMB approval number and expiration date.
The proposed project does not require any exceptions to the Certification for Paperwork Reduction Act Submissions (5 CFR 1320.9).
1 Varmus, H. and Trimble, E. Integrating Cancer Control into Global Health. Sci Transl Med. 2011;101(3):101-102. doi:10.1126/scitranslmed.300.2321.
2 Varmus, H. and Trimble, E. Integrating Cancer Control into Global Health. Sci Transl Med. 2011;101(3):101-102. doi:10.1126/scitranslmed.300.2321.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | PLCO SCREENING TRIAL |
| Author | Linda Suit |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |