National Program of Cancer Registries - Annual Program Evaluation Instrument (NPCR-APEI)

ATTACHMENT 3A

NPCR Program Evaluation Instrument

(previously approved (0920-0706) and currently in use)

[Expiration date to be updated

Upon receipt of OMB approval]

Form Approved

OMB NO. _0920-0706_ Exp. Date _12-31-2008_

National Program

of Cancer Registries

Program Evaluation Instrument

Registry Status

As of June 30, 2007

To visit the National Program of Cancer Registries’ Web site please click here.

Change Password

This site was developed through contract with the Centers for Disease Control and Prevention (CDC).

Public reporting burden of this collection of information varies from 1 1/2 to 2 hours with an estimated average of 1 1/2 hours, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of informaiton unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-0706)

The National Program of Cancer Registries (NPCR)

Program Evaluation Instrument (PEI)

PURPOSE STATEMENT

The NPCR Program Evaluation Instrument (PEI) is a web-based survey instrument designed to evaluate NPCR-funded registries’ operational attributes and their progress towards meeting program standards. The APEI also provides information about advanced activities and “success stories” that highlight ways registry data are used.

Based on CDC’s Updated Guidelines for Evaluating Public Health Surveillance Systems, the PEI monitors the integration of surveillance and health information systems, the utilization of established data standards, and the electronic exchange of health data. Data provided by this report can be used for public health action, program planning and evaluation, and formulating research hypotheses.

Specific knowledge about operational activities NPCR registries are engaged in is used to provide valuable insight to CDC regarding programmatic efficiencies/deficiencies that have contributed to the success/challenges of the NPCR. The results of this instrument inform CDC and NPCR Program Consultants where technical assistance is most needed in order to continue to improve and enhance the NPCR.

Many of the questions in the 2007 PEI provide baseline data that will be used when measuring future progress with the NPCR Program Standards expected to be implemented this year. These questions, and the standard they reference, are noted throughout the instrument (e.g., “Program Standard I.a.”)

Using all available information as of June 30, 2007, the appropriate Central Cancer Registry (CCR) staff should complete the PEI.

ADMINISTRATIVE DATA

State / Territory
NPCR reference year
Registry reference year
Registry Program Director
Cooperative Agreement #	U58/DP000
Most Current Grant Award Amount	$
CDC Program Consultant
Your name
Title
Phone number
Date completed

STAFFING

The following questions use the concept of a "Full-time Equivalent" also known as an "FTE". In each question you will be asked to report the total number of FTEs (FTE count). To do this, please convert each position to the appropriate FTE using the guidelines below, rounding each position to the nearest quarter of an FTE (e.g., 34 hrs/week would convert to 0.75 FTE, whereas 35 hrs/week would convert to 1.0 FTE):

0.25 FTE = 10 hrs/week

0.50 FTE = 20 hrs/week

0.75 FTE = 30 hrs/week

1.00 FTE = 40 hrs/week

Then add each converted position for the total number of FTEs. For example, if the CTR works 35 hours a week and another CTR works 25 hours a week, the combined hours for the CTR positions = 60 hours = 1.5 FTEs.

1. On June 30, 2007, how many total FTE central cancer registry (CCR) staff positions were funded? In this table, you may include positions outside the registry, ONLY IF the registry pays a portion of the salary. Remember to use the calculation method above when computing partial FTEs.

	Total Count FTEs
Funding Category	Filled	Vacant
Number of NPCR-funded (non-contracted) FTE positions
Number of NPCR-funded, Contracted FTE positions
Number of State-funded (non-contracted) FTE positions
Number of State-funded, Contracted FTE positions
Number of non-contracted FTE positions funded by other sources
Number of Contracted FTE positions funded by other sources
TOTALS

2. Please complete this table with the number of FTEs who work in the capacity of the position titles listed. In this table, include both filled and vacant, as well as access to these staff (outside the registry), regardless of funding, in your total FTE count. So, if a position is vacant, it still counts as a position. Remember to use the same FTE calculation method as described above. Please note CTR credentials may be held by several registry positions and should be counted accordingly.

Total Count FTEs
Position (FTE or percentage of FTE)	Non-Contractor	Contractor
Principle Investigator
Program Director
Registry Administrator
Program Manager
Budget Analyst
CTR Quality Control Staff
Non-CTR Quality Control Staff
CTR Education/Training Staff
Epidemiologists
Statisticians
Computer/IT/GIS Specialists
Other staff, specify:
	Total Count CTRs Non-Contracted Contracted
Total Number CTRs (may overlap with above categories)

Staffing Section Comments (You may add comments regarding your responses in the “Staffing” section above)

LEGISLATIVE AUTHORITY

3. Does your state/territory have a current law authorizing a population-based central cancer registry? (Program Standard I.a.)

Yes

No

4. Does your state/territory have current legislation or regulations in support of all 8 criteria of the Public Law authorizing the NPCR? (Program Standard I.b.)

Yes

No

5a. Are there any penalties in place regarding reporting compliance as mandated by current legislation or regulations?

Yes

No

5b. If “Yes”, in which law/regulations are the penalties included? (check only one):

Cancer-specific reporting law/regulations

General public health law/regulations

Both

None of the above

5c. If “Yes,” have you had to impose the penalty?

Yes

No

6a. With passage of Public Law 107-260 (the Benign Brain Tumor Cancer Registry Amendment Act), NPCR-funded registries are required to collect data on benign brain tumors beginning in diagnosis year 2004. Do regulations or legislation in your State or territory authorize you to collect data on benign brain tumors?

Yes

No

6b. If “No,” what are your plans, including timeframes, to modify your State or Territory’s legislation or regulations to allow you to collect benign brain tumor data?

Specify ___________________________________________________

____________________________________________________

7. Does your State or Territory have legislation or regulations prohibiting you from reporting county level data?

Yes

No

8. Does your state law/regulations protect your cancer registry data from the Freedom of Information Act (FOIA)?

Yes

No

9a. Does your state law/regulations protect your cancer registry data from subpoena?

Yes

No

9b. If no, are data received through interstate data exchange protected from subpoena?

Yes

No

Legislation Section Comments (You may add comments regarding your responses in the “Legislation” section above)

ADMINISTRATION

10. Does your CCR maintain an operational manual that describes registry operations, policies and procedures that, at a minimum, contains the following? (Program Standard II.a.) Check all that apply:

	YES	NO
Reporting laws/regulations
List of reportable diagnoses
List of required data items
Data processing operational procedures for (check all that apply):
a. Monitoring timeliness of reporting
b. Receipt of data
c. Database management including a description of the Registry Operating System (software)
d. Conducting death certificate clearance
Procedures for Implementing and maintaining a quality assurance/control program including (check all that apply, f-h)
f. Conducting follow-back to reporting facilities on quality assurance issues
Conducting record consolidation
h. Maintaining detailed documentation of all quality assurance operations
Procedures for insuring confidentiality and data security including disaster planning
Procedures for data release including access to and disclosure of information
Procedures for maintaining and updating the operational manual

11. Do you believe that your CCR policies and procedures are sufficient and clear as to what data may and may not be disclosed and how this should occur?

Yes

No

12. Do you believe that your CCR policies and procedures are sufficient and clear for protection of confidentiality for all routine registry activities?

Yes

No

13. Do you believe that your CCR staff possesses sufficient knowledge and resources to meet risk-appropriate threats to security and confidentiality?

Yes

No

14. Does your CCR produce reports that are used to monitor the registry operations and database, including processes and activities? (Program Standard II.b.) Check all that apply:

Quality control report (central registry)

Quality control reports for each facility

Data completeness report for each facility

Timeliness of data report for each facility

Data workflow report

Other, specify ______________________________________

None of the above

15. Does your CCR have an abstracting and coding manual that is provided for use by all reporting sources? (Program Standard II.c.)

Yes

No

Administration Section Comments (You may add comments regarding your responses in the “Administration” section above)

REPORTING COMPLETENESS

16. What types of facilities and health care providers report to your CCR? Please list the percentage of facilities, by type, that actually reported in the past year (do not record the percentage reporting according to your CCR’s timeliness schedule), and calculate what percentage of the reports, by facility type, are received electronically.

Note:

• "Hospital cancer registry" is defined as one (single or joint institution) who collects data to be used internally and who would continue to do so regardless of the central cancer registry requirements to collect and report cancer data.

• Provide the number of facilities required to report and, where indicated, use your best estimate if the exact number is not available.

• For those facilities which are not applicable to your state/territory (e.g., IHS Hospitals), record zero (0) in ‘Number Required to Report’ and 100 in ‘Percent Compliant with Reporting’. In these instances, ‘Percent Reports Received Electronically’ is to be left blank and will be validated against the ‘Number Required to Report’. (Program Standards III.a-c)

Facilities Required to Report Cancer Cases by Type	Number Required to Report (Denominator)	Percent Compliant by Reporting**	Percent Reports Received Electronically
Hospitals with a cancer registry (non-federal)
Hospitals without a cancer registry (non-federal)
VA Hospitals
IHS Hospitals
Tribally Owned Hospitals
Health Centers (IHS, Tribal)
Surgery Centers
Independent Radiation Therapy Centers
In-State Independent Pathology Laboratories
Out-of-State Independent Pathology Laboratories*
Dermatologists*
Urologists*
Oncologists*/Hematologists*
Other Physicians*
Other facilities, specify:

*Provide best estimate **Those facilities who report rather than those reporting in a timely manner

17. Within 24 months of the close of the diagnosis year, what percentage of physicians, surgeons, and all other health care practitioners diagnosing or providing treatment for cancer patients submit all reportable cases to your CCR? Exception: Physicians are not required to report cases directly referred or previously admitted to, and reported by, a hospital or other facility providing screening, diagnostic or therapeutic services to patients in that State/Territory? (Program Standard V.a.) Check only one:

	100%
	75% - 99%
	51% - 74%
	10% - 50%
	1% – 9%
	None

18. Of the pathology lab reports your CCR receives, what percentage are in the College of American Pathologists (CAP) cancer protocol checklist format? (Provide best estimate)

	100%
	75% - 99%
	51% - 74%
	10% - 50%
	1% – 9%
	None

19. Do you require that non-analytic (classes 3 and 4) cases be reported to your CCR?

Yes

No

20a. Do you receive data from the Department of Defense’s Automated Central Tumor Registry (ACTUR) dataset? (If “No,” skip 20b – 20d)

Yes

No

20b. If yes, how often? Please check only one.

	Every quarter
	Every 6 months
	Once a year
	Other, specify

20c. If yes, have these data proven to be helpful in finding new incident cases?

Yes

No

20d. If not, why not? (Please check all that apply)

	Data are incomplete.
	Data are not in the proper format for us to consolidate with existing records.
	We don’t have time to deal with it.
	Other, specify: ____________________
21. To how many VA facilities do you currently send central registry staff for data collection/abstracting?		Number of Facilities
22. At how many VA facilities are data collected by a combination of VA facility staff and central registry staff?
23. How many VA facilities currently report to your CCR indirectly from the VA central cancer registry in Washington, DC?

24. If there are VA facilities not reporting, please explain why in the space

provided below:

____________________________________________________________

25. Based on historical data, how many cases per diagnosis year do you estimate are missed (i.e., not ever received) by your CCR because of non-reporting by VA facilities?

Number of cases missed: ____________________

Reporting Completeness Section Comments (You may add comments regarding your responses in the “Reporting Completeness” section above)

DATA EXCHANGE

26. Does your CCR use and require the standardized, NPCR-recommended data exchange record layout for the electronic exchange of cancer data for (Program Standards III.a.):

1. Abstract reports (The NAACCR record layout version specified in Standards for Cancer Registries Volume II: Data Standards and Data Dictionary)?

Yes

No

2. Pathology reports (NAACCR Standards for Cancer Registries Volume V: Pathology Laboratory Electronic Reporting)?

Yes

No

Not Applicable, not receiving electronic pathology reports

27. Does your exchanged data meet the following minimum criteria? (Program Standards V.d.):

a. Within 12 months of the close of the diagnosis year, your CCR exchanges data with other central cancer registries where a data- exchange agreement is in place (the data file includes all cases not previously exchanged):

Yes

No

b. Regardless of residency, your CCR collects data on all patients diagnosed and/or receiving first course of treatment in the registry’s state/territory:

Yes

No

c. The recommended frequency of data exchange is at least two times per year. Your CCR exchanges data at the following frequency:

Annually

Biannually (two times per year)

Other, explain _____________________________________

d. Exchange agreements are in place with all bordering central cancer registries:

Yes, with all bordering CCRs

No, not all

List existing agreements here: _________________ ____________________________________________

e. Exchanged data includes a dataset that consists of NPCR core data items:

Yes

No

f. 99% of exchanged data passes an NPCR-prescribed set of standard edits:

Yes

No

g. Exchanged data are transmitted via a secure encrypted Internet-based system:

Yes

No

h. The standardized, NPCR-recommended data exchange format is used to transmit data reports (The NAACCR record layout version specified in Standards for Cancer Registries Volume II: Data Standards and Data Dictionary):

Yes

No

Data Exchange Section Comments (You may add comments regarding your responses in the “Data Exchange” section above)

DATA CONTENT AND FORMAT

28. Does your CCR collect or derive all required data items using standard codes as prescribed by NPCR?

Yes

No

29. Is your CCR able to receive secure, encrypted cancer abstract data from reporting sources via the Internet?

Yes

Currently being developed and/or implemented

No, not able to receive

No, able to receive, but not receiving

30. What is the primary software system used to process and manage cancer data in your CCR? Please check only one:

Commercial Vendor

In-House Software

Registry Plus

Abstract Plus

Prep Plus

CRS Plus

Link Plus

Web Plus

Data Content and Format Section Comments (You may add comments regarding your responses in the “Data Content and Format” section above)

DATA QUALITY ASSURANCE

31. Does your CCR’s quality assurance program consist of, but is not limited to: (Program Standard VI.a.)

	YES	NO
A designated CTR is responsible for the quality assurance program
Qualified, experienced CTRs conduct quality assurance activities
At least once every 5 years, case-finding and/or re-abstracting audits from a sampling of source documents are conducted for each hospital-based reporting facility, and may include external audits (NPCR/SEER)
Data consolidation procedures are performed according to an accepted protocol
Procedures are performed for follow-back to reporting facilities on quality issues

32. Does your CCR have a designated education/training coordinator, who is a CTR, to provide training to CCR staff and reporting sources to assure high quality data? (Program Standard VI.b.)

Yes

No

33. In the past year, which of the following type of quality control audits or activities did your CCR conduct?

Yes No

Casefinding

Re-abstracting

Re-coding

Visual editing

34. Does your CCR match all causes of death against your registry data to identify a reportable cancer?

Yes

No

35a. Does your CCR update the CCR database following death certificate matching:

Death information Yes No

Missing demographic information Yes No

35b. If “Yes”, what percentage(s) is updated manually or electronically?: (Provide best estimate; may be some overlap between automation and manual review)

Death information:

Manually ______% Electronically _____%

Demographic information:

Manually ______% Electronically _____%

36. Does your CCR perform record consolidation on the following:

Data Group Electronic Manual Both Neither

Patient

Treatment

Follow-up

37a. Does your CCR provide a registry-specific edit set to your reporting facilities and/or vendors for use prior to data submissions to your CCR?

Yes

No

37b. If “Yes,” are facilities required to run registry-specific edits prior to their data submission to your CCR?

Yes

No

Data Quality Assurance Section Comments (You may add comments regarding your responses in the “Data Quality Assurance” section above)

DATA USE

38. Within 12 months of the end of the diagnosis year with data that are 90% complete, did your CCR produce pre-calculated data in tables in an electronic data file or report of incidence rates, counts, or proportions for the diagnosis year for Surveillance Epidemiology and End Results (SEER) site groups as a preliminary monitor of the top cancer sites within your state/territory? (Program Standard VII.a.)

Yes

No

39a. Within 24 months of the end of the diagnosis year with data that are 95% complete, did your CCR produce pre-calculated data in tables in an electronic data file or report? (The report should include, at a minimum, age-adjusted incidence rates and age-adjusted mortality rates for the diagnosis year by sex for SEER site groups, and, where applicable, by sex, race, and ethnicity). (Program Standard VII.b.)

Yes

No

39b. What is the most current diagnosis year a data file or report is available?

Year _____

39c. In what format is this report available?

Hard copy

Electronic word-processed file

Web page/query system

40a. Has the CCR, state health department, or its designee used registry data for planning and evaluation of cancer control objectives in at least three of the following ways in the past year: Comprehensive cancer control detailed incidence/mortality estimates, linkage with a statewide cancer screening program to improve follow-up of screened patients, health event investigation(s), needs assessment/program planning, program evaluation, or epidemiologic studies? (Program Standard VII.c.)

Yes

No

40b. If “yes,” indicate the number of times data was used for each category in the table below:

Data Use Category	Number per Year
Comprehensive cancer control
Detailed incidence/mortality estimates
Linkage with a statewide cancer screening program
Health event investigation(s)
Needs assessment/program planning
Program evaluation
Epidemiologic studies
Other, describe:

41a. Have any of the above uses of data been included in a journal publication?

Yes

No

41b. If “yes,” please list the citation(s) in the space provided:

42. During the past year, for which areas of registry data utilization did your CCR acknowledge CDC-NPCR funding, as required in the Notice of Cooperative Agreement Award? Check all that apply:

Publications (e.g.; journal articles, annual report, other reports)

Web site

Presentations, posters

Release of data

Education meeting, training program, conference

Press releases, statements

Requests for proposals, bid solicitations

None

43. Does your CCR use United States Cancer Statistics (USCS) data when performing comparative analyses?

Yes

No, explain: ______________________________________________

Data Use Section Comments (You may add comments regarding your responses in the “Data Use” section above)

COLLABORATIVE RELATIONSHIPS

44. Does your CCR actively collaborate with your state/territory’s comprehensive cancer prevention and control (CCC) planning efforts, including establishing a working relationship to ensure the use of registry data to assess and implement cancer control activities? (Program Standards IX.a.,b.)

Yes

No

Please check all of the ways you collaborate:

Member of our state/territory’s comprehensive cancer control (CCC) planning group (coalition, committee, or workgroup)

Provide data for CCC planning

Provide data for CCC Activities

Provide technical assistance and collaborate on data analyses for CCC program publications

Data linkages

Other, specify _____________________________________

None, Explain_______________________________________

45. Has your CCR established and regularly convened an advisory committee to assist in building consensus, cooperation, and planning for the registry? (Representation should include key organizations and individuals both within and outside the program. Advisory committees may be structured to meet the needs of the state/territory such as the CCC Program committee structure, an advocacy group, or a focus group). (Program Standard IX.c.)

Yes

No

The Advisory Committee includes representation from (check all that apply):

Representatives from all cancer prevention and control components

Vital Statistics

Hospital cancer registrars

American Cancer Society

Clinical-laboratory personnel

Pathologists

Clinicians

Researchers

Other, specify _____________________________________________

46. If you have an Advisory Committee, how often does this group convene, including in-person and teleconferences? Please check only one:

Quarterly

Annually

Biannually

Other, specify __________________________

47. In what ways does your CCR collaborate with the National Breast and Cervical Cancer Early Detection Program (NBCCEDP) and the National Comprehensive Cancer Control Program (NCCCP)?

Please check all that apply:

Regular meetings with NBCCEDP and/or NCCCP departmental staff

Provides assistance in staging NBCCEDP cases

Provides training/technical assistance to NBCCEDP and/or NCCCP staff Provides data to NBCCEDP and/or NCCCP

Provides technical material for publications

Provides subject matter expertise to NBCCEDP and/or NCCCP

Data linkages (NBCCEDP database, Minimum Data Elements (MDE) Study

Other, specify______________________________________

None of the above, explain__________________________

______________________________________________________

Collaborative Relationship Section Comments (You may add comments regarding your responses in the “Collaborative Relationship” section above)

ADVANCED ACTIVITIES

As the capacity of central cancer registries to collect and maintain population-based cancer data increases, so does their ability to engage in new activities designed to improve the completeness, timeliness, quality, and use of their data. In this section, we are interested in learning more about your "advanced activities."

48. Please complete the table below regarding CCR receipt of electronic records from the reporting sources listed. For each facility type, either check “Yes” and enter the format, as text, in which the electronic records are received, or check “No”. No line is to be left blank.

Facility Type	YES	Specify Type of Electronic Format	NO
Hospital Radiation Therapy Dept.
Physician Offices
State-wide Disease Index
Freestanding Radiation Centers
Hospital Disease Indices
Nuclear Medicine Facilities
Other, specify

49. If your CCR receives electronic pathology reports, in which format are these received? (Please check all that apply)

NAACCR, HL7 Format (Volume V)

NAACCR, Pipe Delimited Format (Volume V)

NAACCR, HL7 Format (NAACCR Volume II, Version 10, Chapter VI)

NAACCR, Pipe Delimited Format (NAACCR Volume II, Version 10,

Chapter VI)

Other, specify: ________________________________

Not applicable

50. What method is used to identify reportable conditions from pathology lab reports:

Manual review

Search routine based on NAACCR search term list

Other, specify ________________________________

51. For which of the following cancer surveillance needs has your CCR been in contact with your Health Department's PHIN / NEDSS staff? Please check all that apply.

Pathology laboratory reporting

Physician disease reporting

Other healthcare data reporting

None of the above

52. Has your CCR planned or developed a cancer data collection system that will be integrated into a Public Health Information Network (PHIN) compatible health surveillance system?

Yes

No

53. Does your CCR conduct at least one of the following advanced activities: Check all that apply:

Survival analysis

Quality of care studies

Clinical Studies

Publication of research studies using registry data

Geo-coding to latitude and longitude to enable mapping

Other healthcare data reporting

Other innovative uses of registry data, describe___________________________________________________

None of the above

54. How often does your CCR link to the National Death Index (NDI)? Please check only one. (If Never, skip to question 57.)

Every year

Every other year

Every 3-5 years

Other, specify __________________

Never

55. For which of the following has the NDI linkage proven to be useful? Check all that apply:

Casefinding

Survivorship

Data quality

Research

Other, please specify:______________________________

Not applicable

56. Does your CCR update your database following NDI linkage?

Yes

No

Not applicable

57. With which databases has your CCR linked its records in the past year (2006) for follow-up or some other purpose? Check all that apply:

State Vital Statistics

National Death Index

Department of Motor Vehicles

Department of Voter Registration

Indian Health Service

Medicare (Health Care Financing Administration)

Medicaid

Managed Care Organizations

Breast and Cervical Cancer

Blue Cross/Blue Shield

Hospital Discharge

Other, specify: _____________________________

None

58a. As noted in an August 13, 2004 e-mail, CDC-NPCR has negotiated an agreement with SNOMED International for several tools for use by NPCR registries. Has your CCR downloaded any of these tools (the SNOMED CT CLUE Browser, the SNOMED CT Technical Reference Guide, the ICD-O topography to SNOMED CT Map, the SNOMED CT User's Guide, and the full set of the 42 SNOMED CT encoded CAP cancer protocols and checklists)?

Yes

No

58b. Does your CCR use any of these SNOMED tools?

Yes

No

58c. If “No,” does your CCR have plans to use them in the next year?

Yes

No

58d. Does your CCR need additional information or training on these tools?

Yes

No

Advanced Activities Section Comments (You may add comments regarding your responses in the “Advanced Activities” section above)

SUCCESS STORIES

59. Please provide a summary, as a separate document, of innovative activities in which your CCR has been engaged within the past year. This can include ways in which cancer registry data has been used, journal citations, as well as other activities that may be of interest to other central registries and to NPCR (e.g., advances in any area of electronic reporting, GIS activities, death clearance activities, automated database activities that have improved data processing efficiencies, any other activities that have improved data quality, completeness, or timeliness advances in data security, or implementation of cancer inquiry response system, or success in job re-classifications) in the format suggested below:

Suggested format:

The registry highlights should fit on one page, in 12-point font and single-spaced. Information needs to be in simple language and should avoid public health jargon and scientific language.

Suggested components:

1. The name of the NPCR registry program.

2. Contact name, phone number, and e-mail address for further information

3. Title of the initiative, project, or type of data use

4. General timeframe (year(s) or month(s) during which the initiative/project/data use occurred)

5. A statement of the cancer surveillance issue, concern, or problem

6. Evidence that the activity was effective in addressing the above (#5)

7. Implications regarding the success of this activity or increased data use.

Please contact your NPCR Program Consultant if you need more detailed information about the submission of your cancer registry "success story".

Success Stories Section Comments (You may add comments regarding the “Success Stories” section above; do not record the success story in this comment box)

60. Please comment below about your experience completing this evaluation instrument by selecting the choice which best represents your thoughts and experience:

a. All or most of the questions are clearly stated

Agree Disagree

b. I understand the importance of all or most of the questions

Agree Disagree

c. For the most part, I found the web technology of the instrument to be user-friendly

Agree Disagree

d. For the most part, I consider the time spent completing the instrument to be a worthwhile contribution to NPCR and the cancer surveillance community

Agree Disagree

e. Our central registry uses data that is collected in this instrument

Agree Disagree

OPTIONAL

61. I would like to participate in discussions regarding next year’s evaluation instrument.

Yes Please enter your name and phone number here:___________________________________________

No

62. I have the following suggestions/revisions for questions or web formatting regarding next year’s evaluation instrument (please comment in the space provided below):

Thank you for participating in the NPCR Annual Program Evaluation!