National Program of Cancer Registries - Annual Program Evaluation Instrument (NPCR-APEI)

ADMINISTRATIVE DATA
State/Territory:
NPCR reference year:
Year registry began:
Registry Program Director:
Cooperative Agreement #:
Most Current Grant Award Amount:
CDC Program Consultant:
Date first funded in NPCR:
Type of current funding from NPCR:

Enhancement
Planning

Your name:
Title:
Phone number:
Date completed:

STAFFING
The questions below use the concept of a "Full-time Equivalent" also known as an "FTE". In each question you will be asked to
report the number of FTEs. To do this, please convert each position to the appropriate FTE using the guidelines below, rounding
each position to the nearest quarter of an FTE (e.g., 34 hrs/week would convert to 0.75 FTE, whereas 35 hrs/week would convert to 1.0
FTE):
0.25 FTE = 10 hrs/week
0.50 FTE = 20 hrs/week
0.75 FTE = 30 hrs/week
1.00 FTE = 40 hrs/week
Then add each converted position for the total number of FTEs. For example, if you have one epidemiologist working 35 hours and
one working 20 hours, together they are 1.5 FTEs.
1. On January 1, 2006, how many FTE staff positions were funded at the CCR? Enter the number of filled and vacant NPCR-funded FTEs in
the first row, and the number of filled and vacant non-NPCR funded FTEs in the second and third rows. (Please include contractors in your
totals but do not include positions outside the registry, even if those people sometimes engage in registry activities.)
Filled

Vacant

Number of NPCR-funded FTE
positions:
Number of State-funded FTE
positions:
Number of FTE positions funded
by other sources:
2. How many of the FTEs counted in Question 1 had the following qualifications? (Please include contractors in your totals.)
Number of filled FTE Certified Tumor Registrars (CTR):
Number of filled FTE Epidemiologists (Ph.D., Dr.P.H., or Sc.D.):
Number of filled FTE Epidemiologists (M.P.H.):

Number of filled FTE Medical Doctors (M.D.):
Number of filled FTE Statisticians (master's or doctoral level):
Number of filled Other (B.A., B.S., no degree):

LEGISLATION
3a. With the passing of Public Law 107-260 (the Benign Brain Tumor Cancer Registry Amendment Act), NPCR-funded
registries are required to collect data on benign brain tumors beginning in diagnosis year 2004. Do regulations or legislation in
your State or territory authorize you to collect data on benign brain tumors?
Yes
No
3b. If No, what are your plans, including timeframes, to modify your State or territory's legislation or regulations to allow you to
collect benign brain tumor data?

4. Does your State or Territory have legislation or regulations prohibiting you from reporting county level data?
Yes
No

DATA QUALITY AND COMPLETENESS
5. Does your CCR have at least one staff member responsible for quality control?
Yes
No
6. Does your CCR have at least one CTR who performs abstract review?
Yes
No
7. Does your CCR analyze information from edit procedures on a regular basis to identify trouble spots (e.g., with data sources,
coders, item code structure, or interpretation of instructions in manuals)?
Yes
No
8. Has your CCR included reportable hematopoietic diseases in any casefinding and quality control audits?
Yes
No
9. Does your CCR perform any of the following methods of acceptance sampling? Please check all that apply (See NAACCR
Standards Vol III, Section IIB. for a definition of acceptance sampling)
Automated edit checks
Duplicate data entry
Duplicate coding

Duplicate Abstracting
None
10. When abstracts are corrected or changed at your CCR, is information about the changes returned to the abstractor for review?
Yes
No
11. Does your CCR match all cancer causes of death against your registry data?
Yes
No
12a. Do you update your CCR database following death clearance matching?
Yes
No
12b. If "yes", what is your primary method to perform this update?
Manual
Electronic
13. Does your CCR perform follow back to or on the following sources of death clearance? (Please check all that apply)
Hospitals
Physician(s) / Medical Examiner
Nursing Homes
Hospices
Coroner
Resident, died out of state
Non-resident, died in state
Next of kin
None
14. Does your CCR receive cases from (Please check all that apply):
All bordering States

Yes

No

Some bordering States

Yes

No

Out-of-State Facilities

Yes

No

COMPUTER INFRASTRUCTURE
15. Listed below are commonly used software systems for central cancer registries. What is the primary software system used to
process and manage cancer data in your CCR? Please check only one.
RMCDS (Rocky Mountain Cancer Data System)
ELM (Premier) (IMPAC Medical Systems, Inc.)
CansurFacs (IMPAC Medical Systems, Inc.)

15a. Please indicate which Registry Plus Products software are
used primarily to process and manage cancer data in your CCR?.
Please check all that apply.reporting sources as the primary
software for managing cancer data? Please check all that apply.

IMPAC (IMPAC Medical Systems, Inc.)
MRS (Medical Registry Services, Inc.)
OncoLog (Onco, Inc.)
ERS (Electronic Registry Systems, Inc.)
Registry Plus Products

Abstract Plus
Prep Plus
CRS Plus
TLC Plus

In-house software (developed specifically for your State), specify:
Other, specify:
None

Link Plus
NAACCR Record Conversion Utility
Registry Plus Online Help

In-House, specify:
Other, specify:
16. Listed below are commonly used registry software systems. What software systems are used by most of your reporting
sources as the primary software for managing cancer data? Please check all that apply.
RMCDS (Rocky Mountain Cancer Data System)
Abstract Plus
Registry Plus Online Help
Precis Central (IMPAC Medical Systems, Inc.)
IMPAC (IMPAC Medical Systems, Inc.)
SHACRS (Scotts Hill Associates Cancer Registry Systems)
ERS (Electronic Registry Systems, Inc.)
MRS (Medical Registry Services, Inc.)
In-house software (developed specifically for your State), specify:
Other, specify:
17. Is your CCR able to receive encrypted cancer abstract data from reporting sources via the Internet? Please check only one.
Yes
Currently being developed and/or implemented
No, not able to receive/able but not using encrypted data via the Internet.
18. Which edit programs are used by your CCR to check cases? Please check all that apply.
CDC EDITS (batch)
CDC EDITS (interactive)
Other in-house, specify:
Other vendor, specify:
None
19. On which edit sets are your edits based? Please check all that apply.
NPCR - Required
NPCR - Supplemental
State Example with NPCR RX
CoC (any CoC sets)
NAACCR call-for-data
Extent of disease
Verify ICD-0-2 to 3 conversion
Recodes
SEER
TEXT
Staging

In-house
Other, specify:
20. How are edits applied at your CCR? Please check only one.
Source records
Consolidated records
Both source and consolidated records
21a. Do you perform record consolidation on your data?
Yes
No
21b. If "yes", do you perform record consolidation on the following: Check all that apply.
Data Group

Electronic

Manual

Both

Patient
Treatment
Follow-up
22a.

Do you provide a registry-specific edit set to your reporting facilities and/or vendors for use prior to data submissions to your CCR?

Yes
No
22b.

If "yes", do you require facilities to run registry-specific edits prior to their data submission to your

Yes
No

REPORTING COMPLETENESS
23. What types of facilities and health care providers report to your CCR?
Please list the number of sources in the State that should be reporting, the total number that actually reported in the past year (2004),
and indicate how many report electronically and by paper. Note: "Hospital cancer registry" is defined as one (single or joint institution)
who collects data to be used internally and who would continue to do so irregardless of the central cancer registry requirements to collect
and report cancer data. (*Provide best estimate **Free-standing or independent, where the medical records are owned by the reporting
source.)
Number actually reporting
Type of facility
Non-Federal Hospital Cancer Registries
Non-Federal Hospitals with No Cancer Registry
CoC Approved Hospital Registries
In-State Reference Pathology Laboratories**
Out-of-State Reference Pathology Laboratories *
Radiation Therapy Centers**
Dermatologists*
Urologists*

Number required
to report

Total
reporting

Reporting
Reporting by
electronically
paper

Oncologists*
Hematologists*
Other Physicians*
VA Hospitals
Military Hospitals
Indian Health Services (IHS) Hospitals
IHS Health Centers
Tribally Owned Hospitals
Tribally Owned Health Centers
Surgery Centers**
Other, specify:
24. Of the anatomical pathology lab reports your CCR receives, what percent of these reports is in the College of American Pathologists
(CAP) cancer protocol checklist format? (Provide best estimate)
100%
75%-99%
50%-74%
10%-50%
None
25. Do you require non-analytic (classes 3 and 4) cases be reported to your CCR?
Yes
No
26. Do you require historical cases (history of) be reported to your CCR?
Yes
No

USE OF REGISTRY DATA
27. Is an analytic data set that meets NPCR standards for data completeness and quality available for research within 24 months after
the completion of the diagnosis year?
Yes
No
28a. Will an electronic data file or report be produced this year of cancer incidence in your central registry? Please check all that apply.
Yes, using 12-month-old data
Yes, using 24-month data
No Annual Report will be produced this year (Skip to Q28)
28b. If "Yes", in which format(s) is the most recent "report" available? Please check all that apply.
Hardcopy
Electronic word-processed or pdf file
Web page or query system
Other, specify:
28c. Also, to which population were most recent incidence rates standardized?
2000 U.S. standard

Other, specify:
29. Has your central registry or its designee used registry data for planning and evaluation of cancer control objectives in at least
three of the following ways in the past year: 1) additional detailed incidence/mortality estimates; 2) linkage with a statewide cancer
screening program to improve follow-up of screened patients; 3) health event investigations; 4) needs assessment/program planning;
5) program evaluation; or 6) epidemiologic studies?'
Yes
No
Please select in which ways. Please check all that apply.
Additional detailed incidence/mortality estimates
Linkage with a statewide cancer screening program to improve follow-up of screened patients
Health event investigations
Needs assessment/program planning
Program evaluation
Epidemiologic studies

30. Does the CCR maintain a log of data requests made for the use of registry data?
Yes
No
Choose ONE below if answered "Yes" to Q 30
Less than 10 per year
11-49 per year
50-99 per year
100-199 per year
200-299 per year
299 or greater per year

SELF ASSESSMENT
31. Which of the following reasons are responsible for any difficulties your CCR experiences in meeting NPCR program
objectives for data completeness, quality, and timeliness.
Score each of the reasons below as follows:
2 = very important/critical
1 = relevant but not critically important
0 = not currently relative/important
OR
Select only the last response, "none of the above".
RATING
Not enough staff
Not enough staff with the necessary qualifications
Software inadequate
Hardware inadequate
State data exchange not happening
Reporting facilities lack adequate staff
Other, specify:
None of the above, our CCR does not have difficulty meeting this objective.

32. Which of the following reasons are responsible for any difficulties your CCR experiences in meeting NPCR program objectives for
data use.
Score each of the reasons below as follows:
2 = very important/critical
1 = relevant but not critically important
0 = not currently relative/important
OR
Select only the last response, "none of the above".
RATING
Not enough staff
Not enough staff with the necessary qualifications
Software inadequate
Hardware inadequate
Other, specify:
None of the above, our CCR does not have difficulty meeting this objective.

OUTCOME MEASURES - DATA ITEMS/FORMAT
33. Does your central registry collect or derive information on cancer cases that includes all data elements currently required by the NPCR?
Yes
No
34. Were the following NPCR recommended data items collected for 2003 cases? Refer to NAACCR standards, Vol II, for description of
2003 data items.
Item Name

Item #

Yes

Item Name

RX Summ - Surg
Primary Site

1290

RX Summ - Scope
Reg LN Sur

1292

RX Summ - Surg
Oth Reg/Dis

1294

Reason for No
Surgery

1340

Rad-Regional RX
Modality

1570

RX-Summ-Surg/Ra
d Seq

1380

RX Summ-Trans;
Int/Endocr

3250

RX Summ-Chemo

1390

Primary Payer at
DX

630

RX Summ-Horm
RX Summ-BRM
RX Summ-Other

Item #
1400
1410
1420

35. Does your CCR collect treatment data from: Please check all that apply.
Non-CoC approved facilities
Freestanding treatment facilities
Ambulatory surgery centers
Physicians offices
None of the above
due to lack of resources
due to lack of training
36. Does your CCR currently have the ability to collect data on:

Yes

Advanced directives? (Living Will)
Yes

No

Quality of survival (#1780)?
Yes

No

Pain (or other symptom) management?
Yes

No

37. Does your CCR collect data on family history of cancer, NAACCR data item # 360?
Yes

No

38a. Does your CCR have the ability to collect site-specific data on chemotherapy agents used?
Yes

No

38b. If "YES", how are you able to collect these data?

39. Does your CCR submit census tract data to NPCR as required?
Yes

No

40. If you cannot submit census tract data to NPCR because of existing legislation in your State, are steps being taken to
change this legislation?
Yes

No

Not Applicable

41a. Do you receive data from the Department of Defense's Automated Central Tumor Registry (ACTUR) dataset? (If no, skip to Q42)
Yes

No

41b. If yes, how often? Please check only one.
Every quarter
Every 6 months
Once a year
Other, specify:
41c. If yes, have these data proven to be helpful in finding new incident cases?
Yes

No

41d. If not, why not? Please check all that apply.
Data are incomplete
Data are not in the proper format for us to consolidate with existing records.
We don't have time to deal with it
Other, specify:
42. To how many VA facilities do you currently send central registry staff for data collection/abstracting?
Number of facilities:
43. At how many VA facilities are data collected by a combination of VA facility staff and central registry staff?
Number of facilities:
44. How many VA facilities currently report to the central registry indirectly from the VA central cancer registry?

Number of facilities:
45. If there are VA facilities not reporting, please explain why in the space provided below:

46. Based on historical data, how many cases per diagnosis year do you estimate are missed (i.e., not ever received) by your
CCR because of non-reporting by VA facilities?
Number of cases missed:
Skip to Question 54. (Questions 47-53 will be downloaded from NPCR-CSS data)
47. What was the percentage of expected 2003 cases reported to the CCR by January 1, 2005?
Numerator (# cases registered):
Denominator (# expected cases):
Percentage (use single decimal):
48. What data were used to calculate the expected number of cases listed above? Please check only one.
NAACCR
ACS Estimates
SEER incidence rates
Historical State data
Other, specify:
49. What was the percentage of expected 2003 cases reported to the CCR by January 1, 2006?
Numerator (# cases registered):
Denominator (# expected cases):
Percentage (use single decimal):
50. What method was used to calculate the expected number of 2003 cases listed above? Please check only one.
NAACCR
ACS Estimates
SEER incidence rates
Historical State data
Other, specify:
51. What was the percentage of 2003 cases reported by a death certificate only as of January 1, 2006?
Numerator (# cases death certificate only):
Denominator (# registered):
Percentage (use single decimal):
52. What percentage of 2003 cases had missing or unknown values for the following variables? Values are missing if any
part is missing (i.e. month or year for dates).
Age at diagnosis (item # 230):
Race 1 (item # 160):
Sex (item # 220):
Address at DX - State (item # 80):
County at DX (item # 90):
Primary Site (item # 400):

Date of DX (item # 390):
Diagnostic Confirmation (item # 490):
Summary Stage (item # 760 for cases prior to January 1, 2001):
Summary Stage (item # 759 for DX2001 and DX2002):
53. What percentage of unduplicated 2003 cases were microscopically confirmed?
Numerator (# cases confirmed):
Denominator (# registered):
Percentage (use single decimal):

ADVANCED ACTIVITIES
As the capacity of central cancer registries to collect and maintain population-based cancer data increases, so does their ability to engage in
new activities designed to improve the completeness, timeliness, quality, and use of their data. In this section, we are interested in learning
more about your "advanced activities".
54. Does your central registry conduct at least one of the following advanced activities: 1) receipt of encrypted case reports, 2) automated
casefinding via interfacing with pathology reports, disease indices, or other data sources in addition to vital records; 3) survival analysis; 4)
linkage with the National Death Index for survival analysis; 5) quality of care studies; 6) clinical studies; 7) publication of research studies
using registry data; 8) geocoding to latitude and longitude; or 9) other innovative uses of registry data as determined by CDC?'
Yes

No

Which activities? Please check all that apply.
Geocoding to latitude and longitude
Receipt of encrypted case reports
Automated casefinding via interfacing with pathology reports, disease indices, or other data sources in addition to vital records
Survival analysis
Linkage with the National Death Index for survival analysis
Quality of care studies
Other innovative uses of registry data as determined by CDC
Clinical Studies
Publication of research studies using registry data
55. Do you receive electronic records from any of the following? Please check all that apply.
Anatomical pathology labs
Hospital radiology departments
Physician offices
State-wide disease index
Freestanding radiology centers
Hospital disease indices
Nuclear medicine facilities
Other (specify):
None
56. If you receive electronic pathology laboratory reports, in which format do you receive them? Please check all that apply.
NAACCR format for pathology reporting (NAACCR Vol. II, Version 10, Chapter VI)
HL7, Version 2.X
HL7, Version 3.0
Other Please Specify:
None
57. For which of the following needs of cancer surveillance have you been in contact with your Health Department's PHIN / NEDSS staff
regarding? Please check all that apply.

Anatomical pathology laboratory reporting
Physician disease reporting
Other healthcare data reporting
None of the above
58. Does your CCR geocode cancer cases by latitude/longitude to enable mapping or reporting of cancer cases? Please check yes or no.
Yes

No

59. How often does your CCR link to the National Death Index (NDI)? Please check only one.
Every year

Other Specify

Every other year
Every 3-5 years
Never (skip to Q64)
60. For which of the following has your NDI linkage proven to be useful? Please check all that apply.
Casefinding
Survivorship
Data quality
Research
Other, please specify:
61. Do you update your database following NDI linkage?
Yes

No

62. With which databases has your CCR linked its records in the past year (2005) for follow-up or some other purpose? Please check all
that apply.
State Vital Statistics
National Death Index
Department of Motor Vehicles
Department of Voter Registration
Medicare (Health Care Financing Administration)
Medicaid
Managed Care Organizations
Breast and Cervical Cancer
Blue Cross/Blue Shield
Hospital Discharge
Other, specify:
None
63a. As noted in an August 13, 2004 e-mail, CDC-NPCR has negotiated an agreement with SNOMED International for several tools for use
by NPCR registries. Has your registry downloaded any of these tools (the SNOMED CT CLUE Browser, the SNOMED CT Technical
Reference Guide, the ICD-O topography to SNOMED CT Map, the SNOMED CT User's Guide, and the full set of the 42 SNOMED CT
encoded CAP cancer protocols and checklists)?
63b. Does your registry use any of these
Yes

No

If "No":
63c. Do you have plans to use them in the next
Yes

No

Do you need additional information or training on these tools?
Yes

No

64. Has your CCR planned or developed a cancer data collection system that will be integrated into a Public Health Information Network
(PHIN) compatible health surveillance system?
Yes

No

65. Has registry data been used in the past year (can be ongoing) for the purpose of comprehensive cancer control planning, breast and
cervical cancer programs, or any other cancer program implementation? Please check all that apply.
Comprehensive Cancer Control Planning or Implementation
Breast and Cervical Cancer Program (Planning or Implementation)
Other Cancer Program Planning or Implementation
66. Have any of the above uses of data (Q65) been included in a journal publication?
Yes

No

67. Please summarize additional activities not yet mentioned (i.e., advanced data security, implementation of cancer inquiry response
system) your CCR has been engaged in this past year. Please describe in the space provided below (limit 4,000 characters).

SUCCESS STORIES
68. Please provide a summary of ways in which cancer registry data has been used in the past year in the format suggested below:
Suggested format:
The data-use highlight should fit on one page, front side only, in 12-point font and single-spaced. Information needs to be in simple
language and should avoid public health jargon and scientific language
Suggested components:
1. The name of the NPCR registry program.
2. Contact name, phone number, and e-mail address for further information
3. Title of the initiative, project, or type of data use
4. General timeframe (year(s) or month(s) during which the initiative/project/data use occurred)
5. A statement of the public health issue, concern, or problem
6. Evidence that the use of registry data was effective in addressing the issue, concern, or problem
7. Implications regarding this successful use of cancer registry data
Please contact your NPCR Program Consultant (or Mary Kaeser (770) 488-3231, mkaeser@cdc.gov) if you need more detailed
information about the submission of cancer registry "success stories".