Form 1 2021 UMLS survey

UMLS Survey - 2021
OMB Control Number: 0925-0648
]Expiration Date: 06/30/2024
Public reporting burden for this collection of information is estimated to average 7 minutes per
response, including the time for reviewing instructions, searching existing data sources,
gathering and maintaining the data needed, and completing and reviewing the collection of
information. An agency may not conduct or sponsor, and a person is not required to respond to,
a collection of information unless it displays a current valid OMB control number. Send
comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to NIH, Project Clearance Branch, 6705
Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0648). Do not
return the completed form to this address.
Instructions for completing the UMLS Annual Report
We ask all UMLS licensees to complete the Annual Report to meet the reporting requirement in
the License Agreement for Use of the UMLS Metathesaurus. You may have requested a UMLS
license for the use of the UMLS, SNOMED CT, RxNorm, the Value Set Authority Center
(VSAC), or another terminology product or service provided by the National Library of Medicine.
You may be using this information directly or indirectly via a vendor’s application, such as
medical coding software or an electronic health record (EHR). At the end, please press the
"Next" button to submit your responses. You can also download your responses into a PDF file.

Page 1 of 30

Which of the following terminology products did you use this year?
UMLS (Unified Medical Language System) [NOTE: Do not select if you ONLY
used individual terminology products, such as LOINC or RxNorm, OUTSIDE of the UMLS
Metathesaurus] (1)
RxNorm (2)
VSAC (Value Set Authority Center) (4)
MetaMap (3)
MedlinePlus Connect (21)
NIH Common Data Element (CDE) Repository (5)
SNOMED CT - US Edition (11)
SNOMED CT - International Release (10)
LOINC (Logical Observation Identifier Names & Codes Terminology) (7)
CPT (Current Procedural Terminology) (6)
None of these (18)
I don't know (20)
Other; please specify: (19)
________________________________________________
End of Block: Default Question Block
Start of Block: UMLS block
What terminology or code system would you like to see added to the UMLS?
________________________________________________________________

Page 2 of 30

________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

For what purpose(s) did you use UMLS?
Process texts to extract concepts, relationships or knowledge (5)
Facilitate mapping between terminologies (8)
Develop an information retrieval system (3)
Extract specific terminologies from the Metathesaurus (e.g., MedDRA, MeSH,
SNOMED CT) (1)
Create and maintain local terminology (2)
Develop a terminology server or service (6)
Research terminologies and ontologies beyond any of the above categories (7)
Other; please specify: (16)
________________________________________________

Page 3 of 30

Which features of the UMLS Metathesaurus did you use?
Terms/concept names (MRCONSO) (1)
Relationships among concepts (MRREL) (2)
Semantic Types (MRSTY) (3)
Concept Attributes (MRSAT) (4)
Mappings (MRMAP/MRSMAP) (5)
Concept Definitions (MRDEF) (6)
Indexes (7)
Other; please specify: (8)
________________________________________________

Do you provide an application, tool, or system that utilizes the UMLS?

o Yes (1)
o No (2)
Display This Question:
If Do you provide an application, tool, or system that utilizes the UMLS? = Yes

Please describe your application, tool, or system that utilizes the UMLS.
How does it utilize the UMLS? Are UMLS data exposed to users? Who uses the
application/tool/system?
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 4 of 30

________________________________________________________________
________________________________________________________________

Did you install the UMLS locally?

o Yes (1)
o No (2)
Display This Question:
If Did you install the UMLS locally? = Yes

Did you install the UMLS locally using a database?

o Yes (1)
o No (2)
Display This Question:
If Did you install the UMLS locally using a database? = Yes

Page 5 of 30

When you installed the UMLS locally, which database did you use?
MS Access (1)
MS SQL Server (2)
MySQL (3)
Oracle (4)
PostgresSQL (5)
Semantic Web Triple Store (6)
NoSQL (7)
Other; please specify: (8)
________________________________________________

Display This Question:
If Did you install the UMLS locally? = Yes

What format of Metathesaurus data did you use?

o RRF (Rich Release Format) (2)
o ORF (Original Release Format) (4)
Display This Question:
If Did you install the UMLS locally using a database? = No

How did you implement the UMLS locally?
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 6 of 30

________________________________________________________________
________________________________________________________________

Which UMLS API product(s) did you use?
UMLS REST API (1)
UMLS SOAP API (2)
Legacy UMLSKS API (3)
Did not use any UMLS API (4)

Display This Question:
If Which UMLS API product(s) did you use? = UMLS REST API
Or Which UMLS API product(s) did you use? = UMLS SOAP API
Or Which UMLS API product(s) did you use? = Legacy UMLSKS API

Briefly describe your most common use cases for the UMLS API.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 7 of 30

How did you browse the UMLS data?
UTS (1)
MetamorphoSys (2)
BioPortal (5)
Other; please specify: (4)
________________________________________________

Display This Question:
If How did you browse the UMLS data? = UTS

What can NLM do to improve the UTS?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 8 of 30

Please select and briefly describe any enhancements you would suggest to improve the UMLS.
File Format (4) ________________________________________________
Mapping (5) ________________________________________________
Installation (6) ________________________________________________
Site Navigation (7) ________________________________________________
Customer Support (8)
________________________________________________
Use Case Examples (9)
________________________________________________
User Forum (10) ________________________________________________
Training/Documentation (11)
________________________________________________
Other; please specify: (12)
________________________________________________
End of Block: UMLS block
Start of Block: RxNorm

Page 9 of 30

How did you access RxNorm?
Browsing in RxNav (2)
Download RxNorm files (3)
Through the RxNorm API (5)
RxNav-in-a-Box (container for installation of the RxNorm API and RxNav) (7)
Files extracted from the Metathesaurus (4)
Browsing on the UMLS Terminology Services (1)
Other; please specify: (6)
________________________________________________

Display This Question:
If How did you access RxNorm? = Files extracted from the Metathesaurus
Or How did you access RxNorm? = Browsing on the UMLS Terminology Services

Why did you use RxNorm in the UMLS? What value does the UMLS add to your use of
RxNorm?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 10 of 30

For what purpose(s) did you use RxNorm?
E-Prescribing (1)
Information Exchange (2)
Formulary Development (3)
Reference Value Sets (4)
Analytics (5)
Access to information about obsolete RxCUIs and NDCs (for analytics) (7)
Other; please specify: (6)
________________________________________________

Display This Question:
If How did you access RxNorm? = Download RxNorm files

Which RxNorm release(s) are you using?
Monthly - Full (1)
Weekly - Full (2)
Monthly - Prescribable Subset (3)
Weekly - Prescribable Subset (4)

Page 11 of 30

Do you use the RxNorm SCDF (Semantic Clinical Dose Form) term type? If yes, how are you
using SCDFs?

o Yes; please describe: (1) ________________________________________________
o No (2)
Do you use NDCs from RxNorm? If yes, please describe how are you using them.

o Yes; please describe: (4) ________________________________________________
o No (5)
Do you use the UMLSCUI attribute in RxNorm?

o Yes (1)
o No (2)
What can NLM do to improve RxNorm?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
End of Block: RxNorm
Start of Block: CPT

Page 12 of 30

How did you access CPT?
Download files from the American Medical Association (3)
CPT Developer’s Tool Kit (CPT DTK) (4)
Files extracted from the Metathesaurus (2)
Browsing on the UMLS Terminology Services (1)
Other; please specify: (5)
________________________________________________

Display This Question:
If How did you access CPT? = Files extracted from the Metathesaurus
Or How did you access CPT? = Browsing on the UMLS Terminology Services

Why did you use CPT in the UMLS? What value does the UMLS add to your use of CPT?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
End of Block: CPT
Start of Block: LOINC

Page 13 of 30

How did you access LOINC?
Download files from the Regenstrief Institute (loinc.org) (4)
Browsing on search.loinc.org (2)
RELMA (Regenstrief LOINC Mapping Assistant) (6)
Browsing on the UMLS Terminology Services (1)
Files extracted from the Metathesaurus (3)
Other; please specify: (5)
________________________________________________

Display This Question:
If How did you access LOINC? = Browsing on the UMLS Terminology Services
Or How did you access LOINC? = Files extracted from the Metathesaurus

Why did you use LOINC in the UMLS? What value does the UMLS add to your use of LOINC?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
End of Block: LOINC
Start of Block: SNOMED

Page 14 of 30

How did you access SNOMED CT?
Download the US Edition of SNOMED CT (7)
Download the International Release of SNOMED CT (2)
Download the CORE Problem List Subset (3)
Download the Spanish Edition of SNOMED CT (4)
Browse on the UMLS Terminology Services (1)
Files extracted from the Metathesaurus (5)
Other; please specify: (6)
________________________________________________

Display This Question:
If How did you access SNOMED CT? = Browse on the UMLS Terminology Services
Or How did you access SNOMED CT? = Files extracted from the Metathesaurus

Why did you use SNOMED CT in the UMLS? What value does the UMLS add to your use of
SNOMED CT?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 15 of 30

Have you implemented SNOMED CT in an electronic health record system or a terminology
service used by electronic health record systems?

o Yes (1)
o No (2)
Display This Question:
If Have you implemented SNOMED CT in an electronic health record system or a terminology service
use... = Yes

Which format of SNOMED CT did you use in your implementation?
RF1 (1)
RF2 (2)
OWL (6)
CORE Problem List Subset of SNOMED CT (3)
Don't know (4)
Other; please specify: (5)
________________________________________________

Display This Question:
If Have you implemented SNOMED CT in an electronic health record system or a terminology service
use... = Yes

Page 16 of 30

In what ways do health providers interact with SNOMED CT within the system?
Pulling problems, findings, etc into the problem list from SNOMED CT content
(1)
Enter problems, findings, as text or from institutional supported list and converted
to SNOMED CT via 3rd party software (4)
I don’t know (5)
Other; please specify: (6)
________________________________________________

Display This Question:
If Have you implemented SNOMED CT in an electronic health record system or a terminology service
use... = Yes

Do you use SNOMED CT post-coordinated expressions in your system/application?

o Yes (2)
o No (3)
o Don't know (4)
Display This Question:
If Do you use SNOMED CT post-coordinated expressions in your system/application? = Yes

Please describe how you use SNOMED CT post-coordinated expressions in your
system/application.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 17 of 30

Display This Question:
If Do you use SNOMED CT post-coordinated expressions in your system/application? = No

Please describe how you use SNOMED CT data in your application/system.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Which maps between SNOMED CT and ICD-9-CM or ICD-10-CM have you used?
SNOMED CT to ICD-10 CM Map (1)
SNOMED CT to ICD-9-CM map (2)
ICD-9-CM diagnoses to SNOMED CT map (3)
ICD-9-CM procedures to SNOMED CT map (4)
I have not used any of the maps (5)

Display This Question:
If Which maps between SNOMED CT and ICD-9-CM or ICD-10-CM have you used? != I have not
used any of the maps

Please describe how you used the maps you selected in the previous question.
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 18 of 30

________________________________________________________________
________________________________________________________________
End of Block: SNOMED
Start of Block: MetaMap
Please list which MetaMap options and output formats are you using?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Would you be willing to test if MetaMap Lite can serve your needs?

o Yes (1)
o No (3)
End of Block: MetaMap
Start of Block: VSAC
How do you use the VSAC and value sets?
Author value sets (4)
Report clinical quality measures (CQMs) to CMS (7)
Access content through VSAC FHIR terminology service (8)
Other; please specify: (9)
________________________________________________

Page 19 of 30

Display This Question:
If How do you use the VSAC and value sets? = Author value sets

Would you be interested in creating a program release for easy searching and downloading of
your value sets?

o Yes (1)
o No (2)
Display This Question:
If How do you use the VSAC and value sets? = Author value sets

Are you interested in creating intensional (logically-defined) value sets in VSAC?

o Yes (1)
o No (2)
Display This Question:
If Are you interested in creating intensional (logically-defined) value sets in VSAC? = Yes

What changes or additional functionality do you need to create your intensional value sets in
VSAC?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Display This Question:
If Are you interested in creating intensional (logically-defined) value sets in VSAC? = Yes

Page 20 of 30

Provide specific examples of intensional value sets you want to be able to create in VSAC in the
future, especially with RxNorm, LOINC, and other code system functionality that is not included
in this testing. (For example, a value set of all of the dispensable drugs in RxNorm or a value set
of all of the SNOMED CT codes that have a causative agent of hepatitis B virus.)
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Display This Question:
If How do you use the VSAC and value sets? = Author value sets

What can NLM do to improve the VSAC authoring environment?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
End of Block: VSAC
Start of Block: CDE
Please describe how you used the NIH CDE Repository.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 21 of 30

Have you used or recommended a CDE or Form in the NIH CDE Repository?

o Yes (1)
o No (4)
What can NLM do to improve the NIH CDE Repository?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
End of Block: CDE
Start of Block: Generic Ending block
Display This Question:
If Which of the following terminology products did you use this year? != None of these
And Which of the following terminology products did you use this year? != I don't know

Please provide references (URLs, citations, publications, etc.) or details about your work that
involves terminology products from NLM.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 22 of 30

Display This Question:
If Which of the following terminology products did you use this year? = UMLS (Unified Medical
Language System) [NOTE: Do not select if you ONLY used individual terminology products, such
as LOINC or RxNorm, OUTSIDE of the UMLS Metathesaurus]
Or Which of the following terminology products did you use this year? = RxNorm
Or Which of the following terminology products did you use this year? = MetaMap
Or Which of the following terminology products did you use this year? = VSAC (Value Set Authority
Center)
Or Which of the following terminology products did you use this year? = NIH Common Data Element
(CDE) Repository
Or Which of the following terminology products did you use this year? = CPT (Current Procedural
Terminology)
Or Which of the following terminology products did you use this year? = LOINC (Logical Observation
Identifier Names & Codes Terminology)
Or Which of the following terminology products did you use this year? = SNOMED CT - International
Release
Or Which of the following terminology products did you use this year? = SNOMED CT - US Edition

Page 23 of 30

For which terminology product(s) would you like to see new or additional educational materials?
UMLS (Unified Medical Language System - Metathesaurus, Semantic Network,
and Lexical Tools) (12)
RxNorm (5)
VSAC (Value Set Authority Center) (7)
SNOMED CT (8)
LOINC (15)
NIH Common Data Element (CDE) Repository (9)
MetaMap (13)
None (10)
Other; please specify: (11)
________________________________________________

Display This Question:
If For which terminology product(s) would you like to see new or additional educational materials? =
RxNorm

Page 24 of 30

For which RxNorm topics would you like to see new or additional educational materials?
Understanding term types (1)
Navigating the RxNorm graph (2)
Mapping your formulary to RxNorm (3)
Mapping NDCs to RxNorm (4)
Relating RxNorm to DailyMed (5)
Extracting drug classes (6)
Using the API (7)
Using RxNav (9)
Using RxMix (11)
Using RxClass (12)
Other; please specify: (10)
________________________________________________

Display This Question:
If For which terminology product(s) would you like to see new or additional educational materials? =
UMLS (Unified Medical Language System - Metathesaurus, Semantic Network, and Lexical Tools)

Page 25 of 30

For which UMLS topics would you like to see new or additional educational materials?
Understanding term types (1)
Using the API (2)
Working with databases (4)
Content and/or vocabulary questions (5)
Other; please specify: (6)
________________________________________________

Display This Question:
If For which UMLS topics would you like to see new or additional educational materials? = Using the
API
Or For which UMLS topics would you like to see new or additional educational materials? = Working
with databases
Or For which UMLS topics would you like to see new or additional educational materials? = Content
and/or vocabulary questions
Or For which UMLS topics would you like to see new or additional educational materials? =
Understanding term types

Please provide more information about the UMLS topics for which you would like to see more
educational resources. Are there specific topics, use cases, or questions you would like to see
addressed?
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 26 of 30

Display This Question:
If For which terminology product(s) would you like to see new or additional educational materials? =
SNOMED CT

For which SNOMED CT topics would you like to see new or additional educational materials?
Post-Coordinated Expressions (1)
Reference Sets (RefSets) (2)
Mapping Subsets (3)
Accessing SNOMED CT (4)
Requesting Content Changes (5)
Other; please specify: (6)
________________________________________________

Display This Question:
If Which of the following terminology products did you use this year? = UMLS (Unified Medical
Language System) [NOTE: Do not select if you ONLY used individual terminology products, such
as LOINC or RxNorm, OUTSIDE of the UMLS Metathesaurus]
Or Which of the following terminology products did you use this year? = RxNorm
Or Which of the following terminology products did you use this year? = MetaMap
Or Which of the following terminology products did you use this year? = VSAC (Value Set Authority
Center)
Or Which of the following terminology products did you use this year? = NIH Common Data Element
(CDE) Repository
Or Which of the following terminology products did you use this year? = CPT (Current Procedural
Terminology)
Or Which of the following terminology products did you use this year? = LOINC (Logical Observation
Identifier Names & Codes Terminology)
Or Which of the following terminology products did you use this year? = SNOMED CT - International
Release
Or Which of the following terminology products did you use this year? = SNOMED CT - US Edition

Page 27 of 30

Would you be willing to describe your use of NLM terminology products and services on the
NLM Web site? (If so, NLM may contact you)

o Yes (4)
o No (5)
Display This Question:
If Which of the following terminology products did you use this year? != I don't know

Do you use terminologies, ontologies, or code systems outside of the UMLS license?

o Yes (1)
o No (4)
Display This Question:
If Do you use terminologies, ontologies, or code systems outside of the UMLS license? = Yes

What tools, programs, and services do you use to access terminology products outside of the
UMLS license?
NCI EVS (1)
OHDSI (2)
NCBO BioPortal (3)
NCBI Entrez System (dbGAP, MedGen, ClinVar, etc.) (4)
Other; please specify: (5)
________________________________________________

Display This Question:
If Do you use terminologies, ontologies, or code systems outside of the UMLS license? = Yes

Page 28 of 30

Which of the following do you use outside of the UMLS license?
ICD10CM (1)
UCUM (2)
NDC (3)
WHO Drug Codes (4)
MedDRA (5)
GMDN (6)
NCIt, NCIm (NCI EVS) (7)
MedGen (8)
ClinVar (9)
dbSNP (10)
RefSeq (11)
COSMIC (12)
Other; please specify: (13)
________________________________________________

Display This Question:
If Which of the following terminology products did you use this year? = None of these
Or Which of the following terminology products did you use this year? = I don't know

Why did you request a UMLS license/create a UTS account?
________________________________________________________________
________________________________________________________________
Page 29 of 30

________________________________________________________________
________________________________________________________________
________________________________________________________________
End of Block: Generic Ending block

Page 30 of 30