1 2020 UMLS User Satisfaction Survey instrument

UMLS User Satisfaction Survey

OMB Control Number: 0925-0648
Expiration Date: 05/31/2021
Public reporting burden for this collection of information is estimated to average 7 minutes per response,
including the time for reviewing instructions, searching existing data sources, gathering and maintaining
the data needed, and completing and reviewing the collection of information. An agency may not conduct
or sponsor, and a person is not required to respond to, a collection of information unless it displays a
current valid OMB control number. Send comments regarding this burden estimate or any other aspect of
this collection of information, including suggestions for reducing this burden, to NIH, Project Clearance
Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0648). Do
not return the completed form to this address.

Page 1 of 17

Which of the following terminology products did you use this year?

▢

UMLS (Unified Medical Language System) [NOTE: Do not select if you ONLY used individual
terminology products, such as LOINC or RxNorm, OUTSIDE of the UMLS Metathesaurus] (1)

▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢

RxNorm (2)
VSAC (Value Set Authority Center) (4)
MetaMap (3)
MedlinePlus Connect (21)
NIH Common Data Element (CDE) Repository (5)
SNOMED CT - US Edition (11)
SNOMED CT - International Release (10)
LOINC (Logical Observation Identifier Names & Codes Terminology) (7)
CPT (Current Procedural Terminology) (6)
None of these (18)
I don't know (20)
Other; please specify: (19) ________________________________________________

What terminology or code system would you like to see added to the UMLS?
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 2 of 17

For what purpose(s) did you use UMLS?

▢
▢
▢
▢
▢
▢
▢
▢

Process texts to extract concepts, relationships or knowledge (5)
Facilitate mapping between terminologies (8)
Develop an information retrieval system (3)
Extract specific terminologies from the Metathesaurus (e.g., MedDRA, MeSH, SNOMED CT) (1)
Create and maintain local terminology (2)
Develop a terminology server or service (6)
Research terminologies and ontologies beyond any of the above categories (7)
Other; please specify: (16) ________________________________________________

Do you provide an application, tool, or system that utilizes the UMLS?

o
o

Yes (1)
No (2)

Please describe your application, tool, or system that utilizes the UMLS.
How does it utilize the UMLS? Are UMLS data exposed to users? Who uses the
application/tool/system?
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 3 of 17

Please select and briefly describe any enhancements you would suggest to improve the UMLS.

▢
▢
▢
▢
▢
▢
▢
▢

File Format (4) ________________________________________________
Mapping (5) ________________________________________________
Installation (6) ________________________________________________
Site Navigation (7) ________________________________________________
Customer Support (8) ________________________________________________
Use Case Examples (9) ________________________________________________
Training/Documentation (11) _______________________________________________
Other; please specify: (12) ________________________________________________

How did you access RxNorm?

▢
▢
▢
▢
▢
▢
▢

Browsing in RxNav (2)
Download RxNorm files (3)
Through the RxNorm API (5)
RxNav-in-a-Box (container for installation of the RxNorm API and RxNav) (7)
Files extracted from the Metathesaurus (4)
Browsing on the UMLS Terminology Services (1)
Other; please specify: (6) ________________________________________________

Why did you use RxNorm in the UMLS? What value does the UMLS add to your use of RxNorm?
________________________________________________________________
________________________________________________________________

Page 4 of 17

________________________________________________________________

For what purpose(s) did you use RxNorm?

▢
▢
▢
▢
▢
▢
▢

E-Prescribing (1)
Information Exchange (2)
Formulary Development (3)
Reference Value Sets (4)
Analytics (5)
Access to information about obsolete RxCUIs and NDCs (for analytics) (7)
Other; please specify: (6) ________________________________________________

Do you use the RxNorm SCDF (Semantic Clinical Dose Form) term type? If yes, how are you using
SCDFs?

o
o

Yes; please describe: (1) ________________________________________________
No (2)

Do you use NDCs from RxNorm? If yes, please describe how are you using them.

o
o

Yes; please describe: (4) ________________________________________________
No (5)

Page 5 of 17

Do you use the UMLSCUI attribute in RxNorm?

o
o

Yes (1)
No (2)

What can NLM do to improve RxNorm?
________________________________________________________________
________________________________________________________________
________________________________________________________________
How did you access CPT?

▢
▢
▢
▢
▢

Download files from the American Medical Association (3)
CPT Developer’s Tool Kit (CPT DTK) (4)
Files extracted from the Metathesaurus (2)
Browsing on the UMLS Terminology Services (1)
Other; please specify: (5) ________________________________________________

Why did you use CPT in the UMLS? What value does the UMLS add to your use of CPT?
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 6 of 17

How did you access LOINC?

▢
▢
▢
▢
▢
▢

Download files from the Regenstrief Institute (loinc.org) (4)
Browsing on search.loinc.org (2)
RELMA (Regenstrief LOINC Mapping Assistant) (6)
Browsing on the UMLS Terminology Services (1)
Files extracted from the Metathesaurus (3)
Other; please specify: (5) ________________________________________________

Why did you use LOINC in the UMLS? What value does the UMLS add to your use of LOINC?
________________________________________________________________
________________________________________________________________
________________________________________________________________
How did you access SNOMED CT?

▢
▢
▢
▢
▢
▢
▢

Download the US Edition of SNOMED CT (7)
Download the International Release of SNOMED CT (2)
Download the CORE Problem List Subset (3)
Download the Spanish Edition of SNOMED CT (4)
Browse on the UMLS Terminology Services (1)
Files extracted from the Metathesaurus (5)
Other; please specify: (6) ________________________________________________

Why did you use SNOMED CT in the UMLS? What value does the UMLS add to your use of
SNOMED CT?
________________________________________________________________
________________________________________________________________

Page 7 of 17

________________________________________________________________
Have you implemented SNOMED CT in an electronic health record system or a terminology
service used by electronic health record systems?

o
o

Yes (1)
No (2)

Which format of SNOMED CT did you use in your implementation?

▢
▢
▢
▢
▢
▢

RF1 (1)
RF2 (2)
OWL (6)
CORE Problem List Subset of SNOMED CT (3)
Don't know (4)
Other; please specify: (5) ________________________________________________

Page 8 of 17

In what ways do health providers interact with SNOMED CT within the system?

▢
▢

Pulling problems, findings, etc into the problem list from SNOMED CT content (1)

Enter problems, findings, as text or from institutional supported list and converted to SNOMED
CT via 3rd party software (4)

▢
▢

I don’t know (5)
Other; please specify: (6) ________________________________________________

Do you use SNOMED CT post-coordinated expressions in your system/application?

o
o
o

Yes (2)
No (3)
Don't know (4)

Please describe how you use SNOMED CT post-coordinated expressions in your
system/application.
________________________________________________________________
________________________________________________________________
________________________________________________________________
Please describe how you use SNOMED CT data in your application/system.
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 9 of 17

Which maps between SNOMED CT and ICD-9-CM or ICD-10-CM have you used?

▢
▢
▢
▢
▢

SNOMED CT to ICD-10 CM Map (1)
SNOMED CT to ICD-9-CM map (2)
ICD-9-CM diagnoses to SNOMED CT map (3)
ICD-9-CM procedures to SNOMED CT map (4)
I have not used any of the maps (5)

Please describe how you used the maps you selected in the previous question.
________________________________________________________________
________________________________________________________________
________________________________________________________________
Please list which MetaMap options and output formats are you using?
________________________________________________________________
________________________________________________________________
________________________________________________________________

Would you be willing to test if MetaMap Lite can serve your needs?

o
o

Yes (1)
No (3)

Page 10 of 17

How do you use the VSAC and value sets?

▢
▢
▢
▢

Author value sets (4)
Report clinical quality measures (CQMs) to CMS (7)
Access content through VSAC FHIR terminology service (8)
Other; please specify: (9) ________________________________________________

Would you be interested in creating a program release for easy searching and downloading of
your value sets?

o
o

Yes (1)
No (2)

Are you interested in creating intensional (logically-defined) value sets in VSAC?

o
o

Yes (1)
No (2)

What changes or additional functionality do you need to create your intensional value sets in
VSAC?
________________________________________________________________
________________________________________________________________
________________________________________________________________
Provide specific examples of intensional value sets you want to be able to create in VSAC in the
future, especially with RxNorm, LOINC, and other code system functionality that is not included in
this testing. (For example, a value set of all of the dispensable drugs in RxNorm or a value set of
all of the SNOMED CT codes that have a causative agent of hepatitis B virus.)
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 11 of 17

What can NLM do to improve the VSAC authoring environment?
________________________________________________________________
________________________________________________________________
________________________________________________________________
Please describe how you used the NIH CDE Repository.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Have you used or recommended a CDE or Form in the NIH CDE Repository?

o
o

Yes (1)
No (4)

What can NLM do to improve the NIH CDE Repository?
________________________________________________________________
________________________________________________________________
________________________________________________________________
Please provide references (URLs, citations, publications, etc.) or details about your work that
involves terminology products from NLM.
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 12 of 17

For which terminology product(s) would you like to see new or additional educational materials?

▢

UMLS (Unified Medical Language System - Metathesaurus, Semantic Network, and Lexical
Tools) (12)

▢
▢
▢
▢
▢
▢
▢
▢

RxNorm (5)
VSAC (Value Set Authority Center) (7)
SNOMED CT (8)
LOINC (15)
NIH Common Data Element (CDE) Repository (9)
MetaMap (13)
None (10)
Other; please specify: (11) ________________________________________________

Page 13 of 17

For which RxNorm topics would you like to see new or additional educational materials?

▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢

Understanding term types (1)
Navigating the RxNorm graph (2)
Mapping your formulary to RxNorm (3)
Mapping NDCs to RxNorm (4)
Relating RxNorm to DailyMed (5)
Extracting drug classes (6)
Using the API (7)
Using RxNav (9)
Using RxMix (11)
Using RxClass (12)
Other; please specify: (10) ________________________________________________

For which UMLS topics would you like to see new or additional educational materials?

▢
▢
▢
▢
▢

Understanding term types (1)
Using the API (2)
Working with databases (4)
Content and/or vocabulary questions (5)
Other; please specify: (6) ________________________________________________

Please provide more information about the UMLS topics for which you would like to see more
educational resources. Are there specific topics, use cases, or questions you would like to see
addressed?
________________________________________________________________
________________________________________________________________

Page 14 of 17

________________________________________________________________
________________________________________________________________

For which SNOMED CT topics would you like to see new or additional educational materials?

▢
▢
▢
▢
▢
▢

Post-Coordinated Expressions (1)
Reference Sets (RefSets) (2)
Mapping Subsets (3)
Accessing SNOMED CT (4)
Requesting Content Changes (5)
Other; please specify: (6) ________________________________________________

Would you be willing to describe your use of NLM terminology products and services on the NLM
Web site? (If so, NLM may contact you)

o
o

Yes (4)
No (5)

Do you use terminologies, ontologies, or code systems outside of the UMLS license?

o
o

Yes (1)
No (4)

Page 15 of 17

What tools, programs, and services do you use to access terminology products outside of the
UMLS license?

▢
▢
▢
▢
▢

NCI EVS (1)
OHDSI (2)
NCBO BioPortal (3)
NCBI Entrez System (dbGAP, MedGen, ClinVar, etc.) (4)
Other; please specify: (5) ________________________________________________

Which of the following do you use outside of the UMLS license?

▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢

ICD10CM (1)
UCUM (2)
NDC (3)
WHO Drug Codes (4)
MedDRA (5)
GMDN (6)
NCIt, NCIm (NCI EVS) (7)
MedGen (8)
ClinVar (9)
dbSNP (10)
RefSeq (11)
COSMIC (12)
Other; please specify: (13) ________________________________________________

Page 16 of 17

Why did you request a UMLS license/create a UTS account?
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page 17 of 17