Attachment 1_MOU

EXECUTIVE OFFICE OF THE PRESIDENT
NATIONAL SCIENCE AND TECHNOLOGY COUNCIL
WASHINGTON, D.C. 20502

MEMORANDUM OF UNDERSTANDING BETWEEN
THE NATIONAL INSTITUTES OF HEALTH,
THE NATIONAL SCIENCE FOUNDATION, AND
THE OFFICE OF SCIENCE & TECHNOLOGY POLICY
FOR THE PURPOSES OF ESTABLISHING A MULTI-AGENCY CONSORTIUM TO
SUPPORT THE STAR METRICS PROJECT
1. Agreement Purpose
The purpose of this Memorandum of Understanding (“MOU” or “Agreement”) is to serve as a
written agreement between the National Science Foundation (NSF), the National Institutes of
Health (NIH), and the Office of Science & Technology Policy (OSTP), acting through its
Committee on Science, a committee of the National Science and Technology Council (NSTC), to
partner in the creation of the program “Science and Technology for America’s Reinvestment:
Measuring the EffecTs of Research on Innovation, Competitiveness and Science” (STAR
METRICS). The partners to this program will form the STAR METRICS Consortium
(“Consortium”), and will administer the STAR METRICS Program.

2. MOU Authority
Executive Order 12881 of November 23, 1993, established the National Science and Technology
Council and P.L. 111-8.
The NSF is acting pursuant to the National Science Foundation Act of 1950 as amended
(42 U.S.C. 1861 et seq.).
The NIH is acting pursuant to Sections 301, 402 and 405 of the Public Health Service Act (42
U.S.C. §§241, 281 and 284.)

3. Background and Mission
The Office of Science and Technology Policy, through the NSTC Committee on Science,
established the Science of Science Policy (SoSP) Interagency Working Group to develop an
evidence-based framework for informing policy investments in research and development, and
assess the impacts of those investments broadly. Themes and goals for this emergent research
field were outlined in a Federal Research Roadmap1, and this effort was also highlighted in the
FY 2011 Budget Priorities Memorandum from OSTP and the Office of Management and Budget
(OMB). The memorandum asks Federal agencies to “develop outcome-oriented goals for their
science and technology activities, establish procedures and timelines for evaluating the
performance of these activities, and target investments toward high-performing programs.
Agencies should develop ‘science of science policy’ tools that can improve management of their
research and development portfolios and better assess the impact of their science and technology
1

The Science of Science Policy: a Federal Research Roadmap,
http://www.scienceofsciencepolicy.net/media/p/304.aspx
1

EXECUTIVE OFFICE OF THE PRESIDENT
NATIONAL SCIENCE AND TECHNOLOGY COUNCIL
WASHINGTON, D.C. 20502

investments. Sound science should inform policy decisions, and agencies should invest in
relevant science and technology as appropriate.”2
An important opportunity to initiate pilot studies in the Science of Science Policy is offered by
the 2009 American Recovery and Reinvestment Act (ARRA). The ARRA increased the
investment in America’s science and technology fields to spur innovation and economic
prosperity. With this financial reinvestment comes the responsibility to ensure transparency with
regard to the social, economic, and scientific value being gained as a result of these investments.
Moreover, the ARRA requires quarterly reporting by grant recipients (Section 1512 of ARRA),
as implemented by OMB.3
Currently, the tools available for generating empirical data that would reflect the collective value
of the Government’s grant-based investments in science and technology are inadequate. To
address this inadequacy, SoSP has developed the STAR METRICS Program to create a data
infrastructure that will provide participating agencies with a consistent and reliable means to
account for the staff (Principal Investigators, co-Principal Investigators, graduate and
undergraduate students, lab technicians, science administrators, etc.) supported by Federal funds
and paid via research institution payrolls. This is consistent with ARRA’s mandatory reporting
requirements and the Administration’s commitment to enhancing transparency with regard to
Federal investments. Further, STAR METRICS will facilitate measurement of the impact of
science investments on social and economic outcomes such as job creation, knowledge
generation (as measured, for example, by citations and patents), and potentially, over time, health
outcomes. All data will reside in a protected data enclave featuring appropriate confidentiality
and privacy firewalls (see agency responsibilities in Section 4).
The initial members of the STAR METRICS Consortium are the NSF, the NIH, and the OSTP.
This small number will enable the Consortium to test analytical frameworks, determine best
practices, and ensure scalability if additional science and technology agencies decide to join the
Consortium.
The Consortium will create a database that will combine, in useful fashion, scientific investment
data from the NSF and the NIH with data acquired from voluntarily participating research
institutions. The Consortium will analyze the data to generate various measures of the social,
economic, and scientific value of the analyzed investments, using the metrics the NSF and the
NIH established during the prior STAR pilot program4 (including measures of job creation and
retention and citation generation). Other metrics already created or under development,
including patents issued, patent applications filed, and businesses created also will be analyzed.
In addition, the Consortium will develop and validate new metrics and outcome measures based
2

M-09-27 Memorandum for the Heads of Executive Departments and Agencies, Science and Technology Priorities
for the FY 2011 Budget, August 4, 2009.
3
Implementing Guidance for the Reports on Use of Funds Pursuant to the American Recovery and Reinvestment
Act of 2009, June 22, 2009;
http://www.whitehouse.gov/omb/assets/memoranda_fy2009/m09-21.pdf
4
Previously, NIH and NSF instituted a feasibility pilot program called simply STAR, at 7 volunteer universities
that received ARRA funding. The pilot demonstrated that institutional and agency administrative data on jobs could
be easily gathered and matched in a database. The pilot also demonstrated that web scraping could be utilized as a
tool to track publications and citations that help show the long-term effects of Federal research grant funding.
2

EXECUTIVE OFFICE OF THE PRESIDENT
NATIONAL SCIENCE AND TECHNOLOGY COUNCIL
WASHINGTON, D.C. 20502

on participatory feedback from non-Federal sources, including academic researchers, science
practitioners, and others with expertise in the science of science policy.

4. Roles and Responsibilities
Consortium:
An Executive Committee will manage the Consortium. The Executive Committee will consist of
one member from each participating agency or office, to be designated by each of those entities.
The Executive Committee will make decisions regarding the implementation of the STAR
METRICS Program and will be co-chaired by representatives from the founding three entities:
the NIH, the OSTP, and the NSF. Decisions will be made by consensus.
Members of the Consortium are responsible for ensuring that the data sets provided to the
Consortium by their home agencies comply with policy, regulations, and law (e.g., the Privacy
Act of 1974, 5 U.S.C. §552a; OMB Circular A-130, Appendix III according to FIPS 199). Each
agency will designate a point of contact that will carry out this responsibility. Participatory
agencies will provide monthly updates to the Executive Committee. Each agency will also
provide a list of resources (IT, administrative, full-time employee (FTE), and/or knowledge base)
that will be allocated towards the goals of the Consortium, as detailed below.
OSTP:
•

The OSTP will coordinate interagency thematic leadership for the STAR METRICS
through the NSTC Committee on Science;

•

The OSTP will provide appropriate liaison with other components of the Executive
Office of the President as needed;

•

The OSTP will continue to coordinate the recruitment of other science and technology
agencies to join the consortium;

•

The OSTP will oversee communication activities and press releases together with the
NIH and the NSF.

NIH:
•

The NIH Office of Science Policy (OSP), in collaboration with the NIH Office of
Extramural Research (OER), will coordinate STAR METRICS activities at the NIH and
provide administrative support for the STAR METRICS program using funds from the
Consortium;

•

The NIH, through the Office of Science Policy and the Center for Information
Technology (CIT) will employ appropriate contractors to develop needed IT
infrastructure. The contracting officer will be employed by the NIH and contracts will be
issued by the NIH with funds from the Consortium received by participating agencies
through inter agency agreements;
3

EXECUTIVE OFFICE OF THE PRESIDENT
NATIONAL SCIENCE AND TECHNOLOGY COUNCIL
WASHINGTON, D.C. 20502

•

The NIH will ensure all STAR METRICS data protection and dissemination protocols
are compliant with Government-wide policies, laws and regulations (e.g., the Privacy Act
of 1974, 5 U.S.C. §552a; OMB Circular A-130, Appendix III according to FIPS 199);

•

In addition to data collected from research institutions, the NIH will also provide the
Agency’s data required for the STAR METRICS data infrastructure. NIH will comply
with the Privacy Act of 1974, 5 USC §552a.

NSF:
•

The NSF, through the Social, Behavioral and Economic Sciences Directorate, together
with the NIH, will provide outreach to the academic community through the Science of
Science & Innovation Policy program;

•

The NSF will provide staff and expertise to the STAR METRICS Program, dedicating at
minimum 40 percent of one FTE toward the development of the data infrastructure;

•

In addition to data collected from research institutions, the NSF will also provide the
Agency’s data required for the STAR METRICS data infrastructure. NSF will comply
with the Privacy Act of 1974, 5 USC §552a.

5. Implementation
The Consortium will focus on analyzing data and providing adequate metrics to demonstrate the
impacts of Federal investments in science and technology research and development grants on
job creation/retention and other relevant outcomes. Data collected by STAR METRICS will
provide a tool, among others already existing, to inform agencies’ policy making and research
investments evaluation. Metrics will include, but will not be limited to: patents, patent
applications, new businesses, citations in published works, and creation of new fields of study.
The Consortium may reach out to other agencies for their participation and contribution to the
data set as appropriate.
Phase I: Creation of System and Prototype Metrics (Dec 1, 2009 - August 31, 2010)
•

Gather data requirements from universities and agencies;

•

Create IT infrastructure;

•

Develop initial job creation metrics;

•

Develop submission portal for collection of data from universities and agencies;

•

Develop strategy for the acquisition and protection of personally identifiable information
(PII).

4

EXECUTIVE OFFICE OF THE PRESIDENT
NATIONAL SCIENCE AND TECHNOLOGY COUNCIL
WASHINGTON, D.C. 20502

Phase II: Collect and Analyze Data (Upon completion of Phase I)
•

Collect data through portal (including archival agency data);

•

Analyze and evaluate metrics;

•

Prototype data enclave for researchers.

Phase III: Evaluate and Expand (Upon completion of Phase II)
•

Evaluate prototype developed in Phase I and Phase II with Executive Committee and
design full system;

•

Receive feedback from the public through http://www.whitehouse.gov/Open and
http://www.data.gov/;

•

Include other agency data sets as appropriate.

6. Scope and Monetary Commitment
The first year of the STAR METRICS program will display its ability to allocate resources,
analyze data, and produce empirically sound results. Building a strong foundation will create
momentum and motivation for other agencies to join the Consortium, and data will be available
to all universities and science agencies to quickly respond to State, Congressional and OMB
requests.
Below is the FY 2010 monetary contribution of each agency:
OSTP: Contributed $50,000 in FY09 for IT contractor support.
NIH: Agrees to contribute $500,000 per year.
NSF: Agrees to contribute $500,000 per year. The transfer of funds will occur separately
through use of an interagency agreement. For the first year, NSF funds will be transferred to
NIH within 90 days following the signing, by all parties, of this MOU, or the contracting
process may be delayed. In subsequent years, NSF will transfer funds within 90 days of the
commencement of the fiscal year.
However, nothing in this MOU may be construed to obligate the participating agencies to any
current or future expenditure of resources in advance of the availability of appropriations. Nor
does this MOU obligate the participating agencies to spend funds on any particular project or
purpose, even if funds are available.
Further funding of STAR METRICS will be determined based on its success and value.

5

EXECUTIVE OFFICE OF THE PRESIDENT
NATIONAL SCIENCE AND TECHNOLOGY COUNCIL
WASHINGTON, D.C. 20502

This STAR METRICS MOU between OSTP, NIH, and NSF is agreed to by:

___________________________________
Dr. John Holdren,
Director, Office of Science and Technology Policy

__________
Date

___________________________________
Dr. Arden Bement
Director, National Science Foundation

__________
Date

__________________________________
Francis S. Collins, M.D., Ph.D.
Director, National Institutes of Health

__________
Date

6