0990-EvaluMiscondTitusAAMCresppdf (2)

i!.""#' "a"'f-cA"'·"'"

~ -fit,

DEPARTMENT

OF HEALTH

s, HUMAN SERVICES

.,.,~",;~

Office of Public Health and Science
Office of Research Integrity
1101 Wootton Parkway, Suite, 750
Rockville MD 20852

November 20, 2008
Dear Dr. Korn,
We appreciate the interest and comments from the Association of American
Medical Colleges (AAMC) on the design and execution ofthe DHHS Office of Research
Integrity's (OR!) project entitled Evaluating Institutions' Research Misconduct Education
Efforts. We have thoughtfully considered the comments and hereby offer our responses.
We applaud the commitment of the AAMC and its members "to the conduct of
responsible, ethical scientific research" and the strength of their "efforts to prevent
incidents of misconduct and respond vigorously when such incidents arise." However, we
wish to correct the AAMC's impression that this project is an effort to determine from
researchers "how their institutions are ,complying with PHS research misconduct
regulations." Our effort is much more modest and focused. Our goal is to assess the
effectiveness of institutional efforts to assure that their researchers know what the Federal
definition of research misconduct is and know how to report it when they suspect it. We
ask no questions of the researchers about how the institution handles the allegations of
research misconduct that it receives.
We do not understand the AAMC's expression of concern "about the validity of
conclusions that are based on the assessment of individual perceptions of institutional
performance". As indicated above, our assessment of the institution's performance of its
researcher education responsibility will be based upon the ability of its researchers to
identify behavior that falls within the Federal definition of research misconduct, and to
recognize this misconduct in realistic but fictitious scenarios. Survey items tapping
respondent attitudes (perceptions ala AAMC) will be used in analyses to account for
differences in researcher performance across institutions, but not to assess their
institutions.
The AAMC states it is puzzled by our "decision to limit the data collection
activity to the faculty of medical schools". The reasons we limited this survey to
researchers in one institutional domain is a matter of statistical precision - having enough
cases within individual institutions to be able to represent them adequately - and the
unavailability of time and budget to survey other institutional domains. We have given
priority to the medical school domain because, based on Nlli data, the largest proportion
of PHS research funding is awarded to them, and based on OR! records, most of the
misconduct allegations are generated within this domain.
According to its comments, "of greatest importance to the AAMC is that OR!
must indicate explicitly ... that the Federal definition of scientific misconduct is
purposefully narrowly drawn and limited to fabrication, falsification and plagiarism
(FFP)." We do not agree that we must present the Federal definition. A purpose of the

I

survey is to measure whether researchers know what is included in the Federal definition
of research misconduct and are able to distinguish it from many other activities that can
be problematic and unacceptable in the context of research conducted in the institution.
Providing the Federal definition in the invitation and introduction to the survey would be
reactive with our measure and make it useless for its intended purpose. We understand
that an institution's policy may cover more than the Federal definition of research
misconduct and that education about all of these may be presented together. However, we
have been especially careful to always specify when we want the researchers to refer
specifically to the Federal definition when responding to questions as opposed to some
more expansive institutional definition.
With respect to survey question 4 in Section II, the AAMC notes that the answer
options "extend beyond the federal definition ofFFP and may prompt inaccurate
responses to the questions that follow". The answers options do include more than FFP
because OR! wants to measure whether researchers can distinguish between activities
that the Federal definition includes and those others that it does not include. We disagree
that this may result in inaccurate responses in questions that follow because it is clearly
stated that the questions refer to the institution's policy and procedures.
Further, the AAMC comments that "We believe the extensive emphasis in
Sections II and ITIon procedure and process require far more detailed understanding than
should be expected from all faculty members". We disagree with the characterization of
the level of knowledge represented by the items in these survey sections. The items in
Section II deal with whether the researchers have read the institution's policy and how
familiar they say they are with aspects of it that are critically associated with being able
to make allegations of research misconduct, e.g., to whom and how to make their
allegations. The items in Section III deal with when and how researchers are exposed to
the institution's policy and procedures regarding the making of allegations. Typically, the
response options include "Don't Know".
The AAMC also states its objection to the use of terms "such as feel, believe, and
think" in Section IV, fmding them "subjective and emotive". They would prefer the use
of words like "aware, familiar, know", however these terms do not fit the attitudes we
want to measure. Attitudes represent an predisposition to act or think in a certain way
based on a combination of elements which may include their perspective of facts but also
incorporates feelings and beliefs. The items in Section IV request the researcher's
perspective of the institution's educational efforts with respect to research misconduct
and can be either positive or negative.
In its final comment, the AAMC states that many of the cases presented in
Section VII "describe scenarios that, although troubling ethical behaviors, clearly fall
outside the scope of FFP. We are concerned by the fact that OR! once again seems to be
trying to expand its intentionally circumscribed mandate". To the contrary, these items
represent scenarios that ask researchers to demonstrate their ability to distinguish
between the behaviors that may meet the Federal definition of research misconduct and
other undesirable behaviors that are not included in the Federal definition. A recently

2

published study from ORI suggests that researchers are often not able to distinguish
1
.
between them.
The US Federal system for safeguarding the integrity of PHS-sponsored research
is one that depends upon the good faith reporting of alleged misconduct on the part of
researchers by other researchers. This project is an effort to assess from the perspective of
active researchers the effectiveness of efforts being made by their institutions to impart
the relevant elements of the Federal rules regarding research misconduct on projects
funded by the U.S. Public Health Service (PHS). This should include a recognition and
understanding of what constitutes research misconduct according to the Federal
definition. It should also include the elements necessary to report alleged research
misconduct according to the institution's policy, e.g. how and to whom to report, etc. To
carry out its monitoring and educational mission, ORI needs information on whether
researchers know what is necessary to be able to comply with the Federal rules. This
project was designed to collect this kind of information for OR!.

Sincerely,

~/~
Sandra L. Titus, Ph.D.
Director, Intramural Research

1 Titus, S. L., Wells J. A., and Rhoades, L. J. Commentary:
NATURE June 2008; Vol 453: 980-982.

3

Repairing research integrity.