6 Attachment G Interview Protocol-Grant Reviewer

Attachment G:

INTERVIEW PROTOCOL- REVIEWERS

OMB No.: 0925-0046

Expiration Date: 05/31/16

Collection of this information is authorized by The Public Health Service Act, Section 411 (42 USC 285a). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private to the extent provided by law. Names and other identifiers will not appear in any report of the study. Information provided will be combined for all study participants and reported as summaries. You are being contacted to complete this interview so that we can learn more about satisfaction with the Provocative Questions Initiative.

Public reporting burden for this collection of information is estimated to average 30 minutes per response, including the time for completing the interview. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0046).

DATE OF INTERVIEW:

LENGTH OF INTERVIEW (MINUTES):

NAME OF INTERVIEWEE:

AFFILIATION (AGENCY OR INSTITUTION):

TELEPHONE NUMBER:

E-MAIL:

PRIMARY INTERVIEWER:

SECONDARY INTERVIEWER/WRITER:

COMMENTS ON ANY UNUSUAL CIRCUMSTANCES:

INTRODUCTION

Hello, _________________. My name is _________________ and I am a member of the assessment team working with the Center for Strategic Scientific Initiatives (CSSI) within the National Cancer Institute (NCI). We are working on a project that is assessing the process and outcomes of NCI’s Provocative Questions Initiative. I work for The Madrillon Group, a firm in the metropolitan Washington, DC region, which has been retained to assist CSSI with this assessment.

We greatly appreciate your willingness to help us with this project. We understand that your participation in this interview is voluntary, and we want to assure you that the information we gather from this and other interviews will only be used in aggregate. Responses from individuals will not be identified by name or shared outside the assessment team.

In order to capture your valuable input, I’d like to record this interview. The recordings will only be used internally and will be destroyed when the project is completed.

Do I have your permission to record this interview?

[] YES Thank you.

[] NO This interview will not be recorded. Rather, I will take notes on our conversation.

The interview will include open-ended questions on your views about two topics--the process of developing the Provocative Questions and the scientific outcomes as a result of the Provocative Question Initiative.

Do you have any questions before we begin?

1. REVIEWER’S BACKGROUND

1. I understand that you have served as a reviewer for the Provocative Questions Initiative for [length of time or number of reviews inserted here]?

   1. Were you involved at the start of the Initiative?

   2. What is your involvement now?

2. I understand that you [insert brief review experience summary here]. Is this correct?

3. Apart from your role as a reviewer, I understand that you have [insert brief description of this reviewer’s role here] with the NCI Provocative Questions Initiative questions development process? Does this describe it correctly?

4. Do you currently have, or have you had in the past, any other involvement with NCI in a leadership or advisory capacity, such as the Board of Scientific Advisors, or other boards or committees?

2. PROVOCATIVE QUESTIONS REVIEW PROCESS

A. THE REVIEW PROCESS

5. Please comment on the review process for the Provocative Questions Initiative.

1. How is the process different or similar to from the review processes you have been involved with for other NCI or other NIH grant programs?

2. Was there a pre-review orientation or were materials provided to you in advance? If so did you attend the orientation? Were the orientation and/or materials helpful in providing the groundwork for your review efforts?

3. Were the provocative question(s) you were focused on for this review one(s) that you had preconceived notions of? Did you have direct research experience in the question(s) you served as a reviewer for?

4. In your opinion, were the instructions clear on how the Provocative Questions applications needed to be reviewed differently from other applications?

6. In your opinion, were the Provocative Questions applicants proposing innovative, high risk, or novel research? Please give examples, if any.

   1. Did you notice a difference in the degree of innovation, risk or novelty of the research proposed in Provocative Questions R21s compared with Provocative Questions R01s?

7. Do you think that the range of expertise of reviewers on the Provocative Questions review committee was sufficiently broad? [Probe: How did it compare to other study sections on which you have served? Do you think the range was appropriate for the applications being reviewed?]

8. Since the Provocative Questions target understudied and difficult to address areas and novel approaches are encouraged, did you have a more difficult time ascertaining the quality of the applications? Did you have trouble assessing the feasibility of the proposed concepts and approaches, as compared to other programs for which you have reviewed? Why or why not?

   1. Did other reviewers on the committee have a more difficult time ascertaining the quality and the feasibility of the applications? Why or why not?

9. Did you weight any of the review criteria differently because this program was soliciting high risk research? If so, how?

10. Based on your experience, did Provocative Questions Initiative applications have other unique or different qualities or features compared to more typical grant applications?

11. In your opinion, were the applicants taking advantage of the invitation to propose research with less preliminary data, or was the amount of preliminary data provided similar to applications for other comparable types of grants you’ve reviewed? Please give examples, if any.

12. In your opinion, in general do PQ reviewers evaluate an application for more novel, high risk research differently than an application for less novel, less high risk research with more preliminary data? If so, how?

13. To date do you think the overall Provocative Questions Initiative has succeeded in fostering novel, innovative, and high-risk research? If so, how? Please give examples, if any.

3. INTERVIEW CONCLUSIONS

14. Do you believe the Provocative Questions Initiative is a valuable program for NCI? Why or why not?

15. If you could, what improvements (if any) would you make to the review process for the Provocative Questions Initiative? To the initiative itself?

16. Is there anything else regarding the initiative that you think is important for us to know?

THANK YOU FOR YOUR TIME AND PARTICIPATION IN THIS PROJECT!

4