Overview of the study: The Office of Research Integrity (ORI) recognizes the importance of mentoring students in Ph.D. and Ph.D./M.D. programs. This study will use in-depth personal interviews to find out how faculty and their doctoral students who have graduated in the last five years view the doctoral training process to teach responsible research skills. Interviews with matched faculty/doctoral student pairs will provide a unique opportunity to compare these two perspectives. To the best of our knowledge, research that includes matched faculty and doctoral student perspectives on doctoral training and education has not previously been conducted.
This dyad study does not use statistical means to select the sample and there will not be any quantitative data for analysis. There are two kinds of sample members in the dyad study—the faculty member and the matched doctoral student1 who has graduated in the last five years or, as needed, may also include “all but dissertation” students to increase the number of candidates for interviews. The faculty sample will be purposively selected. However, the selection will be based on information from the ORI Faculty Survey,2 which used a representative sample of 2005 and 2006 NIH grant recipients. The matched sample of graduated doctoral students will be developed based on information primarily collected from faculty. Using the information from the ORI Faculty Survey two types of information will be used to guide faculty selection: the term faculty reported they prefer to be called by their doctoral students—advisor or mentor—and faculty perception of the institutional resources for the training and education of doctoral students. Based on the data from the ORI Faculty Survey, Table B.1 has the expected number of the 100 faculty Table B.1
|
Faculty Role Description* |
|
Institution/Program |
Advisor |
Mentor |
Has guidelines** |
36 |
30 |
Does not
have or does not know |
21 |
13 |
*The ORI Faculty Survey results among all faculty, not just those who agreed to be re-contacted, have estimates of those who prefer advisor (54 percent), mentor (38 percent), supervisor (2 percent), and some other name (6 percent).
** Includes faculty who report guidelines for both their institution and graduate program (46%), only for their graduate program (15%), and only for their institution (5%).
members that will be included in the study to represent these two different experiences that can influence a doctoral student’s training experience.
Using these guidelines, we will begin the faculty selection process by listing the names of faculty who answered “yes” to the question “Someone from the study team may contact you in the future as a follow up to this survey. Are you willing to be contacted?”
Among the 1,686 faculty who agreed to be re-contacted, we will focus on faculty at schools located in the northeast corridor to reduce project travel costs. Using this criterion, there are 340 faculty (20 percent of the 1,686) at 28 academic institutions who agreed to be re-contacted.
The graduated doctoral students for the dyad study have to be matched with the selected faculty described in the prior section. Unlike faculty, we do not have a list of the names and background information of the students who have graduated to use for recruitment. The goal is to select and interview faculty who reported they have had at least five students receive their doctoral degrees in the past five years. There are two reasons for this criterion: (1) the possibility of obtaining a student match with a faculty referral increases with more student names and (2) including more names maximizes the privacy of the student reporting on his or her experiences with the faculty member.
The primary method will be to ask selected faculty to provide a list of doctoral students who have graduated in the last five years and, if needed, students they have had in the last five years—even if they have not graduated. This approach will protect the privacy of students who will be interviewed because those who participate will be one among many. With one request we will obtain a sufficient number of names to identify one matched student without having to re-contact the faculty member for additional names. This approach was used successfully for all of the student recruitment in the pilot study. Although the level of information varies, department and faculty websites typically have lists of current doctoral students and alumni. In some cases, this information includes doctoral students’ contact information, which could be used for recruitment.
Based on faculty reports in the ORI Faculty Survey, about 6 in 10 graduate programs and about half of the academic institutions have written policies or guidelines that describe responsibilities of faculty members who work with doctoral students and, as described above, this information will be used to guide faculty selection. The faculty reports about an academic institution’s guidelines can be compared with information on the institution’s websites. In addition, the survey results provide other information about faculty perceptions of institutional and graduate program resources that we can learn more about during the dyad interview. For example, one in five or fewer report the availability of faculty training in advising and mentoring students or developing students’ research skills. Plus, we will have the student’s perspective on this training, or lack of it. When we have recruited the faculty/student dyad for an interview, we will review available information about the institution to have a context for the interview responses related to this topic.
As described in Section B.1, this qualitative study will use a purposive sample. The purposive sample will be selected using representative data from the ORI Faculty Survey. Using two methods—quantitative and qualitative—will increase confidence in the information ORI has about a complex set of relationships among faculty, students, and institutions that they can use to promote the development of responsible researchers.
Estimation procedures will not be used for the qualitative results.
Degree of accuracy is not applicable to qualitative information.
The data collection process described below was successfully tested in a pilot study. Following faculty and student selection as described in Section B.1, the data collection process will be as follows:
Recruit faculty
Selected faculty will be sent an email message (Appendix C.1.a) inviting them to participate in the study. Because we used these email addresses for the recently completed ORI Faculty Survey, we expect them to be valid. The email invitation will (1) thank faculty for their participation in the ORI Faculty Survey; (2) describe the dyad study and ask them to participate; (3) notify them that they will receive $50 as a token of our appreciation; and (4) provide an email address and toll-free telephone number for them to schedule the interview or to learn more about the study. For faculty who do not respond to the email invitation, as needed, we will make follow-up telephone contact using the script in Appendix C.2.a. Faculty who would like more information will be sent (via email, fax, or U.S. Postal Service) Frequently Asked Questions (Appendix C.5.a).
Develop doctoral student list
When the faculty member agrees to an interview, he or she will be asked to provide eligible students’ names (doctoral students who have graduated in the last five years) and contact information. We expect to get a minimum of five students’ names per faculty member. Although the preference is for doctoral students who have graduated, we will also obtain names of current students who are close to completing their doctoral programs to expand the number of potential students to achieve a match. Getting a complete list of graduates in the past five years will minimize possible bias of having faculty exclude problem students. Appendix C.1.b includes the email message that will be used to obtain students’ names. As with the initial contact, faculty who do not respond to the email request will be contacted by telephone to provide students’ names.
Contact students
Similar to the faculty recruitment, each student whose name we receive from a selected faculty member will be sent an email invitation to participate in the study (Appendix C.1.c). The email invitation will (1) describe the study and invite students to participate; (2) inform students how we obtained their names and contact information; (3) notify them that they will receive $50 as a token of our appreciation; and (4) provide an email address and toll-free telephone number for them to schedule the interviews or to learn more about the study. For students who do not respond to the email invitation, as needed, we will make follow-up telephone contact using the script in Appendix B.2.b. If more than one student expresses interest in participating, they will be prioritized based on criteria such as gender, type of degree (Ph.D. or Ph.D./M.D.), current geographic location, and availability during the scheduled interview period.
Schedule and confirm interview
We will make sure that we have a matched pair—with both a faculty member and one of his or her students agreeing to an interview—and then we will schedule the date, time, and location for each interview. Faculty and students will be interviewed separately. In addition, although faculty and students may inform each other that they are participating in the study, we will not communicate this information. A confirmation message will be sent by email, fax, or U.S. Postal Service (Appendix C.4) and the day prior to the interview a confirmation email or call will also be made. If either member of the pair—faculty or student—is not interested or is not available, we will select a replacement faculty member and begin the process again.
Conduct interviews
Faculty and student interviews will be conducted in person. As needed, telephone interviews will be considered if there are high-priority students whose geographic location prevents an in-person interview. Faculty interviews will be scheduled prior to student interviews to provide a core set of information about the graduate student experience. Interviews will be conducted by trained, full-time Mathematica professional staff including the three who conducted the pilot interviews. Up to four additional staff will be trained to use the interview protocols and will listen to pilot interviews to prepare for their assignments. Interviews will be conducted using faculty and student protocols. Faculty and student interview protocols (Appendixes C.3.a and B.3.b, respectively) have been designed to standardize questions about the topics of interest for the study, to follow up on information from the ORI Faculty Survey, and to address the attributes of faculty/student relationships identified in the conceptual framework.
The interviews will take approximately 1.5 to 2 hours and will be audio-recorded to facilitate analysis. Faculty and student consent forms (Appendices B.6.a and B.6.b, respectively) will be reviewed and signed prior to beginning the interview. Participants will sign a receipt form and receive their $50 after the interview is completed. Following the first interview by each interviewer, there will be a debriefing among all interviewers to discuss the protocol and any other aspect of the interview logistics. Periodic debriefings will be conducted with the team, at weekly meetings or as needed, to inform the interview process.
This survey has a single data collection cycle.
The list of faculty for the dyad study previously agreed to be re-contacted for additional research. For their convenience, faculty can select the date, time, and location for the interview. For selected faculty who do not respond to the initial email invitation, we will make telephone calls and up to two additional email attempts. Professional, highly trained interviewers will make the telephone contacts. These will be the same interviewers who will meet with faculty members to conduct in-person interviews so they will have comprehensive information about the study to use in converting reluctant participants. Since we will have profiles of the selected faculty members based on their responses to the ORI Faculty Survey, we will be able to identify any systematic differences between those who do and those who do not participate in the survey. We expect that doctoral students will be interested in participating and, since they will know their names were provided by a faculty member, will find the study credible. The follow-up telephone calls and email procedures used for faculty will also be used for the doctoral students.
Nine interviews were conducted with faculty and nine with doctoral students to pilot test the data collection process and interview protocols. The selection and recruitment procedures described above achieved the variation in faculty profiles and academic institutions that we targeted. We were able to obtain matched pairs of faculty and students with less effort than expected. Faculty were not hesitant about providing the names and contact information of doctoral students who graduated in the last five years. Since this student information is typically available on public-access websites, potential concerns about privacy were minimal. Protocol instruments were carefully assessed for terminology, clarity, sensitivity, and relevance. Faculty and students who participated in the pilot responded positively to the process and the topics covered during the interview. The testing was used to provide an estimate of respondent burden for completing the interview.
The following people were consulted on the technical aspects of the study design:
Sandra Titus, Office of Research Integrity, 240-453-8437
Janice Ballou, Mathematica Policy Research, 609-750-4049
Gail Baxter, Mathematica Policy Research, 609-936-2787
Eric Grau, Mathematica Policy Research, 609-945-3330
Frank Macrina, Virginia Commonwealth University, 804-827-2262
Frank Potter, Mathematica Policy Research, 609-936-2799
Brian Roff, Mathematica Policy Research, 609-750-4041
This group consists of survey methodologists and statisticians who have extensive experience in the design and implementation of both qualitative and quantitative data collection. Frank Macrina is the subject matter expert on the team.
Arce, C., and W.H. Manning. “Minorities in Academic Careers: The Experience of Ford Foundation Fellows.” New York: Ford Foundation, 1984.
Ballou, Janice, Eric Grau, and Brian Roff. “Office of Research
Integrity Faculty Survey:
Technical Report.” Princeton,
NJ: Mathematica Policy Research, 2009.
Ballou, Janice, and Brian Roff. “Office of Research Integrity Faculty Survey: Profile of Advisor/Mentor Training and Education of Doctoral Students.” Princeton, NJ: Mathematica Policy Research, 2009.
Blackwell, J. Mainstreaming Outsiders: The Production of Black Professionals. Second edition. Dix Hills, NY: General Hall, 1987.
Campbell, Toni A., and David E. Campbell. “Faculty/Student Mentor Program: Effects on Academic Performance and Retention.” Research in Higher Education, vol. 38, no. 6, 1997, pp. 727-742.
Council of Graduate Schools, Ph.D. Completion Project, “Ph.D. Completion and Attrition: Policy, Numbers, Leadership, and Next Steps.” Washington, DC: CGS, 2004.
Cronan-Hillix, T., L.K. Gensheimer, W.A. Cronan-Hillix, and W.S. Davidson. “Students’ Views of Mentors in Psychology Graduate Training.” Teaching of Psychology, vol. 13, 1986, pp. 123–127.
DesRoches, David, Thomas Barton, Janice Ballou, Frank Potter, Zhanyun Zhao, Betsy Santos, and Jaceey Sebastian.”Kauffman Firm Survey Baseline Survey Methodology Report.” Princeton, NJ: Mathematica Policy Research, 2007.
Guston, D.H. “Mentorship and the Research Training Experience.” In Responsible Science, Volume II: Background Papers and Resource Documents. Washington, DC: National Academy of Sciences, 1993. [www.nap.edu/openbook/0309047889/html/50.html]. Accessed June 11, 2006.
Haring-Hidore, M. “Mentoring as a Career Enhancement Strategy for Women.” Journal of Counseling and Development, vol. 66, 1985, pp. 147–148.
Hartnett, R.T. “Environment for Advanced Learning.” In Scholars in the Making: The Development of Graduate and Professional Students, edited by J. Katz and R.T. Hartnett. Cambridge, MA: Ballinger Publishing, 1976, pp. 49–84.
Institute of Medicine, National Research Council of the National Academies, Committee on Assessing Integrity in Research Environments, Board on Health Sciences Policy and Division of Earth and Life Studies. “Integrity in Scientific Research: Creating an Environment That Promotes Responsible Conduct.” Washington, DC: The National Academies Press, 2002.
Langlais, P. “Ethics for the Next Generation.” Chronicle of Higher Education, January 13, 2006. [http://chronicle.com/weekly/v52/19/19b01.htm]. Accessed August 15, 2006.
Martinson, B., M. Anderson, and R. de Vries. “Scientists Behaving Badly,” Nature, vol. 435, June 2005, pp. 737-738.
National Academy of Sciences. “Adviser, Teacher, Role Model, Friend: On Being a Mentor to Students in Science and Engineering.” Washington, DC: NAS, 1997. [www.nap.edu/readingroom/books/mentor/1.html]. Accessed June 11, 2006.
National Academy of Sciences. On Being a Scientist. Third Edition. Washington, DC: NAS, 2009. [www.nap.edu/catalog/12192.html]. Accessed September 23, 2009.
National Science Foundation. “Science and Engineering Indicators. Natural Sciences and Engineering Doctoral Degrees, by Selected Country: 1983–2003.” [http://nsf.gov/statistics/sein06/c2/fig02-34.xls]. Accessed August 15, 2006.
Nettles, M., and C. Millet. Three Magic Letters: Getting to Ph.D. Baltimore: Johns Hopkins Press, 2006.
Office of Research Integrity. “Responsible Conduct in Research Mentoring.” [http://ori.hhs.gov/education/products/niu_mentorship/mentoring/relationship/relationship.html]. Accessed August 15, 2006.
Office of Research Integrity. “Survey of Research Integrity Measures Utilized in Biomedical Research Laboratories. Final Report.” [www.ori.hhs.gov/documents/research/ integrity_measures_final_report_11_07_03.pdg]. Accessed July 17, 2006.
Patton, Michael Quinn. Qualitative Evaluation and Research Methods. Newbury Park: Sage Publications, 1990.
“Ph.D. Completion Project.” [http://www.phdcompletion.org/ information]. Accessed June 11, 2006.
Steneck, Nicholas H. “Fostering Integrity in Research: Definitions, Current Knowledge, and Future Directions.” Science & Engineering Ethics, vol. 12, no. 1, 2006, pp. 53-74.
Thompson, Linda, and Alexis J. Walker. “The Dyad as the Unit of Analysis: Conceptual and Methodological Issues.” Journal of Marriage and the Family, vol. 44, no. 4, November 1982, pp.889-900.
Titus, Sandra L., J. Ballou and B. Roff. “Analysis of Roles, Responsibilities, Rewards, and Institutional Involvement with Mentors and Advisors.” Presented at the Fifth Office of Research Integrity Conference on Research Integrity, Niagara Falls, NY, May 15-17, 2009.
Titus, S. L., J. A. Wells, and L. J. Rhoades. “Repairing Research Integrity.” Nature, vol. 453, no. 7198, 2008, pp. 980-982.
University of Toronto. “Graduate Supervision: Guidelines for
Students, Faculty, and Administrators.” Toronto, Canada, 2002.
[www.sgs.utorontoca/current/supervision/
guidelines.pdf].
Accessed on July 17, 2006.
Wright, David E., Sandra L. Titus, and Jered Cornelison. “Mentoring and Research Misconduct: An Analysis of Research Mentoring in Closed ORI Cases.” Science and Engineering Ethics, vol. 14, no. 3, 2008, pp. 323-336.
1 The doctoral students matched with faculty members have graduated in the last five years. Since the research focuses on the training and educational experience while they were students, they are referenced as doctoral students. Students’ doctoral experiences are cumulative over the time they are working on their degrees. Graduates will have a complete set of research training and educational experiences.
2 For the ORI Faculty Survey, a national random sample of 10,000 2005 and 2006 NIH grant recipients was selected from publicly available information. The 1,686 faculty members who agreed to be re-contacted are a sub-set of the original ORI Faculty Survey sample.
| File Type | application/msword |
| Author | DHHS |
| Last Modified By | DHHS |
| File Modified | 2009-12-18 |
| File Created | 2009-12-18 |