Supporting Statement Part B FIC

3

Supporting Statement B for

Assessing the Long-term impacts of research and training programs supported by the John E. Fogarty International Center (FIC)

Date: August 18, 2010

Name: Rachel Sturke, PhD

Address: Fogarty International Center, 31 Center Drive - MSC 2220, Bethesda, MD 20892-2220

Telephone: 301.480.6025

Fax:

Email: sturkerachel@mail.nih.gov

Contents

B.1 Respondent Universe and Sampling Methods 3

B. 2. Procedures for the Collection of Information 3

B.2.2 Data Analysis Procedures 3

B.3. Methods to Maximize Response Rates 3

B.4. Test of Procedures or Methods to be Undertaken 3

B.5. Individual Consulted on Statistical Aspects and Individual Collecting and/or Analyzing Data 3

List of attachments

Attachment 1: Consent Form for Participants

Attachment 2: Estimate of hourly rate for foreign researchers

Attachment 3: Interview Guide for US-Based Principal Investigators

Attachment 4: Case Study Questionnaire for FIC Trainees

Attachment 5: Focus Group Discussion Guide for Trainees

Attachment 6: Interview Discussion Guide for Trainees (University-Based)

Attachment 7: Interview Discussion Guide for Trainees (Non University-Based)

Attachment 8: Interview Discussion Guide for University Leaders

Attachment 9: Interview Discussion Guide for Policy-Makers and Scientific Leaders

Attachment 10: Conceptual Framework

Attachment 11: Research Questions

B.1 Respondent Universe and Sampling Methods

For each case study institution, the universe of potential informants consists of all individuals who have participated in or have knowledge of FIC research training programs. Specifically, the case study design includes five categories of informants: 1) US-based principal investigators on research training awards partnering with case study institutions; 2) current and former trainees; 3) leaders and administrators at case-study institutions; 4) direct FIC grantees at case study institutions; and 5) policy-makers and/or scientific leaders in case study institution countries. Estimates of the respondent universe for each category and anticipated number of participants are summarized in Table B.1.1.

Table B.1.1. Estimated Respondent Universe and Use of Sampling Methods by Category

Category of Participant	Estimated Size of Respondent Universe per Case Study	Method of Estimation	Expected number of Participants per Case Study	Sampling Methods Used to Reduce Burden?
1) US-based principal investigators on research training awards partnering with case study institutions	20	Based on NIH records of number of awards to US PIs at first two case study institutions	20	No
2) former trainees	300-600	Estimate based on FIC records of trainees associated with first two case study institutions	40 (focus groups) 13 (interviews)	Yes
3) leaders and administrators at case-study institutions	unknown		4	Yes
4) direct FIC grantees at case study institutions	20	Based on NIH records of number of awards to foreign PIs at first two case study institutions	20	No
5) policy-makers and/or scientific leaders in case study institution countries	unknown		8	Yes

For categories 1 and 4 (US-based PIs and foreign PIs), sampling methods have not been used to reduce burden because each PI is expected to have a unique experiences to share that are specific to his or her particular awards. An understanding of the role played by each award will be essential to the case studies.

For categories 3 and 5 (university leaders and policy-makers or scientific leaders), the universe of respondents with potentially relevant knowledge is difficult to estimate. Snowball sampling methods will be used to identify individuals with knowledge of particular impacts specific to each case.

For category 2 (former trainees), the universe of potential respondents is quite large, and sampling methods will be used to reduce burden. Specifically, trainees listed in the FIC database of trainees (CareerTrac) will be stratified by the type of training program in which they participated. Trainees from a representative sample of training programs relevant to the case study institution will then be randomly selected to participate in focus groups (4 per case consisting of 10 individuals each) and interviews (13 per case).

B. 2. Procedures for the Collection of Information

B.2.1 Data Collection Procedures

Semi-structured interviews will be conducted with Principal Investigators on FIC grants with links to the case study institutions via telephone prior to the site visit. A discussion guide for this data collection activity is included as Attachment 3.

During the in-country site visit, the study team will use in-depth semi-structured interviews with the various categories of respondents described above, plus a short structured survey of and focus groups with FIC trainees. The specific data collection instruments to be used include the following:

• A short, structured survey (Attachment 4) to be be administered to those FIC trainees who participate in focus groups, before the start of the focus group. This survey was designed based on previously administered research instruments found in the literature and is targeted around several specific areas, such as: FIC trainee’s job satisfaction, motivation to conduct research, and social networks. The survey will require approximately 30 minutes of a participant’s time.

• Semi-structured focus groups with former FIC trainees in order to gain an understanding of shared experiences and organizational norms as related to research and training experiences. Focus groups will be facilitated by experienced study leaders who are knowledgeable about best practices for this data collection method. It is anticipated that these discussions will also be used as a foundation to inform and adapt the in-depth interviews. Focus groups will require approximately 90 minutes of each participant’ time. The focus group discussion guide is included as Attachment 5.

• Semi-structured interviews with various groups of informants that will be used to gain a thorough understanding of participant perceptions and individual experiences as related to the FIC program. Interviews will be conducted in a location to be arranged by a collaborator at the case study institution and will require approximately 60 minutes of a participant’s time. Interview guides have been developed for each of the following groups:

1. Former FIC trainees currently employed in university settings (Attachment 6);

2. Former FIC trainees not currently employed in university settings (Attachment 7);

3. Leaders and administrators at case study institutions (Attachment 8); and

4. Policy-makers/scientific leaders in the countries of interest (Attachment 9).

B.2.2 Data Analysis Procedures

The data will be analyzed upon return to the US, although preliminary impressions from the data collection will be shared with university leadership prior to departure from the country. Draft findings will be shared with FIC colleagues and the expert panel. The study team will maintain confidentiality of study participants and will not use personal identifiers for any of the information shared or opinions stated, instead respondents will be referred to according to which category they fall into (eg. trainee, university leader, policy maker).

Due to budget and time constraints transcripts will most probably not be made of all qualitative data (although we are looking into the costs of making full transcripts). Rather detailed notes (both handwritten and computer based) will be taken during the interview, and the interviews will be recorded. From these sources a final detailed account of the interview using selected quotations where appropriate will be made. Qualitative data will be analyzed using a software package such as Atlas TI or NVivo.

The data analysis will be centered around the constructs identified in the conceptual framework (Attachment 10) and related research questions (Attachment 11). The study team will use direct interpretation to reorganize the data gathered around the constructs in the conceptual framework. Along this process, the study team will establish patterns between two or more conceptual constructs and experiences from those interviewed.

Quantitative data from the trainee questionnaire will be entered into a database and analyzed using a software package such as Stata, SAS or SPSS. The analysis will include both a descriptive analysis of the career paths of FIC trainees (encompassing for example, their success in attracting additional research grants, and in publishing, and the self-perceived impact of FIC training) but will also seek to analyze how environmental factors, such as how the availability of resources to support their work, and adequacy of remuneration have intervened to affect their productivity and satisfaction.

Data regarding professional collaborations and contacts (section 5 of the case study questionnaire) will be analyzed using social network analysis in order to explore the strength of linkages between those involved in FIC programs, and between FIC trainees and researchers trained through other mechanisms. This analysis will employ software such as NodeXL and will seek to document the extent to which FIC support has led to the development of identifiable professional networks.

Internal validity checks will be carried out through collecting input from the in-country research collaborator, debriefing with University leadership and key informant interviews. Discussions with US based PIs for the respective countries and review of FIC evaluations of study country programs will also be used to validate the findings for each of the case studies. The full internal validity will be limited as the case studies have not been selected through randomization.

External validity checks will be carried out through discussions with the FIC-convened expert panel.

B.3. Methods to Maximize Response Rates

As the proposed data collection is a series of case studies rather than a true statistical survey, response rates will not be calculated. However, the site visits themselves are viewed as a method intended to maximize participation; FIC has learned from experience that, while individuals who have benefited from its international training programs are generally willing and often eager to discuss their experiences, it can be difficult to collect information from individuals in foreign countries via telephone interviews or online surveys. To further maximize participation, the study will employ an in-country collaborator at each institution who will work closely with the study team on logistics, coordination, and recruiting of participants. It is believed that a local collaborator who is familiar with the local language, customs, and institutional culture will have the best chance of maximizing participation.

B.4. Test of Procedures or Methods to be Undertaken

As the respondent group for each data collection instrument is small, a formal pretest of the instruments will not be undertaken before they are fielded. However, the first case study will essentially serve as a pilot for the rest of the study. Methods will be reviewed carefully after the first site visit and adjusted as necessary.

B.5. Individual Consulted on Statistical Aspects and Individual Collecting and/or Analyzing Data

The design for this study was developed primarily by the contractor, Dr. Sara Bennett, an Associate Professor at the Johns Hopkins Bloomberg School of Public Health. Several individuals at FIC also participated in the development of the study plan. Dr. Bennett and her graduate assistants will have primary responsibility for data collection and analysis. Dr. Bennett’s contact information is:

Sara Bennett

Associate Professor, Department of International Health

Johns Hopkins Bloomberg School of Public Health

Telephone: +1 410 614 5137

E-mail: sbennett@jhsph.edu