Supporting Statement Proteomics

This request constitutes a new collection, the purpose of which is to ascertain the perspectives of selected individuals internal and external to the Proteomics Centers Program. A set of qualitative, semi-structured interviews will be conducted to gather position and role-specific insights as to the attainment of specific outcomes related to the coordination, facilitation, management, and operation of a center-based model of scientific inquiry in proteomics research. The key informant interviews are expected to generate information that will enable a response to the following general evaluation questions:

To what extent has the program developed and shared new tools and technologies?
What outputs of the program have matured enough to be of clinical utility? To what extent has the program resulted in inventions, patent applications, spin-off companies, and licensing agreements?
To what extent has the research contributed to the creation and integration of a knowledgebase linking changes in proteomes with molecular phenotypes of disease?
In what areas has the program filled existing knowledge gaps in heart, lung, and blood disease biology?
To what extent have the program investigators been collaborative within and outside their fields?
To what extent was subsequent science informed by the research conducted by the program?
To what extent has the program served as a springboard for subsequent academic appointments and professional recognition?

The interviews will be conducted with key informants from 4 specific groups: (1) principal investigators and key personnel from the seven centers; (2) proteomics investigators not funded by the NHLBI program (external investigators); (3) trainees (current and former) and junior investigators; and (4) NIH staff. Using the primary evaluation questions as the guiding framework for establishing the specific interview questions relative to each development, Concept Systems, Inc. will use the attached interview guides (see appendix) to draw insights from multiple perspectives and triangulate sources of information in an effort to confirm observations and conclusions. Key informants are viewed as individuals who are particularly knowledgeable and articulate, and who have insights that prove particularly helpful in understanding the work of the program. Key informant interviews are necessary in this situation and are preferred over self-reported survey methodologies for several reasons, including:

flexibility and adaptability to the context, questioning, and response;
ability to clarify in situ and facilitate immediate follow-up for more precise understanding;
capture rich and detailed descriptions of experience and events in the language of respondents;
facilitates responsiveness to the respondents and the inquiry;
allows for understanding to emerge within and across interviews;
does not assume or require a priori knowledge to be tested or validated

Given the variability of experience, expectations, and opinions across centers and roles, key informant interviews provide the necessary flexibility and can be appropriately structured to identify and account for common and disparate perspectives. In order to ensure that the semi-structured interview guides appropriately frame the perspective of individuals from the groups, we will implement a pilot testing phase with a select set (less than 10 total) of volunteers from the program.

Sample for Qualitative Interviews

For the final set of key informant interviews, volunteers will be recruited from each of the four groups listed below in Table 1. A set of semi-structured interviews using a group-specific interview guide will be conducted in person at the annual Proteomics Principal Investigator meeting in Bethesda, MD. For those key informants in each of the four groups not able to participate in the face-to-interview at the Proteomics Principal Investigator meeting, either due to attendance at the meeting or scheduling conflicts while at the meeting, a virtual interview (Skype or teleconference) will be arranged. The tentative goal involves interviewing the following number of people from the pilot and final interviews: (1) twenty seven participants from the Principal Investigators and key personnel group; (2) ten current and ten former trainees and/or junior investigators; (3) nine external investigators (investigators in the field of proteomics who are not funded by the program; and (4) five NHLBI and two NCI staff members. Based on the targeted number of key informants across groups, we expect to complete approximately 35 interviews at the meeting and 26 interviews outside of the meeting. Table 1 below outlines the anticipated sampling frame.

Table 1. Key Informant Interviews: Sampling Frame

	Estimated Number
Group	Proteomics Meeting	Outside of Meeting	Total
Principal investigators and key personnel from the seven centers	22	5	27
Proteomics investigators not funded by the NHLBI program (external investigators)		9	9
Trainees (current and former) and junior investigators	10	10	20
NIH staff	5	2	7
Total	35	26	63

Data collection plans

Key informants identified in these respective groups have unique perspectives as to the successes and limitations of the program. Therefore, it is critical that the semi-structured interview guides are designed and facilitated in a way that enables individuals to provide information shaped by their experiences, in the context of the overarching questions. In general, the key informant interview questions will focus on the areas listed in Table 2 below. Using the primary evaluation questions as the guiding framework for the specific interview questions tailored for each group, the interview guides will draw insights from multiple perspectives and triangulate sources of information in an effort to confirm observations and conclusions. Refer to the semi-structured interview guides in the Appendix for the specific questions for each group developed under the areas of inquiry rubric.

Table 2. Areas of inquiry for the Key Informant Interviews

Area of inquiry	Description
Development and sharing of new tools and technologies	The program seeks to develop proteomic technologies to gain a greater understanding of physiological pathways, molecular interactions, and regulatory signals related to heart, lung, blood, and sleep diseases and disorders. Tools developed through the program will be made available to the larger community to further advance heart, lung, and blood research.
Mature, clinically useful outputs - i.e. inventions, patent applications, spin-off companies, and licensing agreements	The program will develop and yield methods, tools, and techniques (technology products) that can be applied to a number of clinically-relevant questions in heart, lung, and blood research. The program seeks to bring innovation to commercial fruition through partnerships with industry.
Contribution to the creation and integration of a knowledgebase	The program will create a knowledge base to facilitate translation of innovative proteomic approaches and technologies to clinical utility. The knowledgebase will constitute the integration of multi-disciplinary proteomic data sets in the context of cardiovascular biology and medicine.
Filling of existing knowledge gaps	Biological and clinical knowledge gaps exist in the understanding of heart, lung, and blood conditions. The program is purposefully targeting proteomic tools and technologies to address relevant and compelling biological and clinical questions. The program seeks to produce innovative methods and techniques, the application of which will advance knowledge in diagnostics and therapeutics.
Collaboration among investigators	Principal investigators work together, concentrating on a particular innovation, then collaborate extensively with clinical researchers. The program anticipates that investigators will develop and maintain internal (within center) and external (outside of the center) collaborations, as well as international partnerships in the pursuit of program goals.
Research informs subsequent science	Center research is expected to influence the future research on the identification of new paradigms, potential therapeutics, and new biomarkers. Clinically actionable research on existing and emerging proteomics approaches is anticipated to revolutionize the understanding of clinical issues (e.g., heart failure) and facilitate rapid translational research.
Subsequent academic appointments and professional recognition	As an indispensable resource young investigators are dedicated and creative. The program seeks to serve as a training ground and platform for professional advancement and recognition. Academic advancement, honors, and awards are key indicators of the development of capacity to support future cutting-edge research.

For the key informant interviews, interviewers from CSI will capture the content through digital recordings, from which summaries and transcripts will be produced. Through an analytic memoing process developed by the principal evaluator, the interviewers will use the interview guide to record insights and summarize reflections after each interview. The analytic memos generated by the interviewers will enable the principal evaluator to ascertain the degree to which the interview process (semi-structured interview guide and the implementation of the interviews) are generating information that will enable key questions to be answered. CSI will transcribe the interviews, and a designated quality reviewer will initiate a quality check of the transcription with the digital recording. This will also enable the principal evaluator to routinely conduct data audits on qualitative interview procedures as well as the data captured during the interview to ensure that the guide is being used as designed and that the guides are yielding the anticipated information. In addition, CSI interviewers will follow up with the interview participants, if necessary, to ensure validity of the data.

Once the transcribed interviews have been checked for completion and errors, the principal evaluator will lead the development of the qualitative data reduction and coding process, following an iterative analytic process using multiple analysts on the team to enhance the credibility of the findings. A codebook will be created to aid in the analysis of the data among multiple analysts. It is through the application of this process that the qualitative interview data will be analyzed, and the thematic results interpreted in the context of the broad evaluation questions. Through CSI's analytic approach to the interview data, the use of participant language and verbatim accounts to highlight and amplify key themes that emerge from the analysis will be emphasized.

Analysis plan

The data reduction and coding plan for analyzing the qualitative information will be based on the need to identify specific information that addresses the evaluation questions. Unlike more general qualitative research, the analysis of qualitative interview data in evaluation requires a utilization-focused approach to keep the findings from becoming too abstract, esoteric, or theoretical and ensure a high degree of relevance to stakeholders in the current system.

The contracted evaluator will analyze the interview data through a cross-case analysis whereby the purpose is to group together answers from different people to common (or similar) questions or responses to central issues. Through the efficient application of qualitative coding methods, we will seek to identify the patterns of practice among centers consistent with the general evaluation questions presented above. Because these findings will be inextricably linked to the setting and context, uncovering explicit details and providing “thick description” as support will be critical. CSI's approach to analysis is built upon the assumptions that the application of qualitative evaluation methods is concerned with process as well as the outcomes or products. Thus, CSI's approach is interested in meaning and meaning-making; that is, in understanding how people make sense of their lives, experiences, and the structures their worlds. Consistent with Guba & Lincoln's (1990)³ four criteria for judging the soundness and quality of qualitative research and evaluation, the design and implementation of CSI's qualitative approach will seek to ensure: credibility or the degree of confidence in the “truth” that the findings have for the participants and in the context of the inquiry; transferability of the results to other contexts or settings describing the context and the assumptions that were central to the inquiry; replicability or repeatability in terms of obtaining the same results by describing the changes that occur in the setting and how these changes affected the way the inquiry was approached, and; confirmability or corroboration of the results by documenting the procedures for checking and rechecking the data throughout the inquiry.

Because CSI will be collecting data in the participants' language by capturing verbatim accounts of experiences, the evaluation team will be managing an extensive volume of narrative data. In the content analysis of the qualitative information, CSI will be employing an iterative, data-reduction process across multiple data sources and analysts. Attention on the collection of “low inference descriptors” or those concrete examples that require very little interpretation and are easily recognizable as a universal experience or situation will enhance the validity of the data and increase the acceptance of the findings. Thus, an extensive coding strategy will need to be employed to identify patterns in the data that address the evaluation questions with clear, concrete descriptors and exemplars. The data reduction and coding process will begin with a chunking process to organizing the transcribed narrative. CSI analysts will chunk the narrative data according the inquiry domain framework developed in consultation with the Advisory Panel. Within this framework, CSI will further divide and organize the data by broad structural themes, such as chronology, key events, settings, people, processes, issues, and results. Once the narrative data has been organized in these structural categories, CSI will employ a coding procedure to sections of the text. Specifically, CSI will use a Keyword-In-Context (KWIC) approach (Krippendorf, 2004)⁴ to code the textual information. The KWIC approach uses simple keywords found in the actual text as semantic anchors of the section of narrative or passage. These keywords can then be grouped into code words that encompass multiple keywords. This process will yield a dynamic codebook, that will be refined and expanded over subsequent analyses of interview data. Analytically, CSI anticipates that the pilot data will produce a codebook that will be applied, refined, and expanded upon in the final set of interviews. The codes will be used to identify patterns in response to the evaluation questions, and because they are linked to specific qualitative descriptors and exemplars, provide rich contextual information. The codes can then be entered into a relational database, with linkages to actual narrative for further pattern analysis.

As CSI looks to identify patterns in the data, we will develop a cross-methods matrix that will be used to cross-validate findings, patterns, and conclusions through a process of examining what information diverges and/or converges (Guba, 1978)⁵. Briefly, as patterns are built upon what is already known, we will identify qualitative information that makes connections among different items, proposes new information that ought to fit within this structure, and verifies the existence of the pattern. Where patterns diverge, CSI will build out categories or patterns through expanding the description of the pattern so that it becomes more widely recognized across different contexts. This dynamic process of refinement will produce sets of categories that become “saturated” so that new sources of information leads to redundancy and clear regularities have emerged that feel robust in their description. This critical iterative process will help to identify those practices, complete with rich qualitative detail that will likely want to be retained, shared, and promoted across the network of grantees.

Utilization of Information

Once CSI has adequately analyzed the interview data, summaries will be produced and the results will be provided to the NHLBI Proteomic Centers Program management and the program Advisory Panel for feedback and input. The information derived through this data collection effort will be used to determine the extent to which the Proteomics Centers Program successfully met the program objectives and expectations. Such information will also enable NHLBI Proteomic Centers Program leadership and management to determine why centers achieved or failed to achieve program objectives and expectations. Information will also be used by NHLBI Proteomic Centers Program leadership and management to inform future developments in the proteomics research agenda, funding mechanisms, and models for advancing and expediting scientific discovery. In the absence of such data, the NHLBI Proteomic Centers Program leadership and management would have to rely on anecdotal evidence, obtained through non-systematic means, to understand the perceived results of center operations.

A3. Use of Information Technology and Burden Reduction

Because these are face-to-face interviews, either direct (in person) or indirect (videoconferencing or telephone) technology is of limited utility in reducing burden. Use of Skype as a platform for facilitating the interviews, rather than telephone, will likely increase the efficiency of the interview process and yield greater responsiveness. Every effort will be made to use videoconferencing (e.g. Skype) instead of a telephone platform to conduct the indirect interviews.

A.4. Effort to Identify Duplication and Use of Similar Information

In consultation with those managing the program at the Federal level and those coordinating activities among the centers, it has been determined that the information outlined in this data collection effort does not currently exist in a way that would serve the purpose of this information collection.

A.5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this study.

A.6. Consequences of Collecting the Information Less Frequently

This information request is a "single time" study. Therefore the information will only be collected once and there is no need for multiple responses.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

No special circumstances relating to the guidelines of 5 CFR 1320.5 are noted in this information request.

Response time will be less than one hour (50 minutes). The information collection outlined fully complies all guidelines of 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

An announcement was placed in the Federal Register on January 27^th, 2015, Document Number 2015-01421, page 4,291 public comment on this data collection system, thereby providing the research community with the opportunity to provide input into the proposed project. No comments were received.

A.9. Explanation of Any Payment of Gift to Respondents

No payment of gift will be offered or made to respondents for participation in this data collection.

A.10. Assurance of Privacy Provided to Respondents

Because this activity is an evaluation for the NHLBI's Proteomics Centers Program specifically, and not intended for dissemination or generalization to other agencies or populations, this is not considered human subjects research for the purpose of IRB review. Consistent with 28 CFR Part 46.102(D), this project is not research and does not require IRB review.

CSI will maintain Privacy (to the extent permitted by law in the event of a FOIA request) of all input from respondents. Data elicited from respondents will not be associated with any individual at any point in the project analysis or reporting. Participants will be made fully aware of these parameters and privacy assurance will occur during the facilitaed consent process. Included within the interview guide administered by CSI interviewers is a statement that will be read to respondents that outlines the assurance to privacy (see Attachments 1-4). The statement will read as follows: The information you provide will be kept private, and will not be disclosed to anyone but the researchers conducting the study, except otherwise required by law.

A.11. Justification for Sensitive Questions

No Personnally Identifyable Questions (PII) will be collected from respondents. Questions of a sensitive nature, such as sexual behavior and attitudes, religious belifs, and other matters commonly considered private will NOT be collected.

A.12. Estimates of Hour Burden Including Annualized Hourly Costs

Table 12-1 includes the the number of respondents, frequency of response, the average time per response, and the annual hour burden. The annual burden is based on the estimated average time of response for each interview, in this case approximatelry 50 minutes (.85 of an hour) mutiplied by the number of respondents across groups. Bescause CSI will be using a semi-structuted interview guide with slight adjustments and tailoring of questions, average time per response is not expected to vary. Respondents are expected to only need to respond to a single interview; multiple interviws are not expected or required. NIH staff are not included in the participant burden. Total burden hours are estimated to be 17.

Table 12-1 Estimates of Hour Burden

Form	Type of Respondents	Number of Respondents	Frequency of Response	Average Time per Response	Annual Hour Burden
Interview Guide – Principle Investigators & Key Personnel	Principal investigators and key personnel	27	1	50/60	23
Interview Guide – External Investigators	External Proteomics investigators	9	1	50/60	8
Interview Guide – Trainees and Junior Investigators	Trainees and junior investigators	20	1	50/60	17

Table 12-2 below displays the estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. Based upon estimates of hour burden from Table 12-1 above, respondent costs have been caluulated using prevailing hourly wage rates (estimated) for the type of respondents.

Table 12-2. Annualized Cost to Respondents

Data is from the Bureau of Labor Statistics (http://www.bls.gov/cps/cpsaat39.htm). Participants include Principal Investigators and other study personnel who are at many levels of expertese and function.

Type of Respondents	Number of Respondents	Frequency of Response	Average Time per Response	Hourly Wage Rate	Respondent Costs
Principal investigators and key personnel	27	1	50/60	$50.00	$1,148
External Proteomics investigators	9	1	50/60	$50.00	$383
Trainees and junior investigators	20	1	50/60	$25.00	$425
Totals	56				$1,956

A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers

As shown in Table 13 below, the cost estimate has been divided into two components: (a) a total capital and start-up cost component (annualized over its expected useful life); and (b) a total operation and maintenance and purchase of services component. The estimates takes into account costs associated with generating, maintaining, and disclosing or providing the information. For this information collection request, the annual cost burden is incurred in the operation/maintence and purchase of services component (column B). The estimated costs reflected in column B are based on the approved budget between NHLBI and its contractor, Concept Systems, Inc,.

Table 13 Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers

Operational/Maintence and Purchase	Amount
Transcription services	$6,800.00
Telecomunications fees	$750.00
Contractor Travel	$6,633.60
Venue Costs	$5,000.00
	$19,138.60

A.14. Annualized Cost to the Federal Government

The annual cost to the Federal Government for this data collection request will be $110,470, of which the breakdown of these costs is listed in Table 14 below. The contractor's professional fees for the data collection reflects a 7% discount on the standard professional rates for projects of similar scope and size.

Table 14. Toal Costs to the Federal Government

Personnel	Amount	GS Grade and Step or Salary	% Time	Cost
Scientific Program Specialist	1	GS 13-3	1%	$970
Program Official	1	GS 15-10	1%	$1,500
Contractor	5	$21,600	100%	$108,000

A.15. Explanation for Program Changes or Adjustments

This is a new collection of information.

A.16. Plans for Tabulation and Publication and Project Time Schedule

There are no plans for statistical analysis with this information collection and the information collected is not intended to be published. It is strictly a qualitative data collection by means of key informant interviews. Consequently, the data analysis plan will be consistent with generally accepted qualitative data reduction and coding procedures.

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable. We are not seeking an exemption to display the OMB Expiratin Date.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

Not applicable. We are not seeking an exception to Certification for Paperwork Reduction Act Submissions

1 Boston University Cardiovascular Proteomics Center; Johns Hopkins University Proteomic Innovation Center in Heart Failure; Harvard-Broad Proteomics Center; UT Health Center at San Antonio Cardiovascular Proteomics Center; Stanford University Proteomics Center; University of Texas Medical Branch NHLBI Proteomics Center at Galveston; UCLA Proteomics Center- Global Proteomic Initiative of Cardiovascular Medicine

2 NHLBI Proteomics Coordinating and Administrative Center - UCLA

3 Guba, E. G., & Lincoln, Y. S. (1990). Fourth-generation evaluation. Newbury Park: Sage.

4 Krippendorf, K. (2004). Content analysis: A introduction to its methodology (2nd ed.). Newbury Park, CA: Sage.

5 Guba, E. C. (1978). Toward a Methodology of Naturalistic Inquiry in Educational Evaluation. CSE Monograph Series in Evaluation no. 8. Los Angeles: University of California, Los Angeles, Center for the Study of Evaluation.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	jessica
File Modified	0000-00-00
File Created	2021-01-25