Supporting Statement A for
Database of Cognitive Training and Remediation Studies (NIMH)
Date
10/08/2014
Sarah E. Morris, Ph.D.
Program Officer
Schizophrenia Spectrum Disorders Research Program
Division of Adult Translational Research
National Institute of Mental Health
6001 Executive Blvd, Room 7122, MSC 9625
Bethesda, MD 20892
Phone: 301-443-9233
Fax: 301-443-4611
Table of contents
A. JUSTIFICATION
A.1 Circumstances Making the Collection of Information Necessary
A.2. Purpose and Use of the Information COLLECTION
A.3 Use of Information Technology and Burden Reduction
A.4 Efforts to Identify Duplication and Use of Similar Information
A.5 Impact on Small Businesses or Other Small Entities
A.6 Consequences of Collecting the Information Less Frequently
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency
A.9 Explanation of Any Payment of Gift to Respondents
A.10 Assurance of Confidentiality Provided to Respondents
A.11 Justification for Sensitive Questions
A.12 Estimates of Hour Burden Including Annualized Hourly Costs
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record
keepers
A.14 Annualized Cost to the Federal Government
A.15 Explanation for Program Changes or Adjustments
A.16 Plans for Tabulation and Publication and Project Time Schedule
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions
List of Attachments
Attachment 1: Cognitive Remediation Experts Workshop (CREW) Member
Attachment 2: Experts in CR for schizophrenia
Attachment 3: Data Submission Agreement
A.1 Circumstances Making the Collection of Information Necessary
In the last decade, the National Institute of Mental Health (NIMH) has funded more than fifteen studies of cognitive remediation (CR) in schizophrenia. There has been a great deal of variability among these projects with regard to the specific type of intervention used, the age and degree of cognitive impairment of the patients, the length of the treatment and follow-up period and the types of measures used to assess proximal and distal outcomes. The results of these studies have been mixed, with some showing promise for improving cognition and functioning and others showing no improvement beyond the tasks used in the training. Due to the importance of cognition as a treatment target in schizophrenia and the appeal of non-pharmacological, computerized interventions with the potential for widespread implementation, this remains a very active area of research.
As the largest funder of research on CR in schizophrenia, in order to maximize the informational return on its research investment and increase the public benefit derived from existing studies, NIMH is interested in compiling and curating an integrated database that includes study- and subject-level data from studies of cognitive remediation in schizophrenia (regardless of funding source). The Database of Cognitive Training and Remediation Studies (DCTRS) would provide a repository for investigators who want to share their data. The integrated database would allow NIMH staff and approved investigators to test novel hypotheses about patient characteristics and treatment features that are associated with optimal treatment outcomes using existing rather than investing in new data collection efforts.
Published cognitive remediation studies typically include a modest number of subjects (approximately 60 per study). Combining data into an integrated database would allow greater statistical power to detect treatment effects, moderating variables, and individual differences that might be important for targeting treatment or for the development of new treatments. It would also allow direct comparisons of different types of cognitive remediation approaches. Taken together, these types of analyses on an integrated database will provide rapid answers which will advance science in this area and will help NIMH to be more strategic in its funding of future studies of cognitive remediation in schizophrenia.
The information to be collected will include details about the study and intervention, and subject-level data from assessments (e.g., of cognition, symptoms, everyday functioning) conducted during the study (see section A.2. for detail).
Because some of the interventions are published by their developers for commercial purposes and some of the investigators are pursuing FDA approval for their interventions, there is reluctance among this scientific community to share data directly with each other. NIMH, however, is in a position to serve as a neutral repository. It is unlikely that any individual investigator could compile this data and to develop this valuable scientific resource to the community.
Information collected from the individuals submitting data to the database, as part of the submission agreement and submission certification, may be made public in part or in whole for tracking and reporting purposes. This Privacy Act Notification is provided pursuant to Public Law 93-579, Privacy Act of 1974, 5 U.S.C. Section 552a. Authority for the collection of the information requested below from the submitter comes from the authorities regarding the establishment of the National Institutes of Health, its general authority to conduct and fund research and to provide training assistance, and its general authority to maintain records in connection with these and its other functions (42 U.S.C. 203, 241, 289l-1 and 44 U.S.C. 3101), and Section 301 and 493 of the Public Health Service Act. These records will be maintained in accordance with the Privacy Act System of Record Notice 09-25-0200 (http://oma.od.nih.gov/ms/privacy/pa-files/0200.htm) covering “Clinical, Basic and Population-based Research Studies of the National Institutes of Health (NIH), HHS/NIH/OD.” The NIH System of Record Notice was previously published in the Federal register on September 26, 2002, Vol 67, No 187. (http://www.gpo.gov/fdsys/pkg/FR-2002-09-26/pdf/02-23965.pdf).
The information in the database would include both subject-level and study-level data. Subject-level data would consist of the following: ratings of symptom severity and everyday functioning, scores on cognitive measures, medication type and dose, information about treatment and illness history (e.g., age at first hospitalization for psychiatric illness, number of psychiatric hospitalizations), number of treatment sessions completed, and demographic information about participants (age, gender, race/ethnicity, marital status) and participant’s parents (education level). Study-level information would include the principal investigator’s name, email address, phone number, institution and Federal-wide Assurance number; the start and end dates of the study, Certificate of Confidentiality status, type and duration of treatment, the treatment targets, information about the individuals who administered the intervention and assessments (i.e., employed by the study or by the clinical setting, degree of training), procedures for randomizing assignment and blinding assessments, subject payment information, eligibility criteria, types of assessment administered and the schedule of administration and information about publications reporting the results of the study.
Inevitably, there will be differences and overlap in the specific measures used by the study investigators to collect data for each study. Not all studies have collected (or will collect) the same data or used the same assessment tools. NIMH staff on the DCTRS workgroup will curate the data in order to maximize the amount of overlap in data among studies (e.g., combining across versions of tests when possible or converting individual item scores to standardized factor scores) so that sample sizes for the purposes of combined analyses are as large as possible.
The information to be collected will be used by NIMH program staff and by approved extramural investigators for the purposes of examining the efficacy of CR on various outcome measures (such as cognitive changes or improvement in everyday functioning), potential moderators of treatment effects (such as patient age), and other scientific inquiries regarding CR in schizophrenia. This information will allow program staff to make strategic decisions about NIMH funding for new studies of CR in schizophrenia and will allow the scientific community to make maximal use of existing data to reduce the need to collect new data that can be answered using existing data, conduct preliminary tests of novel hypotheses prior to starting a new study, and inform new studies to address gaps in the existing body of knowledge. The information regarding the study PI (e.g., name, email address, phone number, and institution) will allow DCTRS staff to communicate with the PI during the data submission process. The negative consequences of not collecting this data are likely to be slowing of progress in this scientific area and the possibility that meta-analyses (in contrast to “mega-analyses” using subject-level data) will distort the results of research in this area because they cannot examine treatment effects and moderators using subject-level data.
The collected information will be reviewed by the DCTRS workgroup (consisting of staff from the NIMH Division of Adult Translational Research, Division of Services and Intervention Research, and NIMH Clinical Research Datasets group). The data will be reviewed for errors and inconsistencies (e.g., in computing total scores, extreme outliers), screened to ensure that no potentially identifying information is included, and integrated across studies, using standardized data dictionaries developed by the DCTRS workgroup. The database will be stored on a secure NIMH server. Investigators who wish to access the data will submit a signed NIMH Data Access Request and Use Certification (DUC) form (OMB# 0925-0667; Expiration Date: 01/31/2016) and, if the request is approved, the investigator is notified via email.
Information about the investigator and the study is collected using the NIMH DCTRS Data Submission Agreement (DSA) (Attachment 3). The DSA consists of a set of terms and conditions to be signed by the PI of the study and their institutional official and a Study Information Form. The terms and conditions must include the Federal-Wide Assurance number of the investigator’s affiliated institution and be co-signed by an NIH-recognized Business/Institutional Official. The signed terms and conditions can be scanned and emailed to DCTRS. The Study Information Form is an electronic form (in Word format) that can be completed electronically and returned to DCTRS as an email attachment. The responses on the form will then be saved as formatted text and added to the database.
The study information form lists the following items that contributing PIs are requested to submit:
De-identified data (SPSS, SAS, Excel, TXT or RTF format), including variable names, definitions and value labels (this could be incorporated into the data file or in the form of a data dictionary or a codebook).
A complete set of the forms that were used to collect data
If the results of the study have been published, a copy of the manuscript
The study protocol, including an assessment schedule The eligibility criteria that were used for this study
These items are typically available as digital files and can be sent to DCTRS as email attachments (with encryption and password-protection for the data files). The study forms are sometimes only in paper format and require scanning before sending.
Privacy Impact Assessment (PIA) was completed for the database being used to collect the information and approved on 12/07/2013.
Although there have been a few meta-analyses of data from studies of CR in schizophrenia published in recent years, these analyses do not included subject-level data.
Based on ongoing monitoring of the scientific publications in this area, there is no database that combines subject-level data across CR studies in schizophrenia. In April 2013, NIMH staff who are leading the DCTRS project met with the members of the Cognitive Remediation Experts Workshop (CREW). The CREW is an international group of researchers who develop and test new CR interventions for schizophrenia (Attachment 1). NIMH staff described the plans for compiling and sharing the database to the committee and there was consensus that (1) no other similar database existed and (2) that it would be useful to the scientific community.
We do not anticipate receiving data from small businesses/entities. Although there may be publishers of cognitive remediation programs that could be qualified as small businesses, they typically have academic partners who would be responsible for contributing data if they choose to do so.
Data and supporting materials are collected only once for each study (after a study is completed). There is no ongoing data collection or re-contact, except when necessary shortly after the submission in order for the DCTRS team to inquire about any issues that are identified during the data checking process.
If the data are not collected, progress in this scientific area will be slowed and it is possible that meta-analyses (in contrast to “mega-analyses” using subject-level data) will distort the results of research in this area because they cannot examine treatment effects and moderators using subject-level data. NIMH will not be able to be as strategic in awarding new grants to improve CR for schizophrenia without the information from the integrated database.
Not applicable.
A Federal Register Notice was published on April 15, 2014, Vol. 79 FR 08533 (https://www.federalregister.gov/articles/2014/04/15/2014-08533/proposed-collection-60-day-comment-request-nimh-database-of-cognitive-training-and-remediation). No public comments were received.
In addition to the meeting with the CREW committee described in Section A.4 at which the plans for creating the database were reviewed, the DCTRS workgroup asked five experts in CR for schizophrenia (Attachment 2) to complete the Study Information Form and provide the requested materials. This effort allowed DCTRS staff to evaluate the feasibility of integrating data across studies, to determine whether there would be sufficient overlap of data among studies, and to begin testing procedures for quality assurance. In addition, two of the experts were asked to review the Data Submission Agreement. Feedback from these individuals was positive with regard to the format and ease of use of the Study Information form and the potential utility of the database to the scientific community.
The DCTRS workgroup, comprised of NIMH staff, will be in frequent communication with members of this scientific community. We anticipate that they will not only be the contributors to the database but also the primary users of the database. This communication will be in the form of meetings with the CREW committee at future scientific conferences (the committee has regular meetings every two years) and in response to inquiries from individual scientists.
No payment of gift will be provided to respondents.
In the terms and conditions to be signed by submitting investigators prior to providing any study-related data, submitters are informed of the following: “Information collected from the Submitter, as part of the submission agreement and submission certification, may be made public in part or in whole for tracking and reporting purposes. This Privacy Act Notification is provided pursuant to Public Law 93-579, Privacy Act of 1974, 5 U.S.C. Section 552a. Authority for the collection of the information requested below from the submitter comes from the authorities regarding the establishment of the National Institutes of Health, its general authority to conduct and fund research and to provide training assistance, and its general authority to maintain records in connection with these and its other functions (42 U.S.C. 203, 241, 289l-1 and 44 U.S.C. 3101), and Section 301 and 493 of the Public Health Service Act. These records will be maintained in accordance with the Privacy Act System of Record Notice 09-25-0200 (http://oma.od.nih.gov/ms/privacy/pa-files/0200.htm) covering ‘Clinical, Basic and Population-based Research Studies of the National Institutes of Health (NIH), HHS/NIH/OD.’ The NIH System of Record Notice was previously published in the Federal register on September 26, 2002, Vol 67, No 187. (http://www.gpo.gov/fdsys/pkg/FR-2002-09-26/pdf/02-23965.pdf). The primary uses of this information are to document, track, and monitor and evaluate the submission of data from clinical, basic, and population-based research activities and to notify submitters in the event a potential error in the dataset is identified or in the event of updates or other changes to the database.”
As noted in the Terms and conditions of the Data Submission Agreement, the name of the Principal Investigator will be included in the DCTRS database and included in data released to individuals who have requested access to the database. The PI name will help the users of the database to easily identify the studies that are included in the database. No other information collected about the individuals contributing data to the DCTRS database will be released.
DCTRS includes data regarding participants’ psychiatric diagnosis, medication type and dose and illness/treatment history (age of onset of psychiatric symptoms, age at first treatment for psychiatric symptoms, age at first psychiatric hospitalization, total number of psychiatric hospitalizations, number of psychiatric hospitalizations in the last year), to the extent that it is included in data from submitting studies. This information is important in order to characterize the subjects included in the database and to examine whether these clinical characteristics are associated with differences in outcomes of CR intervention. For example, it would be important to know whether more severely ill individuals or those with longer duration of illness have a reduced therapeutic response to CR or whether specific medications have a potentiating or dampening effect on response to CR.
In the Terms and Conditions of the Data Submission Agreement, contributing PIs are asked to attest that “data were collected pursuant to an informed consent that is not inconsistent with the data submission.” For studies that have already been completed, DCTRS will not require that researchers re-contact participants. For future studies of CR for schizophrenia funded by NIMH, program staff will encourage investigators to include language regarding sharing data via DCTRS in their consent forms.
No other data of a sensitive nature are requested from contributors or included in DCTRS.
In addition, in the Terms and Conditions of the Data Submission Agreement, contributing PIs are asked to attest that “the data have been ‘de-identified’ according to the following criterion: the identities of subjects cannot be readily ascertained or otherwise associated with the data by the repository staff or secondary data users (45 C.F.R. 46.102(f)). Submitter further agrees not to disclose the identities of research participants to DCTRS in the future and to verify that data lack identifiers after submission. Submitter agrees to notify NIMH as soon as possible if, upon review of DCTRS data, the Submitter discovers identifying information in that data.”
Upon submission of the data, the repository staff performs a quality control review to ensure that no personally identifiable information (PII) is contained in the dataset or supporting documentation. Only data that have undergone this quality control review are approved for sharing with the research community.
The DCTRS dataset will be stored on NIMH’s Clinical Trials Operations and Biostatistics Unit (CTOB) ‘Q’ drive, which only CTOB staff have access to. An excel file containing the names, email addresses and institutions of the contributing investigators will be stored on an NIMH SharePoint site that only members of the DCTRS team can access.
The values in the tables below assume that all effort in completion of forms and preparation of materials is completed by the Principal Investigator, but we anticipate that Principal Investigators may also have a Research Assistant assist with the process, so Respondent Costs might be somewhat over-estimated depending on the extent to which some of the effort will be by Research Assistants.
Individuals who completed the Data Submission Agreement which includes the Submitter Information and Certification, Study Information form and supporting materials (as described in Section A.8) reported that it took them, on average, a total of 5 hours, and that most of that time was spent getting the digital data file(s) and documentation “cleaned up” (e.g., making sure that variable names and labels were clear) and ready to send.
According to a PubMed search, 40 trials of CR in schizophrenia have been published in the last 5 years and there continues to be intense scientific interest in the topic. Our annualized estimates of burden assume a 5-year time frame in which approximately 70% of the studies published in the last 5 years and approximately 30 future trials completed in the next 5 years will be contributed to the database, for a total of 60 studies to be included. Some investigators will have more than one study, but for the purposes of estimating burden, we treat each study as a separate data submission. The annual hour burden will be greater during the first year of the database as we compile a backlog of already-published studies and a reduction in burden in subsequent years as only new studies are added to the database. The values in table A12-1 are averages over a 5-year duration of the project. It is difficult to estimate the burden beyond this timeframe because the number of studies is likely to change depending on the results of future research.
A.12 - 1 Estimates of Hour Burden
|
||||
Form |
Number of Respondents |
Frequency of Response |
Average Time per Response (in hours) |
Annual Hour Burden |
Data Submission Agreement |
12 |
1 |
5 |
60 |
Totals |
12 |
|
|
60
|
A.12 - 2 Annualized Cost To Respondents
Type of Respondents |
Number of Respondents |
Frequency of Response |
Average Time per Respondent |
Hourly Wage Rate |
Respondent Cost |
Researchers |
12 |
1 |
5.00 |
$69.87 |
$349.35 |
Totals |
12 |
1 |
5.00 |
$69.87 |
$349.35 |
Bureau of Labor Statistics/Occupational Employment “Life, Physical, and Social Scientist” http://www.bls.gov/oes/current/oes_13644.htm#19-0000, occupational code 19-2099.
There are no additional costs other than the respondents’ burden given in A.12.
Staff |
Grade/Salary (Percent FTE or Effort) |
Repository Operations Time (in hours) |
NIMH Repository Hourly Rate |
Repository Operation Cost |
Repository Operations Staff Tier 1 |
$46,036 (< 2% Effort) |
36 |
22.06 |
794.02 |
Repository Operations Staff Tier 2 |
$149,994 (< 2% Effort) |
36 |
79.06 |
2846.02 |
DCTRS Workgroup (monthly meetings) |
|
|
|
|
Division Director |
GS-15 Step 4 (137,494) (< 1% FTE) |
12 |
$ 65.88 |
790.56 |
Branch Chief |
GS-14 Step 4 (116,887) (< 1% FTE) |
12 |
$ 56.01 |
672.00 |
Program Officer |
GS-14 Step 4 (109,804) (< 1% FTE) |
12 |
$ 56.00 |
672.00 |
Statistician |
GS -12 Step 3 (80,662) (< 1% FTE) |
12 |
$38.65 |
463.80 |
Contractor |
$149,994 (< 1% FTE) |
12 |
$79.06 |
948.67 |
Total Annual Costs |
|
|
|
$ 7187.06 |
Salary/Wage Source: Office of Personnel Management 2014 General Schedule Locality Salary Table for various GS-levels;
*Tier 1 = $22.06/hr; Tier 2 = $79.06/hr
The total costs of the DCTRS to the Federal government over three years will be $21,561.19. These costs are based on the estimated effort needed to processes approximately 12 new Data Submission Agreements per year and the effort of the 4-person workgroup meeting once per month. Approximately 3 hours of Tier 1 staff are needed to manage the submission of documents and data, update the list of submitted projects, manipulate the data to fit the database structure and perform quality assurance checks. Approximately 3 hours of Tier 2 staff time is needed to review the data from each measure for unusual values, to follow up with the submitting investigator with questions, check that the measures used in the study correspond to those in the database, and other tasks. These estimates are derived from the experience of the DCTRS team in compiling data from 5 sample studies as described in Section A.8. There are no additional costs for supplies or equipment. Although we anticipate that this database will continue to grow and serve as a resource for more than 5 years, it is difficult to project the rate of new data submissions more than 5 years from now. The total cost of the DCTRS data submission over a 5 year period is anticipated to be approximately $36,000.
This is a new data collection.
NIMH anticipates that the bulk of the analyses and publications of the data in DCTRS will be done by investigators who request access to the database to test hypotheses regarding cognitive remediation in schizophrenia. It is likely, however, that in order to publicize the availability of this scientific resource to the community, NIMH staff will write and submit for publication a paper that describes the size and scope of the database and includes some statistical analyses (e.g., testing preliminary hypotheses about the efficacy of CR for younger compared to older participants or the relationships between duration of treatment and cognitive outcomes).
A.16 - 1 Project Time Schedule
|
|
Activity |
Time Schedule |
Begin solicitation of data submission |
1 - 2 months after OMB approval |
Provide submission instructions and documentation in response to inquiries |
Ongoing basis starting 1-2 months after OMB approval |
Initial analyses of combined data |
8 months after OMB approval |
Publication of initial descriptive data about dataset and initial inquiries (see text) |
12 months after OMB approval
|
Not applicable.
Not applicable.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement 'A' Preparation - 04/05/2011 |
| Subject | Supporting Statement 'A' Preparation - 04/05/2011 |
| Author | OD/USER |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |