Download:
pdf |
pdfWhat Works Clearinghouse
Database Forms Revisions
Supporting Statement
CONTROL NO. 1850-0788
Submitted to:
Office of Management and Budget
Submitted by:
Institute for Education Sciences
U.S. Department of Education
July 2012
INTRODUCTION
This submission is a request to continue a currently approved collection under OMB Control
Number 1850-0788 for the What Works Clearinghouse (WWC) [ED-07-CO-0062]. The U.S.
Department of Education (ED) established the WWC to develop, maintain, and make accessible a
system of high quality reviews of studies of the effectiveness of education-related interventions.
In support of this effort, the WWC currently collects the following types of information:
Contact Us Forms
Registry of Evaluation Researchers
Registry of Randomized Controlled Trials
This supporting statement describes the justification for the WWC data collections; the sections
of the forms are detailed in the attachments. This document, which addresses Office of
Management and Budget (OMB) concerns regarding respondent burden and paperwork control,
has been prepared according to guidelines for completing the justification statement to
accompany OMB Form 83-I.
1
JUSTIFICATION
1. Necessity of Information Collection
There is an increasing recognition among policymakers that decisions about educational
programs, products, and practices should be based on evidence of effectiveness from high quality
scientific research. The Elementary and Secondary Education Act, as amended by the No Child
Left Behind Act (PL 107-110), for example, includes numerous requirements to use the findings
from “scientifically based research” in designing and implementing educational interventions.
Identifying evidence-based approaches to meet the educational needs of all children will prove
difficult unless education decision makers, including parents, are provided with the timely
information and support they need to make wise choices from among the ever-increasing options.
Stakeholders urgently want credible and reliable information that enables them to judge the
effectiveness and quality of alternative approaches to raising student achievement. Educators,
policymakers, and the public cannot be expected to do “what works” until they actually know
“what works.” By transforming education into an evidence-based field, ED is working to fulfill
its mission of improving student outcomes and providing the information that decision makers
need.
The WWC is a key part of this effort. Established by the ED’s Institute of Education Sciences
(IES) in 2002, the purpose of the WWC is to offer everyone—from public officials to the public
at large—a central, independent, and trusted source of scientific evidence of what works in
education. To be successful, the WWC involves key constituents in the process of its activities,
the development of its products, and identification of areas for improvement.
In support of this effort, the web-based submission forms currently posted on the What Works
Clearinghouse website (whatworks.ed.gov) (1) allow the public to nominate evidence-based
research studies, interventions, and topics for review by the WWC, (2) collect information about
researchers and study authors who want to supply the public with information about their
evaluation services, and (3) collect information on randomized controlled trials of education
interventions. Accordingly, the WWC will continue to administer the following:
Contact Us Form for study, intervention, and topic nominations
Registry of Evaluation Researchers Form for registration
Registry of Evaluation Researchers Letter of Commitment
Registry of Randomized Controlled Trials Form for registration
Registry of Randomized Controlled Trials Letter of Commitment
Clearance is requested to continue these information collection efforts that support the
development, operation, and evolution of the WWC and provide the public with information they
request.
2
2. Purposes and Use of the WWC Data Collection Forms
Below we explain the purpose of each of the existing voluntary information collection forms.
Contact Us Form. The Contact Us Form allows users to submit correspondence to the WWC and
nominate interventions, topics, and studies to review. Contact Us forms may also be used as an
informal survey for users to submit nominations to inform the WWC of interventions users want
considered for inclusion in WWC Intervention Reports and to prioritize topics and practice guides
proposed and recommended to IES in the WWC Annual Plan. Users may also submit study
nominations to inform the WWC of studies relevant to WWC Intervention Reports. The WWC
collects study submissions to supplement the studies located by staff, and these studies are
considered for inclusion in WWC Intervention Reports. Users nominating studies for review also
are given the option to upload the studies.
The Department does not request changes to the elements of the forms (attachment 01 [text
documentation], attachment 02 [study submission text documentation], and 03 [details]). These
elements are the following:
User email address
Subject
Category
Message
WWC Submission Process. The WWC Submission Process Flowchart shown in Exhibit 1
demonstrates the path that website users will be prompted to take during the online submission
process.
Responses to the Contact Us forms will be captured in a database. Case ID numbers will be used
in association with respondents’ information. Individual identifying information is accessible
only to project staff and will be used only if there are questions about a respondent’s submission.
Information will be stored in a secure database on the WWC private website. At the end of the
contract, the WWC will provide ED with the database created during the project.
3
EXHIBIT 1
WWC Submission Process
Start
(WWC Contact Us Form)
Select a Subject:
Publications, Topics, Evidence Standards,
Submit a Study to Review, Suggest an
Intervention to Review, Suggest a Topic,
Check on the Status of a Submission, or
Other
Publications or
Topics
Evidence Standards,
Suggest an Intervention to
Review, Suggest a Topic,
Check on the Status of a
Submission, or Other
Select a Category
(optional)
Submit a Study to
Review
User Email
Address
User Email
Address
Message Content
(length variable)
Message Content
(length variable)
Optional
Electronic Upload
Done
Done
4
Registry of Evaluation Researchers Form and Registry of Evaluation Researchers Letter of
Commitment. The online Registry of Evaluation Researchers provides potential sponsors of
effectiveness studies with information about individual and organization evaluators who
subscribe to the standards of evidence established by the WWC. This registry contributes to
advancing the quality of educational research by (1) assembling a substantial list of committed
outcome evaluation providers and (2) supplying tools and guides that help consumers decide what
they need, how to assess the qualifications of the evaluators, how to use the information supplied
to find potential evaluation providers, and how to supplement the help provided by the registry
with other sources of assistance. To be included, evaluators must complete a registration form and
sign a commitment letter on the password-protected website (attachment Forms 04 [text
documentation] and 05 [letter]). The Department does not request changes to either of these
forms.
Registry of Randomized Controlled Trials Form and Registry of Randomized Controlled Trials
Letter of Commitment. The online Registry of Randomized Controlled Trials provides users with
information about randomized controlled trials (RCTs) of education interventions, including
those completed and with a final report available, those funded and in progress, and those not yet
reviewed by the WWC under one of its topic review areas. This registry contributes to advancing
the quality of educational research by (1) assembling a substantial list of RCTs in education on
particular topics or interventions that are funded and in progress or completed, and with a final
report available; and (2) supplying tools and guides that help consumers understand how to use
the information supplied in the Registry of Randomized Controlled Trials. To be included, the
researchers or entities conducting an RCT must complete the registration form and sign a
commitment letter on the password-protected website (attachment Form 06 [text documentation]
and 07 [letter]). The Department does not request changes to either of these forms.
The Department does not assess the quality and accuracy of the information supplied by
researchers and entities wishing to list studies in the Registry of Randomized Controlled Trials,
only its completeness. This is clearly stated in the searchable online registry. Participation is
voluntary. Because the registry relies on self-reported data, sufficient information is needed so
that members of the user community can determine for themselves whether the RCTs―either
funded and in progress or completed, and with an available final report―meet their needs.
3. Use of Technology in Information Collection
All WWC databases, customer surveys, and registry forms are web based. Website users may
choose to print out the forms from the website and submit a paper copy, but we expect most users
to submit the forms electronically via the automated website interfaces. These interfaces have
been designed to impose as little collection burden as possible, while still gathering the
information needed to inform the WWC. The Contact Us nomination forms are only 1 to 3 pages,
and each page requires no more than three items from the user.
To reduce the burden on those using the Registry of Evaluation Researchers and the Registry of
Randomized Controlled Trials, returning users may update their forms without having to resubmit
their information. Assistance also is provided in using the web-based forms. For example, the
registry forms link users to definitions of key terms and documents, where appropriate. In
addition, users have easy access to personal and individual guidance if needed, and information
on how to contact the WWC is presented prominently on the website.
5
4. Efforts to Identify Duplication
The information to be collected from the Contact Us forms, the Registry of Evaluation
Researchers database, and the Registry of Randomized Controlled Trials is not available from any
other source and does not duplicate any other existing information collection effort.
5. Burden on Small Business Organizations
Not applicable to this information collection.
6. Consequences of Less Frequent Information Collection
The Contact Us forms are voluntary and must be sought out by WWC website users; therefore,
the Department assumes that users will complete these forms only when they wish to contact the
WWC; submit answers for an informal survey; or nominate an intervention, study, or topic for
review.
Submitting information through the Registry of Evaluation Researchers and the Registry of
Randomized Controlled Trials is also voluntary, but users are prompted to update their
information on an annual basis in order to keep the contents of the database up to date and useful
to the public.
Information about the availability of Contact Us forms will be disseminated periodically at
presentations to encourage users and potential users to visit the website and submit their requests
and feedback. If we did not disseminate information about these opportunities periodically,
existing users might forget that this feature of the website is available to them, and potential users
might not learn how to submit their nominations and information needs.
7. Special Circumstances
Users may submit copyrighted publications. These publications will not be shared with the public
until the appropriate permissions are obtained. Users may also submit unpublished studies and
request that they be kept confidential. These studies also will not be shared with the public until
permission is obtained.
There are no other special circumstances as described in the OMB guidelines.
8. Outside Consultants
Since the contract award date, WWC staff members have given presentations to a wide range of
administrators, researchers, policymakers, practitioners, and others to inform them about the
WWC and allow them to inform us of their information needs. These presentations take place
6
during conferences where people can visit a booth to speak with presenters and informally make
suggestions. Information provided through these interactions and presentations has guided the
redesign of the nomination forms.
No outside consultants have contributed to the design of the registries. No outside consultants
will contribute to the maintenance of the registries.
The 60-day Federal Register Notice published Vol. 77, page 31592 on May 29, 2012. No public
comments have been received.
9. Explanation of Providing Payment or Gifts to Respondents
No payment or gift of any kind will be provided to respondents.
10. Assurance of Confidentiality
For the Contact Us forms, case ID numbers will be used in association with the respondent’s
information. The website has a privacy policy statement indicating that no individual
identification will be posted on the public website without the user’s permission. Information
about users will be reported at the aggregate level only. Contact information on Contact Us forms
is used for the express purpose of clarifying the WWC user’s submission and will not be released
beyond the WWC staff with an immediate need to clarify the submitted information.
The purpose of the Registry of Evaluation Researchers database is to allow evaluators to post
their credentials and contact information to the public; therefore, it is assumed that when
evaluators submit their forms, they are not requesting confidentiality. The WWC does, however,
offer evaluators the option of limited disclosure for studies. For example, respondents can request
that proprietary information be kept confidential. This typically will be requested only when an
evaluator is in the process of getting a study published and would like the WWC to post the study
information after it is publicly available. Additionally, the WWC will post only the provided
contact and experience information; biographies, resumes, and curricula vitae will not be
available through the WWC.
The purpose of the Registry of Randomized Controlled Trials is to enable researchers and other
stakeholders to post information to the public regarding RCTs that are funded and in progress or
completed, and with an available final report; therefore, it is assumed that when stakeholders
submit their forms, they are not requesting confidentiality. The WWC will post only the provided
bibliographic information and study abstract; the full text of the study will not be available
through the WWC.
11. Sensitive Questions
None of the requested information is sensitive in nature.
7
12. Estimates of Hour Burden
In January 2007, the initial collection was submitted for approval. Since the initial collection was
approved, experience in operating the WWC has led to changes in the necessary information
requested from users, which resulted in anticipated changes in hourly burden estimates. The total
number of respondents for all WWC forms is estimated to be 1,740 across three years, with the
total annualized responses at 580. The total number of burden hours for all collections is
estimated to be 488, with the total annualized hours at 163. The annualized burden of 163 hours
represents a decrease of 17 hours from the previous submission, despite a slight increase in
respondents and responses. Hourly burden estimates by form and totals by year are displayed in
Exhibit 2.
The estimated cost to respondents for all WWC forms is $9,403.89 across three years, with the
total annualized cost at $3,134.63. Cost burden estimates by form and totals by year are displayed
in Exhibit 3. Estimates are based on the average hourly wage ($19.29) as of June 2007, as
reported by the Bureau of Labor Statistics National Bulletin.1
1
Available at http://www.bls.gov/ncs/ocs/sp/ncbl0910.pdf. Accessed April 9, 2012.
8
EXHIBIT 2
Estimates of Hourly Burden
Year
Time to
Complete
Number of
Respondents
Hourly
Burden
Contact Us Forms
Y1
3 minutes
350
17.5
Registry of Evaluation Researchers
Y1
120 minutes
30
60
Registry of Evaluation Researchers
Update*
Y1
30 minutes
60
30
Registry of Randomized
Controlled Trials
Y1
30 minutes
80
40
Registry of Randomized
Controlled Trials Update*
Y1
15 minutes
60
15
Total, Year 1
580
162.5
Form
Contact Us Forms
Y2
3 minutes
350
17.5
Registry of Evaluation Researchers
Y2
120 minutes
30
60
Registry of Evaluation Researchers
Update*
Y2
30 minutes
60
30
Registry of Randomized
Controlled Trials
Y2
30 minutes
80
40
Registry of Randomized
Controlled Trials Update*
Y2
15 minutes
60
15
Total, Year 2
580
162.5
Contact Us Forms
Y3
3 minutes
350
17.5
Registry of Evaluation Researchers
Y3
120 minutes
30
60
Registry of Evaluation Researchers
Update*
Y3
30 minutes
60
30
Registry of Randomized
Controlled Trials
Y3
30 minutes
80
40
Registry of Randomized
Controlled Trials Update*
Y3
15 minutes
60
15
580
162.5
1,740
487.5
Total, Year 3
Total, Years 1–3
* The Registry of Evaluation Researchers and the Registry of Randomized Controlled Trials updates are
not different forms. Rather, users are provided an opportunity to edit the information that already exists and
is prepopulated in the appropriate completed form.
9
EXHIBIT 3
Estimates of Annualized Cost
Year
Number of
Respondents
Hourly Burden
Cost
Contact Us Forms
Y1
350
17.5
$337.58
Registry of Evaluation Researchers
Y1
30
60
$1,157.40
Registry of Evaluation Researchers
Update*
Y1
60
30
$578.70
Registry of Randomized
Controlled Trials
Y1
80
40
$771.60
Registry of Randomized
Controlled Trials Update*
Y1
60
15
$289.35
162.5
$3,134.63
Form
Total, Year 1
Contact Us Forms
Y2
350
17.5
$337.58
Registry of Evaluation Researchers
Y2
30
60
$1,157.40
Registry of Evaluation Researchers
Update*
Y2
60
30
$578.70
Registry of Randomized
Controlled Trials
Y2
80
40
$771.60
Registry of Randomized
Controlled Trials Update*
Y2
60
15
$289.35
162.5
$3,134.63
Total, Year 2
Contact Us Forms
Y3
350
17.5
$337.58
Registry of Evaluation Researchers
Y3
30
60
$1,157.40
Y3
60
30
$578.70
Y3
80
40
$771.60
Y3
60
15
$289.35
Total, Year 3
162.5
$3,134.63
Total, Years 1–3
487.5
$9,403.89
Registry of Evaluation Researchers
Update*
Registry of Randomized
Controlled Trials
Registry of Randomized
Controlled Trials Update*
* The Registry of Evaluation Researchers and the Registry of Randomized Controlled Trials updates are
not different forms. Rather, users are provided an opportunity to edit the information that already exists and
is prepopulated in the appropriate completed form.
10
13. Estimate of Annual Cost Burden to Respondents
There are no start-up costs to respondents related to this collection or other costs not accounted
for earlier.
14. Estimate of Annual Cost to the Federal Government
The total cost to the federal government for the Contact Us forms, Registry of Evaluation
Researchers, and Registry of Randomized Controlled Trials is estimated at $35,424.00 across the
three years, with the total annualized cost at $11,808.00. This represent a $193,827.57 decrease
from the original total cost estimate in January 2007. Exhibit 4 displays the costs by year for
maintaining and administering the online forms and databases, promoting and requesting input
from the public, and analyzing and tabulating results.
EXHIBIT 4
Estimated Cost to Federal Government, by Year
Year
Year 1
Total Number
of Respondents
580
Maintenance and
Administration
$11,808.00
Development
$0.00
Total
$11,808.00
Year 2
580
$11,808.00
$0.00
$11,808.00
Year 3
580
$11,808.00
$0.00
$11,808.00
Total
1,740
$35,424.00
$0.00
$35,424.00
15. Program Changes or Adjustment
Based on the numbers of public submissions we have received in the past two years, we have
adjusted the figures to estimate the number of future responses more accurately. We anticipate a
decrease of 17 burden hours. This decrease is considered an adjustment because the decrease is
due to a projected decrease in the Registry of Evaluation Researchers and Registry of
Randomized Controlled Trials updates. The decrease in burden hours is despite the fact that the
total number of respondents and responses have increased slightly on an annual basis.
There are no requested program changes.
16. Plans for Tabulation and Publication of Results
The majority of tabulations of the information collected under this request will be for internal use
by WWC contractors and ED staff. The information will be used to ascertain the needs of the
public and take their suggestions, as well as to monitor how well the WWC is meeting their
needs.
11
17. Approval to Not Display OMB Expiration
This approval is not requested.
18. Exceptions to Certification for Paperwork Reduction Act
Submissions
There are no exceptions.
12
ATTACHMENTS
Forms for Existing and Approved Collection
01
Contact Us Form
02
Contact Us Study Submission Form
03
Contact Us Details
04
Registry of Evaluation Researchers Form
05
Registry of Evaluation Researchers Commitment Letter
06
Registry of Randomized Controlled Trials Form
07
Registry of Randomized Controlled Trials Commitment Letter
13
| File Type | application/pdf |
| File Title | What Works ClearinghouseDatabase Forms Revisions |
| Author | Teresa Duncan |
| File Modified | 2012-07-20 |
| File Created | 2012-07-20 |