WWC_SupportingStatement_final

What Works Clearinghouse Database Forms and Customer Surveys Supporting Statement Control No. 1850-0788 Submitted to: Office of Management and Budget Submitted by: Institute for Education Sciences U.S. Department of Education January, 2007

Introduction

The U.S. Department of Education established the What Works Clearinghouse (WWC) [ED-02-CO-0022] to develop, maintain, and make accessible a system of high-quality reviews of studies of the effectiveness of education-related interventions and of the quality of education-related test instruments. In support of this effort, the WWC collects the following types of information:

• Nominations for WWC Review

• Interventions

• Studies

• Topic Areas

• Information on Outcome Evaluators

In addition, the Clearinghouse is charged with assessing the ongoing information needs of WWC website users as they utilize its databases and the WWC website to locate scientifically proven educational programs, practices, products and policies and to submit requests and information to the WWC. To this end, we have created and administered a query of WWC website services and users in an effort to compile and respond to user feedback about the databases and the website:

WWC Website User Type Query

The Clearinghouse is also responsible for developing and implementing a communication and collaboration strategy. As a means of communicating on a regular basis with WWC users and to provide the users updated information about the Clearinghouse as new developments occur and reports are available, we have created an email distribution list called the WWCUpdate. Users subscribe to the list by keying in their email address; verification of their subscription is sent back to the user. Users may unsubscribe at anytime via the website.

• WWCUpdate Subscription Form

In addition, in order to improve our ability to serve the public’s information needs, we have created and administered a query of WWC website users to better understand the WWC user base:

• WWC Website User Type Query

Clearance is requested to continue these data collection efforts that support the development, operation, and evolution of the WWC and that provide the public with information they request.

This supporting statement describes the justification for the WWC data collections, and each form is detailed in the attachments. This document, which addresses OMB concerns regarding respondent burden and paperwork control, has been prepared according to guidelines for completing the justification statement to accompany OMB Form 83-I.

Justification

1. Necessity of Data Collection

There is an increasing recognition among policymakers that decisions about educational programs, products, and practices should be based on evidence of effectiveness from high quality scientific research. The No Child Left Behind Act (PL 107-110), for example, includes numerous requirements to use the findings from “scientifically based research” in designing and implementing educational interventions. The implementation of this Act—with its promise to serve the educational needs of all children, and its emphasis on “doing what works”— creates new opportunities and a strong impetus to identify, select, and implement effective school improvement strategies.

However, meeting the challenge and achieving the promise of No Child Left Behind will prove ineffective unless education decision-makers, including parents, are provided with the timely information and support they need to make wise choices from among the ever-increasing options. Those who have a stake in improving education urgently want credible and reliable information that allows them to judge the effectiveness and quality of alternative approaches to raising student achievement. Educators, policymakers and the public cannot be expected to do “what works” until they actually know what works. By transforming education into an evidence-based field, the U.S Department of Education is working to fulfill its mission of improving student outcomes and providing the information that decision makers need.

The What Works Clearinghouse (WWC) is a key part of this effort. Established by the U.S. Department of Education’s Institute of Education Sciences in 2002, the purpose of the WWC is to offer everyone—from public officials to the public at large—a central, independent, and trusted source of scientific evidence of what works in education. In order to be successful, the WWC will involve key constituents in the process of its activities, in the development of its products, and in identifying areas for improvement.

In support of this effort, the web-based submission forms posted on the What Works Clearinghouse website www.whatworks.ed.gov: (1) allow the public to nominate evidence-based research studies, interventions and topics for review by the What Works Clearinghouse, (2) collect information about evaluators who want to supply the public with information about their available services, and (3) assess the usefulness of the website in meetingassess who uses the WWC database and website in order to better serve the public’s need for information its users’ needs. The information from the online forms will ensure a greater coverage of research to be reviewed and participation of researchers and product developers in the field. Accordingly, the WWC requests continued OMB clearance to administer the following forms:

• Intervention, Study, and Topic Nomination forms. The online forms allow individuals to nominate a topic area, a replicable intervention, or a research study for WWC review. The website will have a privacy statement that any individual identification will not be posted on the public website without the user’s permission.

• Outcome Evaluator Registration Form. Individuals and organizations that subscribe to the WWC standards of evidence and would like to provide evaluation assistance to sponsors and users of evaluation studies may submit their qualifications via a registration form. This information will be included in an online searchable database for users to find an evaluator for their project and/or product.

• WWCUpdate Subscription Form. On the WWC Website users are offered the opportunity to register for a subscription to the WWCUpdate, an email distribution list to announce new information about the Clearinghouse.

• User Type Query. The WWC User Type Query is a one-question pop-up survey posed to users who register for inclusion on the website’s email distribution list (WWCUpdate), or who submit comments or questions to the WWC via web forms. For that specific visit, users will be asked, “In what capacity are you currently visiting the What Works website?”and presented with a list of categories, including administrator, local education agency, parent, researcher, student, teacher. Data collected will be used to better understand the WWC user base in order to better serve their information needs.

• WWCUpdate Subscription Form. On the WWC Website users are offered the opportunity to register for a subscription to the WWCUpdate, an email distribution list to announce new information about the Clearinghouse.

We would like to note that the WWC also collects information from focus groups, study authors, and from intervention developers. The nature of the information collected is described below; we believe these efforts are exempt from OMB clearance.

• We also plan to conduct two or three focus groups a year, but we will not ask the same or similar questions of more than nine focus group participants and therefore are not seeking clearance for these focus groups. Focus group topics will remain flexible and responsive to feedback from our users and will vary with the evolving activities of the WWC.

• Study authors are contacted at an ad hoc basis to respond to specific questions regarding their particular studies. Because each study is unique, and the questions the WWC poses differ depending on what data are missing or what information is unclear, none of the authors are asked identical questions.

• Study authors are also asked to complete an Author Agreement form. This is a legal document intended to identify any conflict of interest (e.g., a fiduciary relationship between the study author and the developer of the intervention) and is not a survey form that will undergo public reporting or data analysis.

• Intervention developers are contacted and asked to complete an embargo agreement. This is a legal document and not a survey form that will undergo public reporting or data analysis.

2. Purposes and Use of the WWC Data Collection Forms

Below we explain the purpose of each of these voluntary data collection forms.

Intervention Nomination Form. The purpose of the Intervention Nomination Form is to collect information from education producers and consumers about specific interventions within Topic Areas that they would like to be reviewed. Nominations will be used to inform the WWC on interventions that should be considered for inclusion in WWC Evidence Reports.

The long Intervention Nomination Form (attachment Form 01) includes three main data elements that provide the WWC with critical information to assess interventions’ relevance to WWC Topic Areas, and two optional data elements. The short Intervention Nomination Form (attachment Form 02) is similar to the long form, but omits the In-Depth Description and Background of Intervention. Respondents are first asked to identify relevance to the current topic areas, then to provide a general description of the intervention. If the respondent is willing and able to provide additional details about the intervention, the user is directed to an additional details screen (the one screen that distinguishes between the long and the short forms). If the user does not have any additional information to provide, the two optional information screens are presented next, and the nomination process is completed.

• Relevance to current Topic Areas

• General Description of Intervention

• In-Depth Description and Background of Intervention (e.g., target population, history of implementation; LONG FORM ONLY)

• Optional Contact Information

• Optional User Type Categories

Study Submission Form. The purpose of the Study Submission Form is to gather information on studies from members of the public who want to inform the WWC of studies that are relevant to WWC Evidence Reports. The online version of this form will enable users to enter and submit data electronically. The WWC will use these submissions to supplement the studies located by staff, and will be considered for inclusion in WWC Evidence Reports.

The proposed Study Submission data elements (Appendix A.2) have been designed to enable researchers and product developers to voluntarily submit studies to be reviewed by the WWC. Users who submit research will have a user-friendly vehicle to submit their studies and the opportunity to add to the knowledge base of their respective fields. The data elements that will be captured do not contain any sensitive data about the individuals or organizations submitting the information, but rather information about their study and its accessibility. The Study Submission Database elements are attached in Form 03 (long form) and Form 04 (short form). The long and short forms differ by just one screen. Similar to the intervention nomination process, users are directed to the additional Study Profile screen of the long form only if they note that they are willing and able to provide additional information about the study. The elements of these forms are:

• Study Details

• Study Profile (LONG FORM only)

• Optional Contact Information

• Optional Electronic Document Release Form

• Optional User Type Categories

Respondents who submit a full-text version of the study will be sent an Electronic Document Release Form (included at the end of attachments Form 03 and 04). This form is based on two forms that have been used in the past (as ERIC’S Reproduction Release Form and the National Criminal Justice Reference Service’s Request for Reproduction Rights Release). It should be noted that respondents are not required to fill out this form, and may forward it to the appropriate permission-granting entity, ignore it, or fill it out if they have the authority to do so.

Topic Nomination Form. The purpose of the Topic Nomination Form is to gather in-depth information from education producers and consumers about the topic areas they would like to suggest for review by the WWC. Information gathered via the Topic Nomination Form will be used to help prioritize topics proposed in the Annual Plan.

The Topic Nomination Form includes three main data elements that will provide the WWC with critical information to assess the significance of the topic in improving student educational outcomes, and two optional data elements. The form is included as attachment Form 05 and contains the following types of information:

• Description of Topic Area

• Purpose for Nomination

• Optional Contact Information

• Optional User Type Categories

The WWC will review the topic areas nominated by using a set of criteria to prioritize nominations before recommending them to the U.S. Department of Education. These criteria will include the perceived educational importance of that topic, and whether a current supply of methodologically rigorous and substantively relevant research on that topic is available.

The WWC Submission Flowchart (Exhibit 1) demonstrates the path that website users will be prompted to take during the online submission process.

Responses to the intervention, study, and topic nomination forms will be captured in a database. Case ID numbers will be used in association with respondents’ data. Individually identifying information will only be collected when the respondent chooses to submit the optional contact information. This information is for internal staff use only, and will only be used if there are questions about the respondent’s submission. Other identifying information about respondents that will be included in the data file is their response to the following optional question, “In what capacity are you currently visiting the What Works website?” as exhibited in attachment Form 08 (User Type Query). This information is collected only to determine the types of information needs that each of our user groups has, and will not be used for any other purpose.

Data will be stored in a secure database on the WWC private website. At the end of the contract, the WWC will provide the U.S. Department of Education with the database that was created during the project.

EXHIBIT 1
WWC Submission Process

Registry of Outcome Evaluators Form. The Registry of Outcome Evaluators is designed to provide potential sponsors of effectiveness studies with information about individual and organization evaluators who subscribe to the standards of evidence established by the WWC. The goal of this registry is to contribute to advancing the quality of educational research by 1) assembling a substantial list of committed outcome evaluation providers and 2) supplying tools and guides that help consumers decide what they need, how to assess the qualifications of the evaluators, how to use the data supplied to find potential evaluation providers, and how to supplement the help provided by the registry with other sources of assistance. To be included, evaluators must complete a registration form and sign a commitment letter on the password-protected site (see attachment Form 06).

The WWC and the Department do not intend to assess the quality and accuracy of the information supplied by evaluators wishing to be listed in the Registry of Outcome Evaluators, only its completeness. This will be stated clearly in the searchable online registry. Participation is voluntary. Because the registry will rely on self-reported data, sufficient information will be needed so that members of the user community can determine for themselves whether the evaluators can meet their outcome evaluation needs. The registry will provide a valuable resource for developers who wish to commission evaluations of their products. Furthermore, over time, the registry will contribute to the mutual education of consumers and evaluation providers because both will be able to use the database to better understand what research is wanted and what can be reliably delivered given the capacity of the provider, the methodologies employed to answer critical questions, and the resources available for the project.

Information planned for this registry is infrequently available for any field and even less commonly directed to evaluations and evaluators focused on causal inference. We know of no other evaluations registries that seek to provide this much information about provider capabilities. The American Educational Research Association does provide a web-based registry of individual educational researchers, but the information available is limited to contact and location data, educational degree, and research interest. The registry contains more than 1,000 names and approximately 180 different research topics. Although useful for AERA members’ purposes, it is not readily adaptable to the needs of WWC users because it does not focus on evaluators’ experience in conducting impact studies.

To be listed in the online WWC Registry of Outcome Evaluators, registrants must sign a letter acknowledging their commitment to the standards for causal inference studies adopted by the WWC for its reviews and their commitment to related professional standards and ethical practices in their discussions, proposals, and conduct of such studies with sponsors or clients. This feature of the registry is intended to strengthen and improve the quality and rigor of such evaluations and research studies. The primary audiences for the commitment letter will be evaluation providers, professional associations, and the research community in general.

T

he proposed Registry of Outcome Evaluators Form has been designed for use by individual evaluators and organizations. It consists of six main data elements that will provide the end users with crucial information needed to assess the evaluator’s capability and experience to provide a competent causal inference study. The data elements capture such critical elements as relevant methodological experience, staff educational attainment and work experience, and experience with the target groups of interest in the potential study. These data elements are attached in Form 06 and are summarized below:

• Contact Information

• Services Offered by Type

• Geographic Availability of Services

• Experience by Type of Service and Target Population (within the last five years)

• Staff Credentials and Experience (for Organizations only)

• Relevant Publications and Sponsors

Evaluators will self-select to be listed in the Registry of Outcome Evaluators. Evaluators who want to be listed on the registry will be asked to sign a statement saying that they subscribe to and are prepared to meet the standards and criteria of the WWC in their work for clients and sponsors. The commitment letter, included at the end of attachment Form 07, contains the following elements:

• Commitment to the highest standards of scientific evidence for answering questions on the effectiveness of educational interventions, as defined by the WWC

• Commitment to high ethical and professional standards and avoidance of conflict of interest

In the commitment letter evaluators will also provide their signed permission for the WWC to release the information about evaluators through the searchable Registry of Outcome Evaluators. The form does not contain any questions of a sensitive or controversial nature.

The registry will remain current in several ways. There will be an annual data collection effort to contact those included in the registry and remind them to update their entries, and a sunset rule is being implemented, to eliminate registry entries not updated within 18 months. To encourage updates, the WWC will provide evaluators in the registry with automatic prompts about updates and the impending imposition of the sunset rule. These prompts will begin 13 months after the evaluator registers and will repeat monthly until updates are received or 18 months have passed since the last update. Further, evaluators may list only causal inference studies completed within the last five years. Incomplete data element entries will be returned for additional information.

The registration form and the updates will be operated online directly through the website with the exception of those evaluators, if any, who do not have Internet capacity or who choose to submit a paper copy for other reasons. At the completion of the contract, the contractor will deliver all files, including the Registry of Outcome Evaluators, to the Department.

User Type Query. The WWC User Type Query will be a one-question pop-up survey in which users who register for inclusion on the website’s email distribution list, or who submit comments or questions to the WWC via web forms, will be asked to categorize their user type for purposes of the specific visit during which the pop-up survey appears. User Type categories are based on U.S. Department of Education–approved metadata for audience categories. The categories match those used in the Survey of Customer Satisfaction and Needs (whose clearance is addressed under 1800-0011) and Needs and the Nomination Forms submitted with this package. Data collected will be used to better understand the WWC user base so that we can better serve their information needs.

A drop-down screen of user types will be shown when a user signs up for the WWC Update, nominates a topic, intervention, or study, or sends an email. Users may opt out of the query at any time by closing the query window and continue with their web activity. We will analyze and report the percentage rates of WWC website users by type on a monthly basis to assist us and the Department in understanding and better serving WWC’s user base. We will store survey data in a secure database on the WWC private website. We will use SAS for a frequency analysis for each user type, and we will conduct content analyses of the questions and comments received by different types of users. Data collected from WWC Update subscribers and the users submitting questions and comments or submitting nominations will be analyzed separately.

WWCUpdate Collection Form. The purpose for the WWCUpdate Collection Form (attachment Form 09) is to provide the users that are interested in regular Clearinghouse updates a means of receiving that information via the internet using their email addresses. The email addresses will be stored in a secure database and used for email broadcasting as new WWC information is available.

3. Use of Technology in Information Collection

All WWC databases and customer survey and WWCUpdate forms will be web-based. Website users may choose to print out the forms from the website and submit a paper copy, but we expect most users to submit the forms electronically via the automated website interfaces. These interfaces have been designed to make the data collections as low burden as possible, while still gathering the information needed to inform the WWC. For example, the intervention, study, and topic nomination forms use preset response options for many items. The user needs only to click on the appropriate option. There is also an automated skip pattern built in to these forms, allowing users to specify whether or not they would like to give more detailed information on any of their nominations. If someone would like to nominate an intervention, for example, they may only know the name of the intervention, or they may have information about the developer of the intervention. Users will only be asked items that match the level of detail they specify.

To reduce the burden on evaluators using the Evaluator Registry, returning users may update their forms without having to resubmit their data. Assistance is also provided in using the web-based forms. For example, the nominations forms link users to definitions of key terms, where appropriate. In addition, WWC website users have easy access to personal and individual guidance if needed, and information on how to contact the WWC is visibly presented on the website.

4. Efforts to Identify Duplication

The information to be collected from the nomination forms, the Registry of Outcome Evaluators database, the customer survey, and the user query is not available from any other source and will not duplicate any existing data collection effort.

5. Burden on Small Business Organizations

Not applicable to this data collection.

6. Consequences of Less Frequent Data Collection

The nomination forms are voluntary, and sought out by the WWC website users; therefore, it is assumed that these forms will only be filled out when the user wishes to nominate an intervention, study, or topic for review. The Registry of Outcome Evaluators database submitters also participate by self-selection, but are prompted to update their information on an approximately annual basis. This is necessary to keep the contents of the database up to date and useful to the public.

The user query will pop up when users sign up for WWC Updates or submit a comment through the website. This will not be a frequent event; however, if a user is queried more than once in a short period of time, he or she may simply close the window. Also, users subscribing to the WWCUpdate must submit their email address; subscribing to WWCUpdate is voluntary.

The nomination forms and customer survey form will also be advertised periodically at presentations and in emails to What Works Network members, to encourage users and potential users to visit the website and submit their requests and feedback. If we did not advertise these opportunities periodically, existing users may forget that this feature of the website is available to them, and potential users may not learn about how to submit their nominations and information needs.

7. Special Circumstances

Users may submit copyrighted publications. These publications will not be shared with the public until the appropriate permissions are obtained. Users may also submit unpublished studies, and request that they be kept confidential. These studies will also not be shared with the public until permission is obtained.

8. Outside Consultants

Since the contract award date (August 6, 2002), WWC staff members have presented to a wide range of administrators, researchers, policymakers, practitioners, and others to inform them about the WWC and to allow them to inform us of their information needs. For example, WWC staff presented to a group of evaluation researchers and then held an informal discussion session with attendees to go over the types of information that the Registry of Outcome Evaluators should contain. All of the information provided through these types of interactions and presentations have guided the design of the appended forms.

9. Explanation of Providing Payment or Gifts to Respondents

No payment or gift of any kind will be provided to respondents.

10. Assurance of Confidentiality

For the nomination, , user query, and WWCUpdate forms, the website or pop up window will have a privacy statement that any individual identification will not be posted on the public website without the user’s permission. Data about users (e.g., customer survey and user query forms) will be reported at the aggregate level only. Contact information on nomination forms, when provided, is used for the express purpose of clarifying the WWC user’s nomination, and will not be released beyond the WWC staff with an immediate need to clarify the submitted information. The WWCUpdate subscribers’ email addresses will be stored in a secure database and used only for the purpose of updating subscribers on new Clearinghouse information.

The purpose of the Outcome Evaluators database is to allow evaluators to post their credentials and contact information to the public; therefore it is assumed when they submit their form that they are not requesting confidentiality. The WWC does, however, offer the evaluators the option of limited disclosure for studies. For example, there is an item on the Outcome Evaluators database that allows the respondent to request that proprietary information be kept confidential; this will typically only be requested when an evaluator is in the process of getting a study published, and would like us to wait until the study is public to post the study information. Similarly, the study nomination form allows WWC users to request and receive confidentiality for their study nomination. If a nomination indicates that the study should be kept confidential, the WWC agrees to post the bibliographic information, study abstract, and rating, but not the full text of the study.

11. Sensitive Questions

None of the requested information is sensitive in nature.

12. Estimates of Hour Burden

The total number of respondents is estimated to be 12,817 across the three years, with the total annualized responses at 4,272. The total number of burden hours for all data collections is estimated to be 1,055 with the total annualized hours at 352. Hourly burden estimates by form and totals by year are displayed below, in Exhibit 2.

EXHIBIT 2
Estimates of Hourly Burden

Form	Year	Time to Complete	Number of Respondents	Hourly Burden
Intervention Nomination, Long Form	Y1	30 minutes	200	100.00
Intervention Nomination, Short Form	Y1	5 minutes	200	16.67
Study Submission, Long Form	Y1	30 minutes	200	100.00
Study Submission, Short Form	Y1	5 minutes	200	16.67
Topic Nomination	Y1	30 minutes	200	100.00
Evaluator Registration	Y1	120 minutes	60	120.00
User Type Query	Y1	15 seconds	3,000	12.50
WWCUpdate Subscription	Y1	15 seconds	500	2.00
Total, Year 1			4,559560	469.007.84
Intervention Nomination, Long Form	Y2	30 minutes	100	50.00
Intervention Nomination, Short Form	Y2	5 minutes	100	8.33
Study Submission, Long Form	Y2	30 minutes	100	50.00
Study Submission, Short Form	Y2	5 minutes	100	8.33
Topic Nomination	Y2	30 minutes	100	50.00
Evaluator Registration	Y2	120 minutes	30	60.00
Evaluator Update	Y2	30 minutes	100	50.00
User Type Query	Y2	15 seconds	3,000	12.50
WWCUpdate Subscription	Y2	15 seconds	500	2.00
Total, Year 2			4,129130	293.001.16
Intervention Nomination, Long Form	Y3	30 minutes	100	50.00
Intervention Nomination, Short Form	Y3	5 minutes	100	8.33
Study Submission, Long Form	Y3	30 minutes	100	50.00
Study Submission, Short Form	Y3	5 minutes	100	8.33
Topic Nomination	Y3	30 minutes	100	50.00
Evaluator Registration	Y3	120 minutes	30	60.00
Evaluator Update	Y3	30 minutes	100	50.00
User Type Query	Y3	15 seconds	3,000	12.50
WWCUpdate Subscription	Y3	15 seconds	500	2.00
Total, Year 3			4,129130	293.001.16
Total, Years 1-3			12,817820	1,055.000.16

The estimated annualized cost to respondents shown in Exhibit 3 is based on the average hourly wage ($18.62) as of June 2005, as reported by the Bureau of Labor Statistics National Bulletin:¹

EXHIBIT 3
Estimates of Hourly Burden

Form	Year	Number of Respondents	Hourly Burden	Cost
Intervention Nomination, Long Form	Y1	200	100.00	$1,862.00
Intervention Nomination, Short Form	Y1	200	16.67	$310.40
Study Submission, Long Form	Y1	200	100.00	$1,862.00
Study Submission, Short Form	Y1	200	16.67	$310.40
Topic Nomination	Y1	200	100.00	$1,862.00
Evaluation Registration	Y1	60	120.00	$2,234.40
User Type Query	Y1	3,000	12.50	$232.75
WWCUpdate Subscription	Y1	500	2.00	$37.24
Total, Year 1			467.84	$8,711.18
Intervention Nomination, Long Form	Y2	100	50.00	$931.00
Intervention Nomination, Short Form	Y2	100	8.33	$155.10
Study Submission, Long Form	Y2	100	50.00	$931.00
Study Submission, Short Form	Y2	100	8.33	$155.10
Topic Nomination	Y2	100	50.00	$931.00
Evaluator Registration	Y2	30	60.00	$1,117.20
Evaluator Update	Y2	100	50.00	$931.00
User Type Query	Y2	3,000	12.50	$232.75
WWCUpdate Subscription	Y2	500	2.00	$37.24
Total, Year 2			291.16	$5,421.40
Intervention Nomination, Long Form	Y3	100	50.00	$931.00
Intervention Nomination, Short Form	Y3	100	8.33	$155.10
Study Submission, Long Form	Y3	100	50.00	$931.00
Study Submission, Short Form	Y3	100	8.33	$155.10
Topic Nomination	Y3	100	50.00	$931.00
Evaluator Registration	Y3	30	60.00	$1,117.20
Evaluator Update	Y2	100	50.00	$931.00
User Type Query	Y3	3,000	12.50	$232.75
WWCUpdate Subscription	Y3	500	2.00	$37.24
Total, Year 3			291.16	$5,421.40
Total, Years 1-3			1,050.16	$19,553.98

13. Estimate of Annual Cost Burden to Respondents

There are no start-up costs to respondents related to this data collection or other costs not accounted for above.

14. Estimate of Annual Cost to the Federal Government

The total cost to the federal government is $229,251.57 for all data collections.

Because the majority of the cost of data collections is due to the integrated database system, costs are presented by year but not by separate data collection. Exhibit 4 displays the costs for maintaining and administering the online forms and databases, promoting and requesting input from the public on these systems, and analyzing and tabulating the results. The information is broken down by year and by the organizations involved in the data collections.

EXHIBIT 4
Estimated Cost to Federal Government, by Year and Organization

Year	AIR	Caliber	Aspen	Total
Year 1	$17,747.74	$10,314.10	$43,948.67	$72,010.51
Year 2	$18,812.61	$10,932.94	$46,585.59	$76,331.14
Year 3	$19,941.10	$11,588.76	$49,380.06	$80,909.92
Total	$56,501.45	$32,835.80	$139,914.32	$229,251.57

15. Program Changes or Adjustment

This set of data collections will result in an annual burden of 468 hours in Year 291 hours in Year 2, and 291 hours in Year 3. The annualized hours total is 352, which is a decrease in hours. Based on the numbers of public submissions we have received in the past three years, we have adjusted down the figures to estimate the number of future responses more accurately, as well transferred the customer survey hours to 1800-0011.

16. Plans for Tabulation and Publication of Results

The majority of tabulations of the data collected under this request will be for internal use by the WWC contractors and Department of Education staff. These data will be used to ascertain the information needs of the public and to take their suggestions, as well as to monitor how well the WWC is meeting their needs. For example, the user type query data can be crosstabulated with the questions and comments we receive from users to determine whether the WWC is underserving a particular group of users.

Some data collected will be pertinent to the indicators required by the Government Performance and Results Act of 1993, and may be published accordingly.

The GPRA indicators for the WWC are the number of annual web site hits.

17. Approval to Not Display OMB Expiration

This approval is not requested.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions.

1 Downloaded January 18, 2007 from http://www.bls.gov/ncs/ocs/sp/ncbl0832.pdf