Supporting Statement -- Umbrella Generic Clearance

CFACT

A-Z Quick Menu:
AHRQ Directory:

SUPPORTING STATEMENT

Re-approval of the Voluntary Customer Survey Generic Clearance
for the Agency for Healthcare Research and Quality
OMB No. 0935-0106
Offices/Centers/Div

A. JUSTIFICATION

1. Need for Information
Executive Order 12862 directs agencies that "provide significant services to the public" to "survey
customers to determine the kind and quality of services they want and their level of satisfaction with
existing services." This is a request for the Office of Management and Budget (OMB) to re-approve for an
additional 3 years, under the Paperwork Reduction Act of 1995, the generic clearance for the Agency for
Healthcare Research and Quality (AHRQ) to survey the users of AHRQ’s work products and services.
AHRQ is the lead agency charged with supporting research designed to improve the quality of healthcare,
reduce its cost, improve patient safety, decrease medical errors, and broaden access to essential services.
AHRQ sponsors and conducts research that provides evidence-based information on healthcare outcomes;
quality; and cost, use, and access. The information helps healthcare decisionmakers—patients and
clinicians, health system leaders, and policymakers—make more informed decisions and improve the
quality of healthcare services. The mission of AHRQ is to improve the quality, safety, efficiency, and
effectiveness of health care for all Americans. Re-approval of the generic clearance will aid us in achieving
this mission.
Customer surveys will be undertaken by AHRQ to assess its work products and services provided to its
customers, to identify problem areas, and to determine how they can be improved. For example, AHRQ's
Office of Communications and Knowledge Transfer (OCKT) is working with the Federal Consulting Group
within the Department of the Treasury to determine how effective the agency’s web site is and how it might
be improved. Attachment A contains a list of surveys initiated under this guidance during the past three
years as well as surveys being considered to demonstrate the nature of work covered by this clearance.
Surveys conducted under this generic clearance are not required by regulation and will not be used by
AHRQ to regulate or sanction its customers. Surveys will be entirely voluntary, and information provided
by respondents will be combined and summarized so that no individually identifiable information will be
released
In accordance with OMB guidelines for generic clearances for voluntary customer surveys and Executive
Order 12862, AHRQ (1) has established an independent review process to assure the development,
implementation, and analysis of high quality customer surveys within AHRQ; (2) will provide periodic
progress reports on the conduct of surveys under the generic approval, summarizing the actual burden; (3)
will provide OMB with copies of the survey instruments for inclusion in the docket; and, (4) will notify
OMB of any significant changes in proposed survey instruments.

2. How, by Whom, and for What Purpose Information Will Be Used
The information collected through focus groups and voluntary customer surveys will be used by AHRQ to
identify strengths and weaknesses in products and services to make improvements that are practical and
http://intranet.ahrq.gov/ccfs/oc_index.asp?OC_ID=5&oc_name=CFACT (1 of 7)11/13/2006 4:01:06 PM

CFACT

feasible. Information from these customer surveys will be used to plan and redirect resources and efforts to
improve or maintain a high quality of service to the lay and health professional public.

3. Use of Improved Information Technology
Improved electronic technology (e.g., AHRQ Web-based materials) will be used whenever possible to
reduce the burden on the public. In some instances, however, the most appropriate methodology will
involve written or oral responses to brief questionnaires, interviews and focus groups.

4. Efforts to Avoid Duplication
Each survey will be designed to reflect the specifics of the customer population served. During the
development of these voluntary instruments, numerous groups within and outside of AHRQ will be
consulted. Plans to conduct surveys will be reviewed prior to implementation, and any potential duplication
will be identified in the review and approval process.

5. Small Businesses
The survey instruments and procedures for completing the instruments will be designed to minimize burden
on all respondents and will not have a significant impact on small businesses or other small entities. The
burden is entirely voluntary.

6. Consequences of Less Frequent Collection
These surveys are appropriate vehicles to examine public perception of AHRQ's ability to respond timely to
the needs of their customers. Much of AHRQ’s work is rapid-cycle and demand-driven. Collection of data
on a less frequent basis would reduce the practical utility of the information as well as inhibit the ability of
AHRQ to identify and monitor problems and to make changes to improve products and services.

7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)
The data collection efforts will be consistent with the guidelines at 5 CFR 1320.5(d)(2).

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
AHRQ will consult with in-house statistical staff, other Federal agencies, and other organizations, which
have conducted, or may conduct, similar surveys to identify areas of interest and concern to customers. As
appropriate, AHRQ will establish panels of outside experts to assist in design and implementation of the
surveys.
Comments on 60-day Federal Register Notice, if any, will be included as an attachment herein and become
a matter of public record.

9. Remuneration of Respondents
AHRQ will provide no remuneration to respondents to written, telephone, web, or any other form of survey
or interview. AHRQ will, on a case-by-case basis, consider modest remuneration for focus group
participant’s time and travel. In such cases, the remuneration will not exceed $50 per individual.
Remuneration for focus group participation is a recognized standard industry practice, without which, it
http://intranet.ahrq.gov/ccfs/oc_index.asp?OC_ID=5&oc_name=CFACT (2 of 7)11/13/2006 4:01:06 PM

CFACT

would be difficult to achieve appropriate and adequate participation.

10. Assurance of Confidentiality
Respondents will be advised that surveys are entirely voluntary and that any information they provide will
be combined and summarized with information provided by others and no individually identifiable
information will be released. In instances where respondent identity is needed, the information collection
will fully comply with all respects of the Privacy Act.
11. Questions of a Sensitive Nature
No questions of a sensitive nature are anticipated under this generic clearance.

12. Estimates of Annualized Hour Burden
The estimated annual hour burden is as follows:

Average Hours
Number of per Response Total Hours
Wage
Type of Survey
Respondents
Rate*
Total Cost
Mail/Telephone Surveys
51,200
.15
7,680
$25
$192,000
Automated/Web-based
52,000
.163
8476
$25
$211,900
Focus Groups
200
1.0
200
NA
$0
Totals
103,400
NA
16,356
NA
$403,900
*Note: There is no cost to focus group participants inasmuch as it is customary to pay participants a small
honorarium, typically $50, in consideration for their participation and their transportation costs.

13. Estimates of Annual Cost Burden to Respondents
Costs to Respondents: Costs to respondents will be limited to their time to provide the requested
information. Based on a rate of $25 per hour and the annual total burden of 103,200 hours for other than
focus groups, the annual cost to respondents is anticipated to be $403,900. As noted in the annualized
burden table in item 13, there is no cost to focus group participants inasmuch as it is customary to pay
participants a small honorarium, typically $50, in consideration for their participation and their
transportation costs

14. Estimates of Annualized Cost to the Government
The mail/telephone surveys and focus groups will in some cases be carried out under contract. Assuming
the contract cost per survey are $50,000 - $100,000, and for each focus group are $20,000 total contract
costs could run $ 720,000 per year.

15. Change in Burden
Adjustments are based on experience gained as a result of the first three years of processing requests under
this generic clearance (OMB No. 0935-0106).

16. Plans for Analyses
http://intranet.ahrq.gov/ccfs/oc_index.asp?OC_ID=5&oc_name=CFACT (3 of 7)11/13/2006 4:01:06 PM

CFACT

The purposes of these surveys are to identify problems areas, determine their magnitude and scope, and
develop a plan to correct them. It is also the purpose of these surveys to identify successes to learn how
they succeeded and how those lessons can be applied to other AHRQ efforts. The analyses will be
descriptive and, to the extent that they can, inferential and generalizable. The results of these findings are
primarily for internal use but may be shared with key government policy and management officials, AHRQ
staff, public and private health providers, and members of the general public.
For the types of surveys described earlier, the following analyses would be appropriate:
a.
Mail/telephone/electronic surveys: Basic descriptive analyses are expected for these types of
surveys. Probability samples will be selected for these surveys so the opportunity exists for some
generalizations to the population.
b.
Focus groups: Participants will be selected purposively, so that no generalizations to the
population will be possible. Focus groups will be used to identify problems and issues for further
study and, in some instances, "brainstorm" for possible solutions. The analyses will be qualitative
and consist mostly of narrative summaries of the discussions.
c.
Automated/Web-based: electronic technology will be used for this survey. It will be mounted
on the website for voluntary response as an electronic evaluation form. In addition to summarizing
responses to questions, basic demographic information will be collected and summarized.

17. Exemption for Display of Expiration Date
No exemption is being requested.

18. Certifications
These activities will comply with the requirements of 5 CFR 1320.9.

STATISTICAL METHODS
1. Potential Respondent Universe and Sample Selection Method
In some instances statistical methods will apply.
The respondent universe will be separately identified for each program whose customers are to be
surveyed. Surveys will be designed to minimize burden on respondents while obtaining essential
information. The expectation is that information collection instruments will require no more than 20-25
minutes response time, on average. Appropriate probability sampling techniques will be used to select
samples.
In many instances, however, there will be an existing list of "customers" readily available for sampling (e.
g., mailing lists for publications or recipients of particular materials or services within known customer
groups). Appropriate probability sampling techniques will be used to select samples.

2. Information Collection Procedures
All information collection will be conducted in a manner that is consistent with the following guidelines:
• Participation will be fully voluntary, and non-participation will have no effect on eligibility for, or
receipt of, future AHRQ health services research.
http://intranet.ahrq.gov/ccfs/oc_index.asp?OC_ID=5&oc_name=CFACT (4 of 7)11/13/2006 4:01:06 PM

CFACT

• Appropriate sample sizes will be determined for each activity to ensure that burden is minimized
while reliable estimates are produced.
• Information collection will be limited to that needed to assess customer reaction to AHRQ products
and services. Repeated implementation of quantitative surveys will be at an interval appropriate to
measure the impact of changes and to monitor reaction levels.
• Given the voluntary nature of the data collection surveys, efforts will be made to obtain the highest
possible response rates. Efforts will also be made to assess non-response bias, to the extent feasible.

3. Methods to Maximize Response Rate
The design of each quantitative survey will include approaches to maximize response rates, while retaining
the voluntary nature of the effort, consistent with appropriate survey methodology. Additional formal
pretesting will be carried out at a level and in a manner consistent with the specific survey.

4. Tests of Procedures
It is anticipated that most surveys will begin with efforts by AHRQ staff or in some cases by focus groups
to identify the views/concerns of customers. Most formal pretesting will be carried out at a level and in a
manner consistent with the specific survey.

5. Statistical Consultation and Independent Review
Each program will obtain input from statisticians as to the development, design, conduct, and analysis of
customer surveys. This statistical expertise will be available from AHRQ statisticians/contractors.
Technical assistance in survey design and statistics may, in some cases, be sought through The National
Center for Health Statistics.

ATTACHMENT A

CURRENT and PROPOSED
AHRQ CUSTOMER SATISFACTION SURVEYS

Current AHRQ Surveys Being Conducted CUSTOMER SATISFACTION SURVEYS

OMB No. 0937-0201- Project Officer: Gerri Michael-Dyer/OHCI
"Survey to Evaluate the AHRQ Web Site"
This customer satisfaction survey is assessing customer opinions on technical capabilities, the content and
presentation of electronic materials, the ease of usage, and the appropriateness of the information offered
through this electronic dissemination channel, the AHRQ Web Site.
OMB No.0937-0201-Project Officer: Judith Wilcox/OHCI
"Survey of AHRQ Publications Clearinghouse"
http://intranet.ahrq.gov/ccfs/oc_index.asp?OC_ID=5&oc_name=CFACT (5 of 7)11/13/2006 4:01:06 PM

CFACT

The primary use of the data being collected from this customer satisfaction survey is to determine the level
of satisfaction experience by the customer during the telephone ordering process, to solicit customer
recommendations for enhancing the service, for improving satisfaction, and for inclusion by the Agency
administrative reporting.
OMB No. 0937-0201- Project Officer: Donnarae Castillo/OHCI
"Customer Satisfaction Survey of AHRQ's Publications Recipients"
This information is needed to determine if AHRQ's publications and dissemination program are meeting
current customer needs and to identify strengths and weaknesses in current products and dissemination
procedures. It will be used to plan and redirect resources as needed. Also, budgetary constraints require us
to identify how to make dissemination more cost-effective and maximize use of Clearinghouse resources.
OMB No. 0935-0106-Project Officer: Margaret Coopey/CPTA
"Customer Satisfaction Survey to Assess the Usefulness of Evidence-based Practice Centers'
Products"
Assessment of the usefulness of Evidence-based Practice Centers (EPC) from the perspective of the users of
the information is being collected via telephone and mail surveys. These reports are being used as an
important mechanism for dissemination of the review and synthesis of evidence from the EPC Program.
OMB No. 0935-0106 - Project Officer: Debbie Rothstein/ORREP
"Survey of Trainee Satisfaction with AHRQ-Funded Health Services Research Traineeship Program"
The agency currently supports 25 institutional health services research-training programs within the
National Research Service Award (NRSA) program. This survey is assessing customer satisfaction of
scholars whose academic training has been supported by AHRQ institutional training program grants over
the last 13 years. It will examine the adequacy of training experiences as related to career needs. It will
provide AHRQ with concrete suggestions for improvement in future program planning.
OMB No. 935-0106- Project Officer: Marge Keyes/CQMI
"CONQUEST 2.0 Customer Satisfaction Survey"
The purpose of the survey is to assess customer satisfaction with the COmputerized Needs-Oriented
QUality Measurement Evaluation SysTem (CONQUEST 2.0) identify strengths and weaknesses and
enhance the utility and relevance of the information contained in the CONQUEST software. Customer
feedback on the content and presentation of clinical performance and quality measures, the ease of
navigation through the database, and the appropriateness of the information will assist AHRQ with design
of future versions of the software package, CONQUEST must be flexible and responsive to variety of
audiences.
OMB No. 935-0106- Project Officer: Pat Reynolds/CCFS
"Medical Expenditure Panel Survey (MEPS) Web Site, Data Products, and Publications Customer
Satisfaction Survey"
This survey will gather opinions of the users of the MEPS baseline data on: categories of primary users of
MEPS data products and related publications; the types of data products and research used/requested most
frequently; research topics; user ease of access to the web site information; and the quality and timeliness of
responses to follow-up user Email inquiries. The results will be tabulated to identify strengths and weakness
and enhance the utility and relevance of the information being made available.
OMB No. 935-0106-Project Officer: Jean Slutsky/CPTA
"Customer Satisfaction Survey National Guideline Clearinghouse (NGC) Annual Customer
Satisfaction Survey"
The evaluation goals will be achieved through three types of data collection: (1) Written survey
questionnaires, (2) focus groups, and (3) discussions with individuals working in health care who contribute
guideline development and use. Assignments of data collection modes to target audience groups are
http://intranet.ahrq.gov/ccfs/oc_index.asp?OC_ID=5&oc_name=CFACT (6 of 7)11/13/2006 4:01:06 PM

CFACT

designed to reach the maximum number of respondents and the broadest range of groups. Participation will
be minimally burdensome and is voluntary. Both qualitative data will collected to characterize the
experiences and needs of users in a manner most likely to facilitate improvement activities by AHRQ.
OMB. No. 935-0106-Project Officer: Steve Seitz/OHCI
"Customer Satisfaction Survey to Assess the Usefulness of User Liaison Program Workshops"
The goal of this customer satisfaction survey is to learn how User Liaison Program (ULP) workshop
participants use the information they obtain from these workshops in order to improve their content and
format thus enabling participants to more easily apply information gained. The survey will consist of 20minute telephone interviews with 50 respondents who participated in workshops during the 1999 calendar
year. The interviews will be guided by an interview protocol.
OMB No. 935-0106-Project Officer: Charles Darby/CQMI
"Customer Satisfaction Survey to Assess the National (Consumer Assessment of Health Plans)
CAPHS Benchmarking Database (NCBD)"
The primary purpose of the NCBD is to identify strengths and weaknesses in the process we have employed
to collect the data from customers and the strengths and weaknesses of the products i.e., the benchmarked
reports, we will be developing for them. Customer feedback will help fine tune the forms, process, and data
submission specifications.
Proposed Future AHRQ Surveys:
The following are examples of a few of the proposed surveys from various AHRQ Offices and Centers.
Formal titles have not been assigned as these surveys are currently in the planning stage.
Office of Health Care Information (OHCI):
Customer Satisfaction feedback for AHRQ Web-based Information Products will institutionalize feedback
through a Web based form or survey as we do for the AHRQ Web site online evaluation..
Center for Quality Improvement and Patient Safety (CQUIPS)
(1) Customer Satisfaction Survey to Assess Technical Assistance provided by telephone: This effort
would assist assessment as to how useful and accessible some of the computer-based information
products of our Agency are to target audiences. We would send postcard mail outs to a sample of
product users (30 users for each of 10 products) within three months of purchase to solicit customer
feedback on design, utility, and value.

(2) Customer Satisfaction Survey to Assess Technical Assistance provided on the web site: This
effort would involve follow up surveys of User Liaison Program workshop/focus group attendees to
determine how the information presented there was used in future decisionmaking. This proposed mailin, 20-minute questionnaire would be received by approximately 2000 respondents.
Offices/Centers

Search

Site Map

What's New

Comments

Help

http://intranet.ahrq.gov/ccfs/oc_index.asp?OC_ID=5&oc_name=CFACT (7 of 7)11/13/2006 4:01:06 PM

Print Page

Correspondence

Documents

November 13, 2006