Supporting Statement Part A

Supporting Statement Part A

Evaluation of AHRQ’s Guide to Clinical Preventive Services

OMB No. 0935-0106

Introduction:

This is a request for a one-year clearance for data collection efforts on behalf of the Agency for Healthcare Research and Quality (AHRQ). This evaluation is intended to provide AHRQ with valuable reporting information regarding recipients’ knowledge of, attitudes towards and practices/use of the guide in clinical practice. The primary purpose of this project is to gather information from recipients of a specified publication, the 2005 and 2006 versions of Guide to Clinical Preventive Services (Guide), in order to (1) inform future product versions or enhancements and (2) examine the extent to which, and how, they have used the guides in clinical practice.

A.1. Circumstances Requiring the Collection of Data

In compliance with Section 3506(c) (2) (A) of the Paperwork Reduction Act of 1995, this submission requests OMB approval for a one-year clearance for the Agency for Healthcare Research and Quality (AHRQ) to conduct a study on the practices/use of selected guides produced by AHRQ for a key target audience. Members of the target audience for the Guide are primary care providers (e.g., physicians, osteopaths, nurse-practitioners, physician assistants, nurses). This project is authorized under Executive Order 12862 and is consistent with AHRQ’s mission as the lead agency charged with supporting research designed to improve the quality of healthcare, reduce its cost, improve patient safety, decrease medical errors, and broaden access to essential services. The primary purpose of this project is to gather information from recipients of a specified publication, the 2005 and 2006 versions of Guide to Clinical Preventive Services (Guide), in order to (1) inform future product versions or enhancements and (2) examine the extent to which, and how, they have used the guides in clinical practice.

A multi-method approach (e.g., survey, semi-structured-interview) will be employed for this evaluation. Measures will include outcomes associated with the following variables: acceptance by recipients of the guides, perceived utility of the guides, use of the guides, and the impact of the guides on the awareness of AHRQ and its role in the healthcare field. The authorization for this effort was provided by the Director of AHRQ, Dr. Carolyn M. Clancy, M.D. and is in accordance with Executive Order 12862, which directs field agencies that provide significant services directly to the public to survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services. Authorization to collect these data is given under Section 301 of the Public Health Service Act (42 USC 241).

AHRQ is responsible for sponsoring and conducting research that provides evidence-based information on healthcare outcomes, quality, cost, use and access. The information provided by AHRQ helps healthcare decision makers – patients and clinicians, health system leaders, and policymakers – make more informed decisions and improve the quality of healthcare services. The mission of AHRQ is to improve the quality, safety, efficiency, and effectiveness of healthcare for all Americans. Approval of this generic clearance will aid AHRQ in achieving their mission. In particular, this includes ensuring that research findings emerging from the National Institutes of Health (NIH) and other federally sponsored efforts are ready to use, widely available, and actionable. Ensuring the usefulness of AHRQ’s publications is of utmost importance if AHRQ is to be able to continually improve its efforts to improve the quality, safety, efficiency, and effectiveness of healthcare for all Americans. Specifically, this evaluation is intended to provide AHRQ with valuable reporting information regarding recipients’ knowledge of, attitudes towards and practices/use of the guide in clinical practice.

Mechanisms currently used to disseminate this publication to its target audience include mail, email and a pda version (ePSS) available for downloading. The publication can be ordered by telephone from the AHRQ Clearinghouse or electronically via AHRQ’s website (http://www.ahrq.gov/news/pubcat/c_order.htm).

Justification for the evaluation is therefore based on three factors: (1) the need for AHRQ to assess the practices/use of the guides for its’ target audience, (2) the need for information to support decision making regarding changes to the content and format of existing and future publications, and (3) the need to assess the awareness of the products’ recipients of AHRQ and its role in the healthcare field. Ultimately, AHRQ expects to improve the development, design, format, and awareness of current and future versions of The Guide to Clinical Preventive Services to ensure that this publication meets the needs of the primary care physicians and other medical professionals that it seeks to serve.

This request for information is authorized by 42 U.S.C. § 285o.

 A.2. Purposes and Uses of the Data

As stated in Section A.1 above, the purpose of this evaluation is to gather information from recipients of a specified publication, the 2005 and 2006 versions of Guide to Clinical Preventive Services (Guide), in order to (1) inform future product versions or enhancements and (2) examine the extent to which, and how, they have used the guides in clinical practice. To achieve this purpose, the project seeks to meet the following five objectives:

1. To determine the extent to which the target audience accepts these guides.

2. To determine target audiences’ attitudes toward these guides.

3. To determine how the target audience uses these guides and to what extent it has improved care.

4. To learn ways to strengthen the content and format of future versions of these AHRQ guides.

5. To determine the extent to which the target audience is aware of and satisfied with AHRQ.

The evaluation will take a multi-method approach.  Methodologies for this study will include survey (telephone, mailed, web-based and/or e-mailed) and semi-structured interviews. The methods selected for this study will be determined based on several factors including: (1) the most efficient method of reaching the target audience, (2) the most cost-effective method of reaching the target audience, and (3) the best way to maximize response rates while minimizing burden to the participants. 

As the lead federal agency charged with improving the quality, safety, efficiency, and effectiveness of health care for all Americans, AHRQ’s mission is to improve health care through the production and use of evidence-based practices and to establish a broad base of scientific research that promotes improvements in clinical and health practices. Rather than make random changes to current practices, AHRQ would like to obtain sound, research-based information to determine if changes may be warranted for this guide’s development in order to better meet user needs. Evaluation results will therefore enable AHRQ to use funds allocated to guide development more effectively.

Although the evaluation is designed to provide direction for future AHRQ efforts, results from this study are expected to be of interest and use to other agencies and organizations that produce and disseminate products which contain health care information.

 A.3. Use of Information Technology To Reduce Burden

The study will take a multi-method approach.  Methodologies for this study will include survey (telephone, conference, web-based, mail and/or e-mail) and semi-structured interviews (e.g., in person, telephone). All technology used for the survey administration (e.g., Web-linked survey administered via email, Web-linked survey on a Webpage) will meet Federal requirements for Section 508 accessibility. Information technology will be used in the following ways:

1. Participants will be offered the opportunity to respond to mailed surveys electronically by gaining access to the Contractor's web site using a user name and password.  The user name and password will be provided on the paper version of the questionnaire.  Electronic responses will be downloaded directly into the response database. 

2. Telephone survey interviewers will input respondents’ responses electronically. Interviewers will access the relevant survey through the Contractor’s server by using a username and password. Interviewers will then click on the appropriate response as the respondent provides it to them. As with the other electronic survey procedures, all responses will be downloaded directly into the response database.

3. Semi-structured interviews will be conducted face-to-face or by telephone. For respondents who agree to have their interview recorded, interviewers will record answers as they are given and will then upload the answers into an Excel spreadsheet after the interview is complete.

4. Reports generated from this project may be made available to the public through AHRQ's website.

By offering respondents the opportunity to respond online to the mailed survey, burden to respondents is reduced by eliminating the time it takes to write out responses on a questionnaire. This same burden is reduced for respondents participating in telephone interviews (i.e., number 3, above). In addition, by offering respondents the option of responding on-line, the time associated with mailing the hard copy of the questionnaire back to the contractor is eliminated.

 A.4. Efforts To Identify Duplication

The survey that will be implemented will be designed to reflect the specifics of the target audience being assessed. During the development of this voluntary instrument, groups within and outside of AHRQ will be consulted. Plans to conduct surveys will be reviewed prior to implementation, and any potential duplication will be identified in the review and approval process.

 A.5. Small Business

The survey instrument, procedure for completing the instrument and semi-structured interview moderator guides will be designed to minimize burden on all respondents. Based on a preliminary review of the list of individuals who have agreed to participate, information collection for this study is not anticipated to overly burden small businesses. Thus, it is not anticipated that there will be a significant impact on small businesses or other small entities.

A.6. Consequences of Not Collecting the Information

If the information is not collected, AHRQ will be deprived of important information on which to base decisions regarding changes to their current guide development plan. Lack of information on which to base such decisions will hinder AHRQ's ability to ensure the use of the results of this research to significantly improve the existing information and assess recipient awareness.

The vast majority of respondents will be asked to participate one time. A small subset of participants (n=30) may be asked to participate in this evaluation up to two times (e.g., a survey, semi-structured interview), because measures will include a follow-up to the survey (i.e., semi-structured interview). For instance, the first time individuals are contacted, they will be asked to participate in the survey, and at that time they will be asked questions about their current knowledge of and attitudes to AHRQ’s Guide to Clinical Preventive Services.  Then, the same individual may be asked to participate in a follow-up semi-structured interview in order to further assess their knowledge and attitudes and to delve for more detail, in order to learn ways to strengthen the content and format of future versions of the AHRQ guide and determine the extent to which the target audience is aware of AHRQ and its role in the healthcare field.  Not all individuals will be contacted twice, but a small subset of group of individuals may be contacted a second time.  If individuals are asked to participate a second time, it will be to follow up on their original responses.

A.7. Special Circumstances Justifying Inconsistencies with Guidelines in 5 CFR 1320.6

This information collection fully complies with 5 CFR 1320.5(d)(2) guidelines.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. Federal Register

A notice published in the Federal Register is not required for this project since it is being submitted under AHRQ’s generic clearance 0935-0106.

2. Efforts to Consult Outside the Agency

Consultations on the design, sampling plan, instrumentation, and analysis of this study have occurred throughout the planning phase of this project. These consultations have provided, and will continue to provide the opportunity to ensure the technical quality and appropriateness of the overall project design, sampling approaches, and data analysis plans; to obtain advice and recommendations concerning the instrumentation; and to structure the study and instruments so as to minimize overall and individual response burden. Both formal and informal consultations have occurred with the following individuals in connection with this study. 

Hayashi, Susan PhD, JBS International, Inc. (Phone: 301-495-1080, ext 4588).

Kretz, Lisa PhD, JBS International, Inc. (Phone: 301-495-1080, ext. 4385).

A.9. Payments or Gifts to Respondents

Although participation in the project is voluntary, respondents are likely to perceive a time cost and burden associated with their participation.  The use of tokens of appreciation to increase response rates has been well documented by Dillman (1978, 2000).  We, therefore, intend to offer non-monetary tokens of appreciation to encourage responses and to increase response rates.  A selection of AHRQ items valued at less than $4 gifts will be identified to be used for the evaluation activities (e.g., survey participants might be offered an AHRQ decorative calculator/clock or an AHRQ pen).  As part of instrument pilot testing, we will ask respondents to comment on the type of AHRQ item that is appropriate for participating in the data collection activity.

A.10. Assurance of Confidentiality

The current project will fully comply with the Privacy Act of 1974 (5 U.S.C. Section 552a, 1998).  Respondents will be advised that surveys and semi-structured interviews are entirely voluntary and that any information they provide will be combined and summarized with information provided by others and no individually identifiable information will be released. The Privacy Act may apply to some data collection activities (e.g., some participants may require identifiable responses due to a planned follow-up). When the Privacy Act is applicable, respondents will be told:

(a) the statutory authorization for asking for the information (i.e., 42 USC 285o)

(b) the purpose for which the information is being asked

(c) whether or not responding to the request for information, in whole or in part, is voluntary

(d) the consequences, if any, of not responding

(e) the extent of confidentiality to the extent permitted by the Privacy Act

However, all respondents will be assured that their participation is voluntary, that no adverse consequences will accrue to non-respondents, and that their comments and opinions will be kept confidential to the extent permitted by the Privacy Act. In addition, cover letters will accompany all questionnaires and will indicate AHRQ’s Federal status and the purpose of the study.

In order to ensure confidentiality of that data to the extent permitted by the Privacy Act, the Contractor will use several procedures. For mailed, Web-linked and e-mailed surveys, the contractor will give each individual a unique identifying number. For mailed surveys, each person’s identifying number will be entered in the database used to generate mailing labels and track responses; the number will also be entered on each individual’s questionnaire so the Contractor can identify respondents. For e-mailed surveys, each person’s identifying number will be entered in the database to track responses. These procedures will allow the Contractor to follow up with individuals who have not responded and to increase the response rates. Upon completion of the study, the Contractor will destroy the database with individuals’ names, addresses, and identification numbers. For Web-linked surveys, the responses are anonymous and the responses will be entered into a database. For telephone surveys, the contractor will only use a unique identifying number on top of the survey. This number will be used to differentiate among survey responses in the response database, not to identify any respondents.

To ensure confidentiality, to the extent permitted by the Privacy Act, of semi-structured interview respondents, all names and any organizational affiliation which might inadvertently identify a respondent will be removed from written transcripts of the interview. Each written transcript will be given a unique identifying number that will be used for all analyses. Each respondent’s personal information (name, address, transcript identifying number) will be entered into a separate database that will be used to generate mailing labels and track responses should follow-up with that respondent be needed. Upon completion of the study, the Contractor will destroy the database with individuals’ names, addresses, and transcript identification numbers.

When respondent identifying information is needed to distribute incentives, every effort will be made to collect that information independently from any responses. For example, for mailed surveys, questionnaires will be designed so that the page containing the respondent's name, address, and incentive requests can be removed and processed separately from the questionnaire.  Incentive information for e-mailed and mailed surveys will be written directly into a separate database from that which contains the respondent's responses.

A.11. Questions of a Sensitive Nature

None of the questions will request any personally invasive or sensitive information.

A.12. Estimates of Respondent Burden Hours

Table 1 contains estimated response burden hours for the target audiences included in this study.

Estimates for response burden were calculated based on the methodology (survey data collection) being used and are based on previous experience collecting similar data. For example, for emailed and mailed surveys, burden estimates of 0.25 hours were used.  For telephone surveys, estimates of 0.50 hours were used. For semi structured interviews, estimates of 1.0 hours were used. When multiple methodologies are proposed for a given target audience, the average response burden across methodologies is presented.  For example, this study will use surveys (telephone, mailed surveys, Web-linked and/or e-mailed surveys) and semi-structured interviews.  It is estimated that 135 individuals will respond to the telephone survey at an estimated burden of .50 hours for a total burden for the telephone survey of 67.5 hours.  Similarly, it is estimated that 6610 individuals will respond to the emailed/mailed survey at an estimated burden of 0.25 hours for a total burden for the web-based survey of 1652.5 hours. Finally, it is estimated that 30 individuals will respond to the semi structured interviews at an estimated burden of 1.0 hours for a total burden of 30.0 hours. Taking the average burden across the different types of surveys for this study (1750/6745 individuals + 30 follow up interviews), gives an overall total burden estimate of 0.26 hours per individual or 1750 total burden hours for the estimated 6,745 individuals expected to participate in the project.

For the two Web-linked surveys to the AHRQ Webpages (i.e., Preventive Services and Electronic Preventive Services Selected (ePSS)), the link to the survey is expected to be placed on the Webpages for one month and the estimated burden is calculated as follows:

(a) Average visitors per month for the Preventive Services Webpage = 5, 400

(b) 16,957 is the average number of visitors who visited more than once to the ePSS (16,956 is calculated by taking an average of the number of visitors who visited more than once at 3 separate weeks ((3,880 + 4,208 + 4,630) / 3 = 4239.33 x 4 = 16,957.33) (Note: Number of visitors who visited more than once is used because it is anticipated that this group potential respondents are more familiar with the product and more likely to provide feedback to AHRQ. In addition, recipients of the AHRQ products are the primary target audience).

(a) 5,400 + (b) 16,957 = 22,357 estimated number of visitors who visit the Webpages where the survey link will be posted. It is estimated that 25% of the expected 22,357 visitors to the two Webpages will select to complete the survey: 22,357 x .25 = 5,589.25 estimated number of respondents.

Table 1:  Estimates of Annualized Response Burden Hours

Type of Survey	Number of Respondents	Number of Responses per Respondent	Average Hours per Response	Response Burden Hours
Emailed/Mailed Surveys	6,610	1	0.25	1652.5
Telephone Surveys	135	1	0.50	67.5
Web-linked Surveys	5,589	1	0.25	1397.25
Semi structured interviews	30	1	1.0	30
Totals	12,364		---	3,147.25

A.13. Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to Respondents or Record Keepers

No capital, start-up, or operational and maintenance costs are incurred by study participants in this information collection activity.

A.14. Estimates of Costs to the Federal Government

The total cost for the project is $114,097 over a one-year period.  These costs cover all aspects of survey design, testing, data collection, and analysis.

In addition, it is estimated that one full-time equivalent AHRQ staff member will spend 25% of his or her time (520 hours) to manage and administer the project.  Assuming an annual salary of $90,000, government personnel costs will be $22,500 over a one-year period.

Total project costs are thus, $136,597 over a one-year period.

A.15. Changes in Burden

This is a new project and is a new collection of information.

A.16. Plans for Publication, Analysis, and Schedule

Time Schedule

The project covers a one-year period from the beginning of May 2007 through the end of April 2008.  Clearance is requested for a one-year time period.  Data collection activities and data analysis will be conducted over this time period.  Attachment A-1 indicates when each of the activities associated with the project will occur. 

Publication Plans

Results of all studies completed under the project will be presented to AHRQ in a briefing to the agency, accompanied by a final written report and executive summary.  Upon AHRQ approval, these documents will be made available for dissemination to the public in hard copy; they will also be uploaded to the AHRQ web pages where they can be viewed directly or downloaded.  A copy of the executive summary will be sent to individual participants who expressed an interest in receiving it.

Analysis Plan

The primary purpose of this evaluation is to gather information from recipients of a specified publication, the 2005 and 2006 versions of Guide to Clinical Preventive Services (Guide), in order to (1) inform future product versions or enhancements and (2) examine the extent to which, and how, they have used the guides in clinical practice.

The analyses will be descriptive and contingent on the final sample size, some higher level analysis will be conducted. The results of these findings are primarily for internal use but may be shared with key government policy and management officials, AHRQ staff, public and private health providers, and members of the general public.

For the types of surveys described earlier, the following analyses would be appropriate:

Mail/telephone/electronic surveys: Basic descriptive analyses (including means, standard deviations, and percentages) are expected for these types of surveys. Other analytic techniques (e.g., regression, analysis of variance) may be employed dependent on the final sample size. Non-probability samples will be selected for these surveys. Specifically, convenience samples will be utilized which is useful for exploration of issues not previously examined such as these being explored in this study of the Guide to Clinical Preventive Services (Guide).

Semi-structured interviews: Participants will be selected purposively, so no generalizations to the population will be possible. Semi-structured interviews will be used to elicit more detailed information, to identify problems and issues for further study and, in some instances, “brainstorm” for possible solutions. The analysis will be qualitative and consist mostly of narrative summaries of the discussions as well as the examination of emerging themes.

 A.17. Approval to Not Display Expiration Date

The expiration date will be displayed on all instruments approved for this study.

A.18 Exceptions to Item 19 of OMB Form 83-I

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions

B-11