SUPPORTING STATEMENT
Part A
Assessing Organizational Responses to AHRQ’s Health Literacy Pharmacy Tools
Version: August 5, 2009
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 5
3. Use of Improved Information Technology 6
4. Efforts to Identify Duplication 8
5. Involvement of Small Entities 8
6. Consequences if Information Collected Less Frequently 8
7. Special Circumstances 9
8. Consultation outside the Agency 9
9. Payments/Gifts to Respondents 9
10. Assurance of Confidentiality 9
11. Questions of a Sensitive Nature 10
12. Estimates of Annualized Burden Hours and Costs 10
13. Estimates of Annualized Respondent Capital and Maintenance Costs 12
14. Estimates of Annualized Cost to the Government 12
15. Changes in Hour Burden 13
16. Time Schedule, Publication and Analysis Plans 13
17. Exemption for Display of Expiration Date 17
List of Attachments 17
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions.
According to its authorizing legislation, AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
Over the past several years, low health literacy has been identified as an important health care quality issue. In 2003, the Institute of Medicine identified health literacy as a cross-cutting area for health care quality improvement, and defined health literacy as ‘the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions’.1 According to the 2003 National Assessment of Adult Literacy, only 12 percent of adults have proficient health literacy.
Persons with limited health literacy face numerous health care challenges. They often have a poor understanding of basic medical vocabulary and health care concepts. A 1995 study of patients in a large public hospital showed that 26 percent did not understand when their next appointment was scheduled and 42 percent did not understand instructions to “take medication on an empty stomach”.2 In addition, limited health literacy leads to more medication errors, more and longer hospital stays, and a generally higher level of illness, resulting in an estimated excess cost for the US health care system of $50 billion to $73 billion per year.3
Pharmacists are the most accessible health care providers and serve as an important source of information for patients. Through a previous task order, AHRQ supported the development of the following four new health literacy tools to facilitate pharmacies’ communications with their patients who have low health literacy:
Is Our Pharmacy Meeting Patients’ Needs? A Pharmacy Health Literacy Assessment Tool User’s Guide (Jacobson et al., 2007)
Strategies to Improve Communication between Staff and Patients: Training Program for Pharmacy Staff (Kripalini & Jacobson, 2007)
How to Create a Pill Card (Jacobson et al., 2008)
Telephone Reminders: A Tool to Help Refill Medications on Time (Jacobson et al., 2008)
These tools have shown promise in institutional settings, but it is not yet known how non-institutional pharmacies will respond when the tools are widely disseminated. Through a contract with Abt Associates, AHRQ now proposes to raise awareness of these tools among a large and more diverse set of pharmacies, and to use a survey and in depth case studies to enhance understanding about the best channels for diffusion of the tools, as well as of the conditions or factors that may facilitate or impede the adoption of the tools in diverse pharmacy settings. AHRQ would use insights gained to develop materials (promotional and implementation guides) that would continue to increase awareness of the tools and assist interested pharmacies in putting the tools into practice.
As part this study, the tools will be made available to pharmacies nationwide through an AHRQ dedicated website. Announcements about the tools and the website will be e-mailed or mailed to representatives of nearly all 60,000 pharmacies nationwide through standard communication channels used by pharmacy professional associations. Through web-based data collection and a web-based survey (described in detail in section 3 below) we will gain important insights into the effectiveness of methods used to raise awareness of the tools.
The second part of this study will use a comparative case study approach to gain detailed insight into pharmacies’ experience with, and barriers to, using the pharmacy health literacy tools. We will conduct 9 case studies: 7 at sites that implement one or more of the tools and 2 at sites that are aware of the tools but choose not to implement them.
Of the 7 case study sites where at least one tool is implemented, 5 have already been recruited and have agreed to implement the Pharmacy Health Literacy Assessment Tool. These 5 pharmacies include an independent pharmacy; a 340B pharmacy; an institutional pharmacy; a grocery chain; and a traditional chain. Following our wide distribution of the four tools (further described below) we will recruit 2 additional pharmacies that attempt to implement at least one of the tools. Finally, we will recruit 2 sites that, while aware of the tools, have chosen not to implement any of them.
To ensure that information-rich cases are generated, pharmacies have been or will be selected that (1) represent different business models; (2) serve patient populations with low health literacy; (3) are located in different states, and different regions when possible; and (4) vary in their internal capacity to implement the tools. To maximize variation across the 9 case studies, we will include pharmacies from a range of pharmacy types, geographic areas, and populations served.
Case study data collection techniques will include site visits, interviews with pharmacy staff, and review of documents recommended by key informants, such as organizational charts; pharmacy operations data (number of prescriptions filled per day, staff levels, staffing, hours of operations, etc.); patient population demographic estimates; relevant policies & procedures; reports from existing QI/QA programs or efforts; and memos, minutes, and other materials related to tool implementation.
This study also supports AHRQ’s special interest in minority populations. The selected case study sites serve diverse patient populations that are more likely to have limited health literacy, including American Indian/Alaskan Native, black, and Hispanic adults; individuals in poorer health; individuals who have limited English proficiency (LEP); and
adults age 65 and older.
AHRQ would like to promote broad uptake of its four previously developed tools designed to improve the quality of the care that pharmacies deliver to individuals with limited health literacy. Research suggests that merely raising awareness of the tools is not likely to result in their widespread adoption, or in their certain success where they are implemented. The successful adoption and implementation of these tools will depend not only on the design or attributes of the tools themselves, but also on contextual factors that vary greatly across the universe of pharmacies that face common problems of serving populations with limited health literacy.
In other words, broad uptake will be more likely to occur if pharmacists have the means to assess the compatibility of the tools with their culture, goals and needs, as well as the tools’ potential benefits and the factors that can facilitate or impede implementation. Data collected through this study will provide an account of the experiences of a diverse group of pharmacies with one or more of the tools, including: the benefits they report, the relative ease of tool implementation, resources required for implementation, and factors facilitating or impeding implementation.
Data collected for this study will be used to develop implementation guides to accompany further dissemination of AHRQ’s pharmacy health literacy tools. These guides will help pharmacies understand how they might implement the tools, what obstacles they might encounter, and how they might overcome those obstacles. Ultimately, the purpose of this project is to expand the number and diversity of pharmacies within which these tools are implemented and ultimately to increase the likelihood of their broader adoption.
We make no claim that the results from this study will be generalizable in the statistical sense. Rather, this sample of information-rich cases will be illustrative and informative and will generate lessons learned regarding organizational, structural, resource, staffing, and cost factors that may affect the tools' adoption and implementation, which will help to inform the shaping and spread of these and other related tools in the future. (For a more complete discussion of “lessons learned” --“principles of practice that must be adapted to particular settings in which the principle is applied” see the case study analysis section, below). The study may also produce new theoretical insights4 which may guide or spur future research.
This study will implement the following four data collection efforts that require OMB review and approval:
1) On-site and telephone interviews with the staff at 7 pharmacies that implement at least one of the health literacy tools (see Attachment B).
2) Telephone interviews with the staff at 2 pharmacies that were aware of the tools but did not implement the health literacy tools (see Attachment C).
3) Pharmacy staff survey to be administered to the staff at the 7 pharmacies that implement the health literacy tools (see Attachment D).
4) A web-based survey of visitors to the health literacy tools' website (see Attachment E).
Study Limitations
We make no claim that the results from this study will be generalizable in the statistical sense. Rather, this sample of information-rich cases will be illustrative and informative and will generate lessons learned regarding organizational, structural, resource, staffing, and cost factors that may affect the tools' adoption and implementation, which will help to inform the shaping and spread of these and other related tools in the future. (For a more complete discussion of “lessons learned” --“principles of practice that must be adapted to particular settings in which the principle is applied” see the case study analysis section, below). The study may also produce new theoretical insights5 which may guide or spur future research.
Besides issues of generalizability related to the known selection bias of the website visitor’s survey (see discussion p 16) and the use of non-statistical methods (e.g., interviews), the study holds additional limitations.
With regard to the website visitors’ survey:
To better allow for statistical analyses and facilitate comparison among types of respondents (e.g., pharmacists working in independent pharmacies versus chain pharmacies) we have designed a website visitors survey that utilizes largely closed-ended questions. A resulting limitation is we are potentially forcing respondents to provide simplistic answers to complex issues. However, we have aimed to mitigate this limitation by included an open-ended response option (i.e. Other [please specify]) for many of the survey questions.
Respondents at the time of taking the survey may not have had sufficient time to download and/or review the tools prior to completing the survey. This may result in a potentially large number of respondents citing as their main reason for not using a tool their not having had a chance to do so.
We were not able to pretest the survey instrument and therefore were not able to benefit from feedback from pharmacists in the field. However, we had pharmacist consultant review the survey and provide feedback from a pharmacist’s perspective and a survey methodologist with extensive experience with survey design and cognitive testing of surveys provide feedback on the design, questions, and response categories.
With regard to the qualitative interviews:
A potential limitation of any interview data is the possibility for distorted responses due to “personal bias, anger, anxiety, politics, and simple lack of awareness” (Patton, 2002). To help mitigate this risk, we are utilizing experienced interviewers and qualitative-researchers to conduct the interviews. However, in this study the respondents are likely to be motivated pharmacy professionals, given their willingness to participate in the study, and we therefore anticipate few distorted responses. Also, we anticipate little to no anger or politics on this topic. Finally, to help mitigate this risk, we are utilizing experienced interviewers and qualitative-researchers to conduct the interviews.
A potential limitation of qualitative research is the difficulty with assessing, maintaining, and demonstrating rigor. To that end, we have employed accepted qualitative design techniques to ensure high quality, credible qualitative analyses. Specifically, we have designed the study and all data collection efforts and analyses using Rogers (1995) Diffusion of Innovations theory to address the aims of the study. Additionally, we strengthen the credibility of qualitative research by triangulation: we have designed the interviews to collect interview data from multiple perspectives (i.e., different pharmacy staff) to help identify consistencies and inconsistencies. A second form of triangulation arises from the understanding that is gleaned by being guided by a well-founded theory while the collecting data using multiple methods and sources.
One final limitation emerges from the fact the tools have not been validated. Thus, just as we note that the effectiveness of the tools will in part depend on how well they are implemented, the converse is equally true: there may be aspects of the tools themselves that may hinder their successful adoption/implementation. The data collection instrument includes a few question that may begin to get at this issue, but this study is not designed to investigate it and does not purport to do so.
Part 1: Web-based data collection
The project’s website tracking system will be used to track and monitor the following four domains: Site Usage, Visitor Usage, Traffic Usage and Content Usage.
Site Usage
Site usage data will indicate how popular the site and its related pages are to the internet community by measuring: the number of users visiting the site (“visits”), the average amount of time users spent on the site and its related pages (“average time on site”), how many pages they linked to within the site during their visit (“pages per visit”), and what pages were most frequented (“page views”) will be tracked. In addition, the percentage of new users visiting the site (“percent new visits”) and the rate at which new users leave the main site without linking to other pages within the site (“bounce rate”) will be determined. This information will be used to help determine how appropriate the content and related topics captured on the site’s respective pages are to our target population.
Visitor Usage
Visitor usage data provides information on the users who visit the website, like the number of unique visitors.
Traffic Usage
Traffic usage data provides information on each of the ways visitors can access a webpage: direct access, site referral or through a search engine. For each visit to the website, the traffic sources will be tracked and recorded and the percentage of associated with each means of access direct traffic will be calculated. Top traffic sources and keywords used in search engine searches will also be determined to aid in dissemination efforts.
Content
Usage
Content usage data will be collected to determine which pages on the website are most popular within our user community. Page views, percent page views and bounce rate will be determined for each page on the website. The number of times a new user views a specific page (“unique views”) will also be collected in this report.
Web-based survey
In addition to the automated data collection described above, we will use a website visitor survey to collect information about respondents’ perceptions of the health literacy tools and distribution mechanisms. Conducting the survey via the web will reduce respondent burden compared with telephone or mail surveys. All visitors to the website will be invited to complete the survey. The aims of the survey are:
To identify the distribution mechanism(s) by which the respondent became aware of the tools and the website;
To understand respondents’ attitudes and beliefs about health literacy prior to learning about the tools;
To assess respondents’ intention to use the tools;
To understand what factors led to the decision to use or not use the tools.
The database will be kept in a shared, write-protected folder on a secure drive at Abt Associates.
As described above, we will use a comparative case study approach with nine study sites. This approach involves conducting case studies of nine pharmacies, assessing their organizations’ responses to AHRQ’s pharmacy health literacy tools, and then using the information to conduct a cross-site analysis, comparing the nine sites’ experiences and the factors that facilitated or hindered their adoption and implementation of these tools.
In its comprehensive 2004 review of the peer-reviewed literature on health literacy,6 the Institute of Medicine inventoried a large body of research documenting the problem of health literacy, but little research identified tools to address this problem. The IOM report did mention two instruments that may be used to assess patient health literacy, the REALM and TOHFLAs. However, no instruments have been developed to facilitate the assessment of pharmacies’ or other health care facilities’ responsiveness to patients with low health literacy. A PubMed search using the terms “health literacy” and “pharmacy” produced similar results. Of the 34 articles retrieved, most described the relationship between health literacy and health outcomes. Some articles described a limited set of methods to help specific clinical settings address patient literacy, such as focus groups7 and patient-centered design of pharmacy labels.8 However, none of the literature reported on implementation of tools such as those included in the proposed study.
Many pharmacies are small businesses. Automatic web-based data collection, and the web-based survey described above, will minimize the burden on these entities. In depth interviews will only be conducted with staff at a small number of pharmacies (9). A two-day site visit will be conducted only with the 7 pharmacies that have begun to implement one or more of the tools. Participation for all parties involved is voluntary.
6. Consequences if Information Collected Less Frequently
AHRQ is applying for a one-time data collection effort. This data collection effort is necessary to assess and improve the performance of AHRQ’s pharmacy quality improvement tools in general, and the quality of AHRQ’s pharmacy health literacy tools in particular. Not conducting the data collection could result in slower adoption of AHRQ’s health literacy tools, and therefore slower adaptation of pharmacies to their clients with low health literacy. This would leave AHRQ’s priority populations at risk for dangerous medication errors.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on January 29th, 2009 for 60 days (see Attachment F). No comments were received.
No honoraria or incentives will be offered to web survey participants or key informants in each pharmacy. However, we are offering non-financial technical assistance to the participating pharmacies through the Pharmacy Health Literacy website. The website will contain information on pharmacy health literacy and quality improvement topics and will host an online version of the pharmacy staff survey that is part of the health literacy assessment tool.
Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.
Individuals and organizations contacted will be further assured of the confidentiality of their replies under 42 U.S.C. 1306, and 20 CFR 401 and 4225 U.S.C.552a (Privacy Act of 1974). In instances where respondent identity is needed, the information collection will fully comply with all respects of the Privacy Act.
Respondents will be
informed in the introduction to web survey, telephone interviews and
in-person interviews that their answers will be kept strictly
confidential.
Participation will be entirely voluntary, and the study will conform to the requirements of the Privacy Act by omitting individuals’ names, addresses, telephone numbers and other personal identifiers in the final data file.
The firm that will conduct the data collection, Abt Associates, has conducted numerous projects and surveys involving sensitive information; consequently, facilities and procedures have been developed to maintain respondent confidentiality. All staff assigned to Abt Associates projects sign confidentiality agreements specifying that no identification of respondents or their answers will be revealed to other persons who are not specifically involved with this project as an employee. All databases will be password protected, with only the data administrators having write authority over files. If electronic data transfer is necessary, the data will be transferred via diskette or CD-ROM to clients in an encrypted and password-protected format before shipping via a bonded courier.
Abt Associates also emphasizes the importance of protecting the data while it is stored in their facilities. Abt Associates frequently maintains and manages large datasets, which frequently include highly sensitive information. In over a decade of conducting surveys on sensitive topics, Abt Associates has never suffered a breach of any respondent's privacy.
The web survey and case study interview protocols do not contain any questions concerning sexual behavior and attitudes, or religious beliefs.
Case study interviews may, however, elicit sensitive, proprietary business information. Additionally, staff interviewees could potentially provide information that reflects negatively on their employer, supervisor or co-workers. Respondents to the survey will be explicitly informed that their participation is voluntary, information they provide is confidential, and they may choose to withdraw from the study or not respond to specific items without penalty. We will also remove respondent and pharmacy names from written interview records to maintain respondent confidentiality.
Case Studies
Through
its contractor, AHRQ proposes to conduct 7 in-depth case studies to
assess pharmacies’ experiences with implementation of one or
more of these four health literacy tools, using interviews, site
visits, review of documents and a survey of pharmacy staff from case
study pharmacies. In addition, AHRQ will conduct 2 more limited
studies of pharmacies that were aware of the tools but chose not to
implement them.
A 1-day site visit will be conducted with each of the 7 sites that implemented at least one of the tools. Each site visit will include a walk-through of the pharmacy site to see the physical layout, an interview with the key informant or contact person, and interviews with up to four additional pharmacy employees, including the pharmacy manager, staff pharmacist, pharmacy technician, and pharmacy clerk.
Therefore, up to 35 interviews will be completed across the 7 sites that implement at least one of the tools. In addition, up to 12 pharmacy staff at each of the 7 implementation sites will complete the tool’s Pharmacy Staff Survey contained in the Pharmacy Health Literacy Assessment Tool.
For each of the two pharmacies that do not implement the tools, more limited interviews will be conducted with up to 2 informants per site.
Website Visitors' Survey
For pharmacists and other visitors to the AHRQ website, we will conduct a voluntary survey regarding health literacy in general, and feedback regarding AHRQ's health literacy tools. The website visitors’ survey will be available on-line.
ESTIMATED ANNUAL RESPONDENT BURDEN:
Exhibit 1 shows the estimated annualized burden hours for the respondents' time to participate in the case studies. The staff interview at the implementing sites will be completed with up to 5 total pharmacy staff members from each of the 7 pharmacies that implement at least one of the health literacy tools. These interviews are estimated to last up to 2 hours. Staff interviews at the two non-implementation sites will be completed with up to 2 individuals per pharmacy; these interviews are estimated to last 30 minutes. The pharmacy staff survey will be completed by up to 12 staff from the 7 implementation pharmacies and is estimated to take approximately 20 minutes. Lastly, we estimate that the web site visitor’s survey will be completed by about 150 respondents and is estimated to take up to 12 minutes to complete. The revised total burden hours for all data collections is estimated to be 130 hours .
Exhibit 2 provides the revised estimated annualized cost burden for the respondents' time to provide the requested data is $3,944.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of sites/ respondent |
Number of responses per site/ respondent |
Hours per response |
Total burden hours |
Staff interview – implementing sites |
7 |
5 |
2 |
70 |
Staff interview – non-implementing sites |
2 |
2 |
30/60 |
2 |
Pharmacy staff survey |
7 |
12 |
20/60 |
28 |
Web site visitors survey |
150 |
1 |
12/60 |
30 |
TOTAL |
166 |
na |
na |
130 |
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of sites/ respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Staff interview – implementing sites |
7 |
70 |
$30.33 |
$2,124 |
Staff interview – non-implementing sites |
2 |
2 |
$30.33 |
$61 |
Pharmacy staff survey |
7 |
28 |
$30.33 |
$849 |
Web site visitors survey |
150 |
30 |
$30.33 |
$910 |
TOTAL |
166 |
95 |
na |
$3,944 |
*The average hourly wage rate of $30.33 was calculated based on the following mean hourly wage rates: pharmacists - $47.58; pharmacy manager [medical & health services manager category] - $50.34; pharmacy technicians - $13.25; and pharmacy aides - $10.15. The mean hourly wage rates for these occupations were obtained from the Bureau of Labor & Statistics on “Occupational Employment and Wages, May 2007,” found at: http://www.bls.gov/OES/current/oes291051.htm.
Case Studies
Through
its contractor, AHRQ proposes to conduct 7 in-depth case studies to
assess pharmacies’ experiences with implementation of one or
more of these four health literacy tools, using interviews, site
visits, review of documents and a survey of pharmacy staff from case
study pharmacies. In addition, AHRQ will conduct 2 more limited
studies of pharmacies that were aware of the tools but chose not to
implement them.
A 1-day site visit will be conducted with each of the 7 sites that implemented at least one of the tools. Each site visit will include a walk-through of the pharmacy site to see the physical layout, an interview with the key informant or contact person, and interviews with up to four additional pharmacy employees, including the pharmacy manager, staff pharmacist, pharmacy technician, and pharmacy clerk.
Therefore, up to 35 interviews will be completed across the 7 sites that implement at least one of the tools. In addition, up to 12 pharmacy staff at each of the 7 implementation sites will complete the tool’s Pharmacy Staff Survey contained in the Pharmacy Health Literacy Assessment Tool.
For each of the two pharmacies that do not implement the tools, more limited interviews will be conducted with up to 2 informants per site.
Website Visitors' Survey
For pharmacists and other visitors to the AHRQ website, we will conduct a voluntary survey regarding health literacy in general, and feedback regarding AHRQ's health literacy tools. The website visitors’ survey will be available on-line.
ESTIMATED ANNUAL RESPONDENT BURDEN:
Exhibit 1 shows the estimated annualized burden hours for the respondents' time to participate in the case studies. The staff interview at the implementing sites will be completed with up to 5 total pharmacy staff members from each of the 7 pharmacies that implement at least one of the health literacy tools. Staff interviews at the two non-implementation sites will be completed with up to 2 individuals per pharmacy. The interviews are estimated to last 1 hour for each of the 5 staff interviews at the 7 implementing sites and 30 minutes for each of the 2 staff interviews at the 2 non-implementing sites. The pharmacy staff survey will be completed by up to 12 staff from the 7 implementation pharmacies and is estimated to take approximately 20 minutes. Lastly, we estimate that the web site visitor’s survey will be completed by about 150 respondents and is estimated to take up to 12 minutes to complete. The total burden hours for all data collections is estimated to be 95 hours.
Exhibit 2 shows the estimated annualized cost burden for the respondents' time to provide the requested data. The estimated total cost burden is $2,882.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of sites/ respondents |
Number of responses per site/ respondent |
Hours per response |
Total burden hours |
Staff interview – implementing sites |
7 |
5 |
1 |
35 |
Staff interview – non-implementing sites |
2 |
2 |
30/60 |
2 |
Pharmacy staff survey |
7 |
12 |
20/60 |
28 |
Web site visitors survey |
150 |
1 |
12/60 |
30 |
TOTAL |
166 |
na |
na |
95 |
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of sites/ respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Staff interview – implementing sites |
7 |
35 |
$30.33 |
$1,062 |
Staff interview – non-implementing sites |
2 |
2 |
$30.33 |
$61 |
Pharmacy staff survey |
7 |
28 |
$30.33 |
$849 |
Web site visitors survey |
150 |
30 |
$30.33 |
$910 |
TOTAL |
166 |
95 |
na |
$2,882 |
*The average hourly wage rate of $30.33 was calculated based on the following mean hourly wage rates: pharmacists - $47.58; pharmacy manager [medical & health services manager category] - $50.34; pharmacy technicians - $13.25; and pharmacy aides - $10.15. The mean hourly wage rates for these occupations were obtained from the Bureau of Labor & Statistics on “Occupational Employment and Wages, May 2007,” found at: http://www.bls.gov/OES/current/oes291051.htm.
13. Estimates of Annualized Respondent Capital and Maintenance Costs
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
The total cost of this contract to the government
is $400,000. The project extends over three fiscal years. Exhibit 3
shows a breakdown of the total cost as well as the annualized cost.
Exhibit 3.
Cost Component |
Total Cost |
Annualized Cost |
Project Development |
$54,822 |
$18,274 |
Data Collection Activities |
$111,509 |
$37,170 |
Data Processing and Analysis |
$129,089 |
$43,030 |
Publication of Results |
$63,736 |
$21,245 |
Project Management |
$40,845 |
$13,615 |
TOTAL |
$400,000 |
$ 133,333 |
Exhibit 4 Project Timeline
|
|
Description (in chronological order) |
Due Date
|
Finalize health literacy tool distribution plan |
June 2009 |
Distribute tools |
Dec. 2008 - Oct. 2009 |
Complete distribution report |
Nov. 2009 |
Conduct case study research |
Sep. 2009 - Mar. 2010 |
Complete case study report |
May 2010 |
Develop implementation guides |
Aug. - Sep. 2010 |
Disseminate implementation guides |
Sep. - Nov. 2010 |
Submit manuscript to peer-reviewed journal |
Dec. 2010 |
Submit manuscript to trade journal |
Dec. 2010 |
Complete final report |
Jan. 2011 |
Publication plan:
Study results will be disseminated through peer-reviewed publications, professional presentations, and AHRQ’s website. Our manuscripts and presentations will clearly state the limitations of the study findings including the lack of generalizability of the specific results associated with the research methods.
We will submit at least one article in a peer-reviewed journal and one in a pharmacy trade journal. Relevant peer-reviewed pharmacy journals include the Journal of the American Pharmacists Association, American Journal of Health-Systems Pharmacists, Journal of Managed Care Pharmacy, American Journal of Pharmacy Education Relevant pharmacy trade journals include: U.S. Pharmacist, Pharmacy Today, Drug Topics and America’s Pharmacist.
Additionally, we plan to submit for presentations at pharmacy conferences (or other relevant conferences), and AHRQ’s Annual Conference.
We will produce promotional guides based on the study results, and we will submit an innovation profile for study to AHRQ’s Health Care Innovations Exchange.
Analysis plan
Web survey analysis
We estimate that nearly 60,000 pharmacies in the U.S. will receive announcements from pharmacy associations about AHRQ’s health literacy tools and pharmacy health literacy website. We will use statistical tools to summarize and analyze the data for notable patterns related to following survey aims:
To identify the distribution mechanism(s) by which the respondent became aware of the tools and the website;
To understand respondents’ attitudes and beliefs about health literacy prior to learning about the tools;
To assess respondents’ intention to use the tools;
To understand what factors led to the decision to use or not use the tools.
Descriptive statistics will be calculated for all survey items. For the responses to our categorical survey questions, we will use statistical tools to summarize and analyze the data for notable patterns. Survey responses will be aggregated and frequency distributions for each survey item will be compiled. Cross-tabulations will be prepared that display any differences on the survey items among the groups, for example, the difference in item response between pharmacists working for an independent pharmacy versus a chain pharmacy.
Responses to two open-ended questions will be coded using content analytic techniques. This will require coding the text into manageable categories on a variety of levels – word, word sense, phrase, sentence, or theme – and then examining the coded text using one of the basic methods of content analysis (i.e. conceptual analysis or relational analysis). We will devise a list of discrete codes representing the various open-ended question responses to further facilitate analysis.
It is difficult to estimate the number of individuals who will visit the pharmacy health literacy website, and complete the survey. We expect it to be difficult to achieve a high number of completions for this voluntary, uncompensated survey.
A key limitation for the survey is the possibility of selection bias. Respondents decide whether to participate or not in the survey, and respondents’ decisions to participate in the survey may be correlated with traits that are important to the study (e.g., stage of innovation, perceived need to address health literacy, perceived benefits of the tools). Additionally, individuals that visit the website and are willing to complete the survey may be different from the pharmacist population. However, from this survey we expect to learn about pharmacists’ perceived need to address health literacy and their impressions of the perceived benefits of the health literacy tools. We may also expect to learn more about what pharmacy characteristics are correlated with a perceived need to address health literacy and the perceived benefits of the tools.
Given the potential for low response rates and selection bias and the resulting need to reduce the risk of over interpreting the survey results, response rates will be disclosed as well.
Case study analysis
While the small sample size will not allow confident empirical generalization from the sample to the larger population, in-depth analysis of a small number of cases can produce important insights that are particularly useful in addressing “how” and “why” questions.
The three aims of this study are to:
increase knowledge and understanding of why the distribution of the AHRQ health literacy pharmacy tools did or didn’t trigger their adoption and/or the adoption of other health literacy quality improvement activities in pharmacies
increase knowledge and understanding of the factors affecting pharmacies’ experiences with implementation of a quality improvement tool and the effects of the tool on the pharmacies’ quality improvement activities or goals
increase knowledge and understanding of how future pharmacy quality improvement tools might be designed and disseminated to increase their visibility, adoption and impact
The data collection approach, guided by Yin (2003) and Stake (1995), will facilitate individual case and cross-case analysis and comparison by collecting comparable information from each case study pharmacy as directed by the protocols (see Attachment B). The theoretical framework, informed largely by Rogers’ Diffusion of Innovations, will guide the analysis for each pharmacy case as well as the cross-case analysis and comparison.
Additionally, Kurt Lewin’s Force Field Analysis (1951) model, will further inform the analysis. Lewin’s model may help us understand what pharmacy facilitators must be enhanced and what pharmacy barriers must be alleviated to encourage adoption of pharmacy QI tools in the future. Moreover, Lewin’s theory underscores the need to remove barriers as a first step, so understanding the barriers to pharmacy with this QI tool could inform AHRQ’s future pharmacy QI efforts.
The challenge with cross-case analysis is reconciling “an individual case’s uniqueness with the need for more general understanding of generic processes that occur across cases.”9 One way to reconcile the unique and generic aspects of each case is to offer a rich case study narrative overlaid with key themes addressing each variable or factor of interest. Therefore, for each pharmacy, we will provide a narrative that offers a rich contextual understanding of the reasons for tool adoption and implementation (or not).Error: Reference source not found Additionally, we will organize case-specific information along the following factors/variables.
Pharmacy’s culture, workforce, work routines, and setting
Pharmacy’s organizational and decision-making structure
Perceived benefits and risks of using the health literacy tools
Communication procedures and information systems
Interplay with existing quality improvement/assurance programs
The pharmacy’s market (patient demographics) and regulatory environment
Pharmacy supports and resources available for health literacy
Patient population served
Change champion (or individual who championed the tool being adopted and used)
Stage in the Innovation-Decision process
Additional facilitators and barriers such as resource constraints, management support, competing QI initiatives.
Impact of tool adoption, including planned and unplanned, intended and unintended consequences of the use of the health literacy tools.
The analysis will involve a comparison of the nine sites’ experiences, highlighting which variables ostensibly facilitate or hinder tool adoption and implementation across sites, which factors have a variable effect depending on the pharmacy-specific context, and which unanticipated factors played a key role. The cross-case analysis is a comparative method that allows for “analysis of multiple cases, using key variables, preserving their configuration case by case”10 and will involve a synthesis of a case-oriented and variable-oriented approach to understand case dynamics and see the effect of key variables.4 As Noblit and Hare (1983) suggested, the theory informing the research can also be used to preserve the uniqueness of a case while allowing for cross-site comparisons.11
Given that one of the key research questions for this project is to understand how pharmacy quality improvement tools might be designed and disseminated to increase their uptake, we will focus on identifying lessons learned from the case study pharmacies. Lessons learned are not traditional empirical generalizations, but “principles of practice that must be adapted to particular settings in which the principle is applied.”5 We will use the cross-site analysis to develop high-quality lessons learned. High-quality lessons learned consist of knowledge that is transferable or its relevance can be extrapolated and applied to future action,5 like future pharmacy QI tool design and dissemination. Questions we will ask in developing high-quality lessons learned include: What is the evidence supporting each lesson? What are the contextual boundaries around the lesson? Is the lesson specific, substantive, and meaningful enough to guide practice in some concrete way?5
The following are topics for which we might expect to provide AHRQ with principles or lessons learned that could be informative for future efforts:
What pharmacy characteristics (e.g., high prescription volume) affected the case study pharmacies’ ability to implement the assessment tool and what might that mean for pharmacies with similar characteristics.
The resources needed to implement each of the tools
What may be the unique benefits and challenges implementing health literacy tools specific to each pharmacy type
Which tools may require outside resources to implement
Which tools were readily adopted and implemented by pharmacies
Which staff, at a minimum, may be needed to adopt and implement pharmacy health literacy tools
Were the perceived benefits or value propositions of the tools clear for pharmacies to consider adopting the tool
What are the success stories
What are the costs and benefits to a pharmacy of completing each of the phases of the pharmacy assessment tool (e.g., patient focus groups)
How can the tools be modified and the implementation be conducted to reduce costs and increase benefits to pharmacies.
An important limitation of any case study is that it forgoes breadth of experience for depth of understanding. However, as was stated earlier, while our findings will not be generalizable in a statistical sense, we hope to generate transferable knowledge about the barriers and facilitators to implementing the tools among a diverse group of pharmacies and to use this knowledge to shape dissemination activities to support their spread.
AHRQ does not seek this exemption.
List of Attachments:
Attachment A: Healthcare Research and Quality Act of 1999
Attachment B: Staff interview protocol – implementing sites
Attachment C: Staff interview protocol – non-implementing sites
Attachment D: Pharmacy staff survey
Attachment E: Web site visitor's survey
Attachment F: 60 Day Federal Register Notice
1 Institute of Medicine. 2004. Health Literacy: A Prescription to End Confusion. Washington, DC: National Academies Press.
2 Williams, M. et al., “Inadequate Functional Health Literacy among Patients at Two Public Hospitals,” Journal of the American Medical Association. 274, no. 21 (1995): 1677–1682.
3 Friedland, R.B. 1988. Understanding Health Literacy: New Estimates of the Costs of Inadequate Health Literacy. Washington, DC: National Academy on an Aging Society.
4 Patton, MQ. 2002. Qualitative Research and Evaluation Methods, 3rd ed. Thousand Oaks, CA: Sage Publications.
5 Patton, MQ. 2002. Qualitative Research and Evaluation Methods, 3rd ed. Thousand Oaks, CA: Sage Publications.
6 Bohlman LN, Panzer AM, Kindig DA, Eds.. 2004. Health Literacy: A Prescription to End confusion. Committee on Health Literacy, Institute of Medicine. Washington, DC: National Academies Press.
7 Huston SA, Hobson EH Using focus groups to inform pharmacy research. Res Soc Adm Pharm, 4(3): 186-205.
8 Wolf MS, Davis TC, Bernadella P, Clayman ML, Parker RM, Adler D, Wolf MS. 2008. A patient-centered approach for improving prescription drug warning labels. Patient Educ Couns. 72 (3): 443-9.
9 Miles M.B. & Huberman A.M. (1994). Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks, CA: SAGE Publications.
10 Ragin C.C. (1987). The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley: University of California Press.
11 Noblit G.W. & Hare R.D. (1983, April). Meta-ethnography: Synthesizing Qualitative Studies (Qualitative Research Methods Series, Vol. 11). Newbury Park, CA: SAGE Publications.
| File Type | application/msword |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| Last Modified By | DHHS |
| File Modified | 2009-08-05 |
| File Created | 2009-08-05 |