PRETESTING OF NIAID’S HIV VACCINE RESEARCH EDUCATION INITIATIVE COMMUNICATION MESSAGES
REQUEST FOR OMB REVIEW
AND SUPPORTING STATEMENT
Project Officer:
Katharine Kripke, Ph.D.
Assistant Director, Vaccine Research Program
Division of AIDS, NIAID, NIH, DHHS
6700 B Rockledge Drive, Rm 5144
Bethesda, MD 20892
Telephone: 301-594-2512
Fax: 301-402-3684
Supporting Statement Section A
Table of Contents
A. Justification 1
A.1. Circumstances Making the Collection of Information Necessary 1
A.2. Purpose and Use of the Information 4
A.3. Use of Information Technology and Burden Reduction 7
A.4. Efforts to Identify Duplication and Use of Similar Information 8
A.5. Impact on Small Business or Other Small Entities 8
A.6. Consequence of Collecting the Information Less Frequently 9
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 9
A.8. Comments in Response to the Federal Register Notice and Efforts to
Consult Outside Agency 9
A.9. Explanation of Any Incentive or Gift to Respondents 10
A.10. Assurance of Confidentiality Provided to Respondents 11
A.11. Justification for Sensitive Questions 12
A.12. Estimates of Hour Burden Including Annualized Hourly Costs 13
A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers 15
A.14. Annualized Cost to the Federal Government 16
A.15. Explanation for Program Changes or Adjustments 16
A.16. Plans for Tabulation and Publication and Project Time Schedule 16
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 18
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions 18
SUPPORTING STATEMENT
A. JUSTIFICATION
A.1. Circumstances Making the Collection of Information Necessary
The National Institute of Allergy and Infectious Diseases (NIAID) supports basic and applied research to prevent, diagnose, and treat infectious and immune-mediated illnesses, including illness from human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS). NIAID’s activities are authorized under 42 USC 285f, wherein is stated, “The general purpose of the National Institute of Allergy and Infectious Diseases is the conduct and support of research, training, health information dissemination, and other programs with respect to allergic and immunologic diseases and disorders and infectious diseases, including tropical diseases.” Developing a vaccine that protects against HIV infection is one of the highest priorities of the NIAID HIV/AIDS research program. NIAID has undertaken a range of activities to support and facilitate HIV vaccine research. For example, in 2000, the HIV Vaccine Trials Network (HVTN) was initiated to build the capacity of HIV vaccine researchers and trial sites in the United States and abroad. Given the daunting complexity of the HIV virus, developing a safe and effective vaccine ultimately will require tens of thousands of HIV-negative volunteer clinical trials participants. Minority participation in HIV vaccine clinical trials is essential; nearly two-thirds of people diagnosed with HIV in the United States are African American or Latino. Historically, recruitment of racial/ethnic minorities has been a critical challenge for medical researchers; initiatives to increase recruitment of these groups into cancer and chronic disease trials have been only partially successful.
To address the need for HIV vaccine clinical trials volunteers and therefore allow NIAID to fulfill its Congressional mandate to prevent infectious disease, NIAID created the National HIV Vaccine Research Education Initiative (NHVREI). The goal of the NHVREI is to create support for current and future HIV vaccine trials among populations most affected and infected by HIV/AIDS, specifically Blacks/African Americans, Hispanics/Latinos, men who have sex with men (MSM) of all racial/ethnic groups, and among key influencers of these populations. Program objectives will include (1) Increasing awareness of the need for an HIV vaccine in communities most affected and infected by HIV/AIDS, (2) Improving the public’s knowledge of and attitudes toward HIV vaccine research, (3) Enhancing the partnership between community and HIV clinical trial researchers, and (4) Creating support for current and future HIV vaccine trials and foster an environment that supports clinical trial volunteers.
As part of this effort, NIAID will design, develop and disseminate a wide variety of messages for diverse audiences and a wide range of materials that will include print (for example, brochures, posters, fact sheets, information kits), radio, TV and internet/Web resources.
These materials will be disseminated by NIAID through the various NHVREI program activities (local and national partnerships, trainings, conferences, symposiums) as well as by NIAID-funded partners and other governmental and nongovernmental organizations.
Communicating about any clinical trials research is complex. Communicating about HIV vaccine research to audiences with cultural and historic barriers to participating in such research requires in-depth understanding of the context in which messages and materials are received and what response(s) they evoke. NIAID must be able to empirically determine that its messages and materials have the potential to be understood and useful to the intended audiences in order to carry out its mandate effectively and adequately serve its partners, constituencies, and the affected populations. Additionally, the program activities that are designed increase support for HIV vaccine research among the intended audiences must also be acceptable if the strategies are to be effective.
To ensure that the strategies have the potential to be received, understood, and accepted by those for whom they are intended, NIAID plans to employ formative evaluation. For the NHVREI, this type of evaluation involves (1) assessing audience knowledge, attitudes, behaviors and other characteristics for the planning/development of health messages, education products, communication strategies, and public information programs; and (2) pretesting these health messages, products, strategies, and program components while they are in developmental form to assess audience comprehension, reactions, and perceptions. The information obtained from audience research and pretesting results in more effective messages, materials, and dissemination strategies. By maximizing the effectiveness of these messages and strategies for reaching targeted audiences, the frequency with which publications, products, and programs need to be modified is reduced.
Formative (pretest) evaluation is an activity that NIAID plans to perform on almost all NHVREI print pieces, broadcast products, Internet/Web resources, and informational messages to maximize their usefulness. Such evaluations are conducted on a small scale and focus on potential effectiveness with specific target audiences.
NIAID is requesting generic clearance for a range of formative evaluation data collection procedures to support implementation of the NIAID HIV Vaccine Education Research Initiative (NHVREI). Approval is requested for up to 18 pretests annually using methods described in Section B with respondents from audiences targeted by messages to be developed by NIAID. The total number of respondent burden hours will not exceed 1,230 annually.
A.2. Purpose and Use of the Information
Formative evaluation is critical to developing messages, materials and program strategies that are both effective at communicating their message and invoking the desired audience response and efficient at doing so for the least cost to the Government.
The formative evaluation process is used to determine whether a draft message, message, or program concept is effective in reaching and communicating with its audience. Pretesting involves presentation of draft messages, materials, and program concepts designed to convey specific information to a sample of the audience for whom the strategies are intended. These respondents are asked to give their reaction to the messages through either individual or group interviews. Messages, materials, and program concepts are assessed for their:
Attention - Do the concepts attract and/or hold the audience's attention? For example, if they were to see a particular brochure in their doctor's office, would they pick it up and look at it?
Comprehension - Are the messages or main points clearly understood? Does the main theme of the message get across to the audience? Is the language clear? For example, a respondent may be asked whether a fact sheet clearly explains the process of vaccine research and benefits and risks of participation in it.
Personal Relevance and Self-efficacy - Do members of the target audience perceive the message as personally relevant? For example, do the respondents think a website applies to them—that the information is important to them? Do the respondents see themselves as capable of acting on the message or participating in the program?
Believability - Is the message and/or its source perceived as credible? For example, does the respondent believe the message in an article that one cannot contract HIV from a candidate vaccine? Is the person delivering the message credible?
Acceptability - Is there anything in the message that may be offensive or unacceptable to the target audience? For example, does the respondent react negatively to a print message that attempts to correct misperceptions regarding the current availability of an HIV vaccine? Is the piece culturally sensitive?
Accessibility - Will the message reach the target audience? For example, will the planned dissemination of a PSA or brochure reach target audience members in the course of their daily life?
Usability - How likely is the respondent to use the information in the format provided? For example, is the content provided on a web site presented in a well-organized, logical, and user-friendly way?
Behavioral Intent - Do respondents think they will take action as a result of seeing/hearing the message? For example, does an educational piece containing information on how to learn more about HIV vaccine research in one’s local area motivate target audience members to click on a website link or call for information?
Respondents' input and reactions to each of these areas provide insight into how the audiences for these messages and activities may react and how the concepts should be formulated, revised, or disseminated to communicate most effectively.
Other information gathered on respondents such as gender, age, socioeconomic level, race/ethnicity, and relationship status provides a basis for evaluating whether the messages may be perceived differently by different segments of the audience. For example, selected racial/ethnic groups may find a particular brochure or message on the process and safeguards of clinical trials vaccine research more relevant than other age groups.
Systematic formative evaluation has been widely adopted by health education program planners as an integral step in the development and targeted dissemination of messages and materials. Through pretesting NIAID will be able to:
Understand characteristics of the target audience—its attitudes, beliefs, and behaviors—and use these in the development of effective communications tools;
Design messages and select formats that have the greatest potential to influence the target audience’s attitudes and behavior in a favorable way;
Determine the best promotion and distribution channels to reach the target audience with appropriate messages; and
Expend limited program resource dollars wisely and effectively.
Without clearance to conduct formative evaluation, NIAID will be unable to assess the NHVREI’s messages, materials, and program activities prior to implementation, which may result in strategies that at best – fail to achieve the desired target audience response, and at worst – unintentionally lead the audience to draw erroneous conclusions regarding HIV vaccines, clinical trials, clinical trial participants, or the safety of various behaviors.
A.3. Use of Information Technology and Burden Reduction
The information will be collected through the use of one-on-one interviews, group interviews, or self-administered questionnaires, depending upon the target audience being questioned and the messages, materials, products or activities being addressed. Improved technology in the collection and processing of data will be used to reduce respondent burden and make processing maximally efficient. For example, telephone focus groups will be convened when geographic diversity or the relative anonymity of a phone call is important and a face-to-face setting is not necessary to accomplish evaluation objectives. When telephone interviews are used, computer-assisted telephone interviewing (CATI) will be employed whenever possible. For self-administered questionnaires, closed-ended questions (for example, multiple-choice items or Likert scales), machine-readable answer sheets, and computer-administered questionnaires will be used when feasible. Transmission of data collection instruments and responses by electronic means will be utilized as appropriate.
As computer technology has continued to improve and become more widespread, opportunities to pretest messages using computers and the Internet have increased. Using computer-assisted information technology to transmit data collection instruments and/or collect responses will continue to reduce the burden on respondents; for example, respondents can access and respond to data collection requests at a time and place that is convenient to them, eliminating the need to travel for in-person or group interviews. Wherever possible, NIAID will make use of Web- or computer-based data collection methods.
A.4. Efforts to Identify Duplication and Use of Similar Information
The general areas in which information needs to be gathered (as described in A.2. above - attention, comprehension, etc.) to pretest effective HIV vaccine clinical trial messages (brochures, media campaigns, websites, etc.) are generally similar from pretest to pretest. However, the specific questions that are asked of respondents will differ with the message content, audience targeted, and medium of the message.
As each new message, strategy, or product is developed, NIAID will review existing literature and data bases, including pretesting reports on existing messages, materials, and programs, and consult with outside experts to evaluate available information on similar messages with comparable audiences. However, since each message is essentially different, new data collection instruments generally must be prepared for each pretest.
A.5. Impact on Small Businesses or Other Small Entities
Community-based organizations and physicians or other healthcare providers may sometimes be the target audience for NIAID information materials. When testing of these materials is required with physicians or other healthcare providers, NIAID will generally work through established medical and professional societies to gain access to the audience, and to obtain feedback on its instruments and data collection plans. As a result of this contact, NIAID will be able to minimize the placement of additional burden on health care providers.
A.6. Consequence of Collecting the Information Less Frequently
Information will be collected only one time for each print, broadcast, or electronic message, product or strategy tested. Respondents will not be recontacted.
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
NIAID recognizes the need to collect information in a manner that places minimal burden on each respondent. Therefore, when NIAID requires responses to a self-administered written questionnaire in less than 30 days, receipt of the questionnaire generally will be preceded by advance notification to respondents explaining the purpose of the questionnaire, the approximate length of time that the questionnaire will take, and the voluntary nature of participation. All efforts are made to keep such questionnaires short and focused.
Because NIAID’s pretesting activities are primarily qualitative in nature, most results are not generalizable to the population at large or to the particular audience under study. However, the nature of pretesting is such that generalizability is not a critical feature; the emphasis is on obtaining timely, useful information that can be fed back into the development of new messages or materials or the revision of existing ones.
There are no other special circumstances.
A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside
Agency
The required 60-day notice appeared in the Federal Register on August 28, 2007 (Volume 72, Number 166, p. 49282), soliciting comments on the requested new data collection project. One public comment was received, stating that this research has been done previously. Given that the research will test new materials and strategies for effectiveness, it is not correct to say that the proposed research has been done before.
NIH, along with other Public Health Service agencies, has been a leader in the development of methods for developing, testing, and disseminating health information. A number of outside health communications experts were consulted to review the plans contained herein for program development research and evaluation of NIAID communications programs and their comments and suggestions have been incorporated into these data collection plans. The list of experts consulted is contained in Attachment 1.
NIH is the US Government lead for HIV vaccine research and development, and coordinates with other US Government agencies on all HIV vaccine research efforts through the Partnership for AIDS Vaccine Evaluation (PAVE). Through this mechanism, NIAID consults regularly with other agencies, to help ensure accuracy and consistency, and to avoid duplication of effort. NIAID also regularly consults and coordinates with non-US Government HIV vaccine research organizations through formal and informal channels.
A.9. Explanation of Any Incentive or Gift to Respondents
It is standard practice in commercial market research to offer recruited respondents a token of appreciation in order to obtain participation from those who may not otherwise participate. This practice is particularly important when recruiting hard-to-reach and minority respondents. These tokens of appreciation may include small amounts of money, a free meal or snack scheduled around the time of the pretest, and/or parking and/or transportation expenses. Market research literature suggests that monetary incentives have a strong positive effect on the response rate and no known adverse effect on reliability. (Response rate and participant objectivity are further encouraged by reminding participants, either orally or in writing, about the importance of providing both negative and positive feedback.) Circumstances, however, do not always require that incentives be given; many audiences including some members of the public and health and other professionals often participate gratis because of their interest or involvement in the topic, or as a professional courtesy. For each information collection submitted under this umbrella information collection request, NIH will specify what incentives are being offered and justify the provision of the incentives as well as the amount of incentives.
A.10. Assurance of Confidentiality Provided to Respondents
Information provided by respondents will be kept private to the extent permitted by law. This will be communicated to respondents by means of introductory letters, explanatory texts on the cover pages of questionnaires, scripts read prior to focus groups or telephone interviews, and consent forms. Respondents will also be advised of the following: the nature of the activity, the purpose and use of the data collected, NIAID sponsorship, and the fact that participation is voluntary at all times. Because responses are voluntary, respondents will be assured that there will be no penalties if they decide not to respond, either to the information collection as a whole or to any particular questions.
In order to protect respondents’ privacy, all presentation of data in reports will be in aggregate form, with no links to individuals preserved. Reports will be used only for research purposes and for the development of communication messages and educational materials.
The NIH Privacy Act Officer has reviewed the work scope of this proposal and determined that the Privacy Act is applicable to this data collection (Attachment 2). The data collection is covered by NIH Privacy Act Systems of Record 09-25-0156, which is available at http://oma.od.nih.gov/ms/privacy/pa-files/0156.htm.
Pretesting efforts described in this proposal are considered exempt from the “Regulations for the Protection of Human Subjects” in accordance with paragraph (b)(3) of 45 CFR Sec. 46.101.
A.11. Justification for Sensitive Questions
As mentioned in sections A.2. and A.10. above, some studies require the inclusion of people who match selected characteristics of the target audience that NIAID is trying to reach. This sometimes requires asking a question about race/ethnicity, income, education, sexual behavior (specifically whether an individual has had sex with a man), and/or health status on the initial screening questionnaire used for recruiting. Potential participants are informed that this is being done to make sure that NIAID speaks with the kinds of people for whom its messages are intended. Again, respondents are assured that the information is voluntary and will be treated as private to the extent permitted by law. All information on race/ethnicity will comply fully with the standards of OMB Statistical Policy Directive No. 15, October 1997 (http://www.whitehouse.gov/omb/fedreg/1997standards.html).
Since NIAID communications are concerned with HIV/AIDS and participating in HIV vaccine clinical trials, some projects may involve asking questions about (or discussing) how one perceives his/her own personal risk for serious illness. Fears of HIV/AIDS may also be covered, however, respondents will be asked for their HIV status in very limited circumstances. This information is needed to gain a better understanding of the target audience so that the messages, strategies, and materials designed will be appropriate and sensitive. Questions of this nature require some sensitivity in how they are worded and approached. In face-to-face data collections, questions of this kind are generally asked later in the interview or group discussion when respondents are more comfortable with the interview situation and are more at ease with the interviewer/moderator. As noted in section A.10., participants are informed in advance about the nature of the activity and the voluntary nature of their participation. The interviewer/moderator makes it clear that they do not have to respond to any question that makes them uncomfortable.
Raw data from data collections that include sensitive information (for example, screening questionnaires and audio tapes) are not retained once the data have been extracted and aggregated; nor does the information become part of a system of records containing permanent identifiers that can be used for retrieval.
A.12. Estimates of Hour Burden Including Annualized Hourly Costs
The number of respondents to be included in each new pretest will vary, depending on the nature of the material or message being tested and the target audience. However, for illustrative purposes, Table A.12-1 below provides an example of a distribution of respondents and hours by type of data collection. Time to read, view, or listen to the message being tested is built into the "Hours Per Response" figures. Proposed data collection methodologies are described in more detail in Section B.
A.12-1 Estimates of Hour Burden by Anticipated Data Collection Methods
Note: The burden table below reflects what NIAID anticipates would be accomplished over the total 3-year life of the project. (Annual burden, therefore, is one-third of the total figures presented here.)
|
Total Number of Respondents |
Frequency of Response |
Hours Per Response |
Total Hours |
Individual In-Depth Interviews (in person or telephone) |
600 |
1 |
.75 |
450 |
Focus Group Interviews |
972 |
1 |
1.75 |
1701 |
Intercept Interviews: Central Location |
1800 |
1 |
.25 |
450 |
Website Surveys |
3000 |
1 |
.08 |
240 |
Self-Administered Questionnaires |
900 |
1 |
.25 |
225 |
Gatekeeper Reviews |
225 |
1 |
.50 |
113 |
Omnibus Surveys |
3000 |
1 |
.17 |
510 |
|
|
|
|
|
Totals |
10,497 |
|
|
3689 |
(Note: On an annual basis, the total number of respondents is 3,499; the annual number of responses is 3,499; and the total annual hours are 1,230)
Table A.12-2 presents the cost to respondents over the 3-year life of the project. Annual cost, therefore, is one-third of the total figures shown.
A.12-2 Cost to Respondents
Type of Respondents |
Number of Respondents |
Frequency of Response |
Hourly Wage Rate |
Respondent Cost |
General public and communication directors |
10,347 |
1 |
$17.00 |
$61,565.00 |
Physicians |
150 |
1 |
$63.00 |
$7,088.00 |
|
|
|
TOTAL |
$68,653.00 |
The cost to individual respondents who are members of the general public is approximately $5.95 based on the estimate of $17.00/hour and an average respondent burden of .35 hours per respondent.
A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers
There are no capital or start-up costs to the data collection efforts requested; nor are there any costs associated with operation, maintenance or purchase of services.
A.14. Annualized Cost to the Federal Government
The total annual cost to the Federal Government will not exceed $448,000 (or $1,344,000 over 3 years). This estimate is based on annual performance of up to: 2 in-person in-depth interview studies at $22,000 each ($44,000 total); 2 telephone in-depth interview studies at $16,000 each ($32,000 total) 3 focus group studies at $80,000 each ($240,000 total); 3 central location interview studies at $12,000 each ($36,000 total ); 2 self-administered questionnaire studies with at $10,000 each ($20,000 total); 2 Website visitor surveys at $8,000 each ($16,000 total); 3 gatekeeper reviews at $10,000 each ($30,000 total); one telephone or Internet survey at $30,000. These figures include the costs of study design, facility rental (e.g., for focus groups), data collection, analysis, and report/publication writing.
A.15. Explanation for Program Changes or Adjustments
This is a new collection of information.
A.16. Plans for Tabulation and Publication and Project Time Schedule
The process for developing the analytical plan for the pretest is similar to that used in any formal evaluation. Staff will review the material to be pretested, discuss the objectives with the individuals responsible for developing the materials, determine the analytic questions to be addressed in the pretest, and then prepare the pretest procedures, instruments, and data analysis plan. The analyses conducted for each pretest will be determined by the objectives of the pretest, the messages being pretested, and the audience for the messages. Specifics of the analyses cannot be determined until the messages to be pretested are prepared.
Techniques include primarily qualitative analyses (for example, content analysis for in-depth interviews), although some results such as those from central location intercept interviews or telephone or website surveys are summarized quantitatively using descriptive statistics. No complex analytic techniques are used.
While the primary purpose of a pretest is to provide information to the developers of the messages for the purpose of improving them, NIAID anticipates making pretest results available to a variety of health program planners at Government agencies, voluntary organizations, health professional organizations, and medical institutions. In addition, NIAID anticipates presenting the findings of its pretest work at professional associations, including the American Public Health Association, Society for Public Health Education, and the Association for Health Care Research, and publishing its findings in professional journals such as the American Journal of Public Health.
The specific messages that will be pretested and the timing of these messages are not known at this time. However, as indicated in section A.1. above, approximately 18 annual pretest studies are planned. While the pretesting period varies somewhat depending on the complexity of the testing and number of respondents required, the typical pretest will require approximately 8 weeks from initial design to preparation of the report of pretest findings. A schedule for a typical pretest is shown below:
A.16-1 Project Time Schedule
Activity Time Schedule
Initial review of materials 1 week after OMB approval
Preparation of pretest design 2 weeks after OMB approval
Review of design 3 weeks after OMB approval
Collection of data 5 weeks after OMB approval
Analysis of data 7 weeks after OMB approval
Report on pretest 8 weeks after OMB approval
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
NIAID will display the OMB control number and expiration date in the upper right-hand corner of all data collection instruments.
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions
NIAID is in full compliance with the provisions contained within the Certification for Paperwork Reduction Act Submissions.
| File Type | application/msword |
| File Title | NHVREI clearance package |
| Last Modified By | kripkek |
| File Modified | 2008-02-15 |
| File Created | 2008-02-15 |