OMB fast track ss PART A 4-26-11

Generic Clearance for the Collection of Qualitative Feedback on FDA Service Delivery

0910-NEW

Supporting Statement

1. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

Executive Order 12862, “Setting Customer Service Standards” directs Federal Agencies to provide service to the public that matches or exceeds the best service available in the private sector. In order to work continuously to ensure that our programs are effective and meet our customers’ needs, The Food and Drug Administration (FDA) seeks to obtain Office of Management and Budget (OMB) approval of a generic clearance to collect qualitative feedback on our service delivery. By qualitative feedback we mean information that provides useful insights on perceptions and opinions, but are not statistical surveys that yield quantitative results that can be generalized to the population of study.

This collection of information is necessary to enable FDA to garner customer and stakeholder feedback in an efficient, timely manner, in accordance with our commitment to improving service delivery. The information collected from our customers and stakeholders will help ensure that users have an effective, efficient, and satisfying experience with FDA’s programs. This feedback will provide insights into customer or stakeholder perceptions, experiences and expectations, provide an early warning of issues with service, or focus attention on areas where communication, training or changes in operations might improve delivery of products or services. These collections will allow for ongoing, collaborative and actionable communications between FDA and its customers and stakeholders. It will also allow feedback to contribute directly to the improvement of program management.

2. Purpose and Use of the Information Collection

Improving Agency programs requires ongoing assessment of service delivery, by which we mean systematic review of the operation of a program compared to a set of explicit or implicit standards, as a means of contributing to the continuous improvement of the program. FDA will collect, analyze, and interpret information gathered through this generic clearance to identify strengths and weaknesses of current services and make improvements in service delivery based on feedback. The solicitation of feedback will target areas such as: timeliness, appropriateness, accuracy of information, courtesy, efficiency of service delivery, and resolution of issues with service delivery. Responses will be assessed to plan and inform efforts to improve or maintain the quality of service offered to the public. If this information is not collected, vital feedback from customers and stakeholders on FDA’s services will be unavailable.

FDA will only submit a collection for approval under this generic clearance if it meets the following conditions:

• The collections are voluntary;

• The collections are low-burden for respondents (based on considerations of total burden hours, total number of respondents, or burden hours per respondent) and are low-cost for both the respondents and the Federal Government;

• The collections are non-controversial and do not raise issues of concern to other Federal agencies;

• Any collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the near future;

• Personally identifiable information (PII) is collected only to the extent necessary¹ and is not retained;

• Information gathered will be used only internally for general service improvement and program management purposes and is not intended for release outside FDA (if released, procedures outlined in Question 16 will be followed);

• Information gathered will not be used for the purpose of substantially informing influential policy decisions ²; and

• Information gathered will yield qualitative information; the collections will not be designed or expected to yield statistically reliable results or used as though the results are generalizable to the population of study. 

If these conditions are not met, FDA will submit an information collection request to OMB for approval through the normal PRA process.

To obtain approval for a collection that meets the conditions of this generic clearance, a standardized form will be submitted to OMB along with supporting documentation (e.g., a copy of the moderator guide, a comment card or questionnaire). The submission will have automatic approval, unless OMB identifies issues within 5 business days.

The types of collections that this generic clearance covers include, but are not limited to:

• Customer comment cards/complaint forms

• Small discussion groups

• Focus Groups of customers, potential customers, delivery partners, or other stakeholders

• Qualitative customer satisfaction surveys (e.g., post-transaction surveys; opt-out web surveys)

The FDA has established a manager/managing entity to serve for this generic clearance and will conduct an independent review of each information collection to ensure compliance with the terms of this clearance prior to submitting each collection to OMB.

3. Use of Improved Information Technology and Burden Reduction

Consideration will be given to collecting information electronically or using online collaboration tools to reduce burden. However, the applicability of such tools to in-person discussions is at present relatively unlikely.

4. Efforts to Identify Duplication and Use of Similar Information

No similar data are gathered or maintained by the FDA or are available from other sources known to the FDA.

5. Impact on Small Businesses or Other Small Entities

Small business or other small entities may be involved in these efforts but FDA will minimize the burden on them of information collections approved under this clearance by sampling appropriately, asking for readily available information, and using short, easy-to-complete information collection instruments.

6. Consequences of Collecting the Information Less Frequently

Without these types of feedback about its service delivery, FDA will not have timely information to adjust its services to meet customer needs.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this collection of information. The information collected will be voluntary and will not be used for statistical purposes.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

8a. Publication in the Federal Register

In accordance with 5 CFR 1320.8(d), FDA published a 60 day notice for public comment in the FEDERAL REGISTER on December 22, 2010 (75 FR 80542). No comments were received.

9. Explanation of Any Payment or Gift to Respondents

FDA as a rule will not provide payment or other forms of remuneration to respondents of its various forms of collecting feedback. Focus groups and cognitive laboratory studies are the exceptions.

In the case of in-person cognitive laboratory and usability studies, the FDA may provide stipends of up to $40. In the case of in-person focus groups, the FDA may provide stipends of up to $75. If respondents participate in these kinds of studies remotely, via phone, or Internet interview requiring travel, any proposed stipend needs to be justified to OMB. Normally, a stipend of $40 will be provided. If such information collections include hard-to-reach groups and FDA plans to offer non-standard stipends, FDA will provide OMB with additional justifications in the request for clearance of these specific activities. If OMB guidance for the stipend level is adjusted upward, the stipends may also be increased accordingly.

10. Assurance of Confidentiality Provided to Respondents

Information provided by respondents will be kept private and anonymous, except as otherwise required by law. This will be communicated to respondents by means of introductory letters, explanatory texts on the cover pages of questionnaires, scripts read prior to focus groups, telephone interviews, and consent forms. Respondents also will be advised of the following: the nature of the activity; the purpose and use of the data collected; FDA sponsorship (when appropriate³); and the fact that participation is voluntary at all times. Because responses are voluntary, respondents will be assured that there will be no penalties if they decide not to respond, either to the information collection as a whole or to any particular questions.

Only personnel from a contractor conducting the information collection will have access to focus group, individual-level survey or interview data. All project staff from a contractor conducting the information collection must take required measures to ensure the privacy and anonymity of data. Personally identifiable data shall be limited to information that may be required in the process of respondent enrollment. Personally identifiable information will be accessible to only those contractors who need them and will not be linked to interview data collected in the focus group discussion. All personally identifiable data will be destroyed following data collection at the completion of the study. Neither FDA employees nor any Federal employee of any other Agency will have access to this information.

All electronic and hard copy data will be maintained securely throughout the information collection and data processing phases. While under review, electronic data will be stored in locked files on secured computers; hard copy data will be maintained in secure building facilities in locked filing cabinets. As a further guarantee of privacy and anonymity, all presentation of data in reports will be in aggregate form, with no links to individuals. Reports will be used only for research purposes and for the development of communication messages.

Communications testing efforts are typically considered exempt from the “Regulations for the Protection of Human Subjects” in accordance with paragraph (b)(3) of 45 CFR Sec. 46.101 (Attachment 5). Before data are collected, FDA researchers must obtain either an exemption or a full approval for all research from FDA’s Investigational Review Board (IRB), the Research Involving Human Subjects Committee.

Minors (or children) are persons who have not attained the legal age for consent to treatments or procedures described in the study are covered under the applicable law of the jurisdiction in which the research will be conducted. Where FDA’s IRB determines that minors are capable of giving an assent, the IRB shall determine whether adequate provisions are made for soliciting assent. Generally, assent requires securing the signature of a minor to the research in a separate assent form, in addition to the consent form the parent or legal guardian signs. An assent document should contain an explanation of the study, a description of what is required of the subject (e.g., what they will experience (in what setting the interview or focus group will take place, whether the child's parents will be with him or her, etc.)), an explanation of any risks or mental anguish associated with the study topic, and an explanation of the benefits to the child or others.

11. Justification for Sensitive Questions

As mentioned in Section 10, some studies require the inclusion of people who match selected characteristics of the target audience that FDA is trying to reach. This may require asking a question about race/ethnicity, income, education and/or health status on the initial screening questionnaire used for recruiting. Potential participants are informed that this is being done to make sure that FDA speaks with the kinds of people for whom its messages are intended. Again, respondents are assured that the information is voluntary and will be treated as private and anonymous. All information on race/ethnicity will comply fully with the standards of OMB Statistical Policy Directive No. 15, October 1997 (http://www.whitehouse.gov/omb/fedreg/1997standards.html).

Because FDA communications may be concerned with the prevention of premature mortality, some projects may involve asking questions about (or discussing) how one perceives his/her own personal risk for serious illness. The probability of sensitive questions occurring depends upon the topic of the communication. This information is needed to gain a better understanding of the target audience so that the messages, strategies, and materials designed will be appropriate and sensitive. Questions of this nature, while not as personal as those about sexual behavior or religious beliefs, still require some sensitivity in how they are worded and approached. In face-to-face data collections, questions of this kind are generally asked later in the interview or group discussion, when respondents are more comfortable with the interview situation and are more at ease with the interviewer/moderator. As noted in Section 10, participants are informed prior to actual participation about the nature of the activity and the voluntary nature of their participation. The interviewer/moderator makes it clear that they do not have to respond to any question that makes them uncomfortable.

Raw data from data collections that include sensitive information (for example, screening questionnaires and audio tapes) are not retained once the data have been extracted and aggregated. The information is never a part of a system of records containing permanent identifiers that can be used for retrieval.

12. Estimates of Annualized Burden Hours and Costs

A variety of instruments and platforms will be used to collect information from respondents. The annual respondent burden hours requested (4,967) are based on the number of collections we expect to conduct over the requested time frame for this clearance.

12a. Annualized Hour Burden Estimate

Table 1 --Estimated Annual Reporting Burden
Type of Collection	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)4	Total Hours
Focus groups	725	1	1.75	1,269
Customer comment cards/forms	1,200	1	15/60	300
Small discussion groups	725	1	1.75	1,269
Customer satisfaction surveys	6,450	1	20/60	2,129
TOTAL	9,100			4,967

12b. Annualized Cost Burden Estimate

The annualized cost burden for respondents is estimated for all occupations and for the health professions, as the primary occupational group likely to be involved in studies undertaken under this generic clearance. Studies conducted under this generic clearance will provide annualized respondent cost burden for the study population of interest.

	Total Burden Hours	Hourly Wage Rate5	Total Respondent Costs
All occupations	2,429	$20.90	$50,766
Physicians	1,269	$82.02	104,172
Allied Health Professions	1,269	$ 27.96	$35,481
TOTAL			$190,420

13. Estimates of Other Total Annual Costs to Respondents and/or Recordkeepers/Capital Costs

No costs are anticipated.

14. Annualized Cost to Federal Government

FDA incurs costs to set up small discussion and focus groups, including potentially hiring a contractor to provide a facilitator/moderator, rent meeting space, travel to conduct the groups, and provide respondents with minimum payment cost in the form of a token stipend. For these expenses, FDA spends approximately $200,000 annually.

For customer satisfaction surveys and other interview surveys, FDA incurs costs to hire a contractor to select respondents, draw up the questionnaire, cognitively test the questionnaire, and conduct and analyze the survey, for which FDA spends approximately $7,500 per question per survey. For a 20-question survey, the average cost is expected to be $150,000.

Therefore, FDA’s total annualized estimated cost to the Federal government is $350,000.

15. Explanation for Program Changes or Adjustments

This is a new information collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Feedback collected under this generic clearance provides useful information, but it does not yield data that can be generalized to the overall population. Findings will be used for general service improvement, but are not for publication or other public release.

Although FDA does not intend to publish its findings, FDA may receive requests to release the information (e.g., congressional inquiry, Freedom of Information Act requests). FDA will disseminate the findings when appropriate, strictly following FDA's "Guidelines for Ensuring the Quality of Information Disseminated to the Public," and will include specific discussion of the limitation of the qualitative results discussed above.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

We are requesting no exemption to this requirement.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Not applicable.

B. STATISTICAL METHODS

Data collection methods and procedures will vary. However, the primary purpose of these collections will be for internal management purposes. There are no plans to publish or otherwise release this information.

1. Respondent Universe and Sampling Methods

The activities under this clearance may involve identifying samples of self-selected customers, as well as using convenience and quota samples, with respondents selected either to cover a broad range of customers or to include specific characteristics related to certain products or services. Results will not be used to make statements representative of any universe of individuals, to produce statistical descriptions (careful, repeatable measurements), or to generalize the data beyond the scope of the sample. The specific sample planned for each individual collection and the method for soliciting participation will be described fully in each collection request.

Qualitative surveys are tools used by program managers to change or improve programs, products, or services. The accuracy, reliability, and applicability of the results of these surveys are adequate for their purpose.

The samples associated with this collection are not subjected to the same scrutiny as scientifically drawn samples where estimates are published or otherwise released to the public.

2. Procedures for the Collection Information

Data collection methods and procedures will vary and the specifics of these will be provided with each collection request. FDA expects to use a variety of methodologies for these collections. For example, FDA or its contractors may use commercial survey-specific software to automate its collection and analysis of feedback. In addition to physical copies, information collection instruments may be electronically disseminated and/or posted on target pages of FDA’s Web site. Telephone scripts and personal interviews may also be used.

3. Methods to Maximize Response Rates and Deal with Non-response

Information collected under this generic clearance will not yield generalizable quantitative findings; it can provide useful customer input, but it does not yield data about customer opinions that can be generalized to the larger population.

4. Test of Procedures or Methods to be Undertaken

Pretesting may be done with internal staff members, a limited number of external colleagues, and customers who are familiar with the programs and products. If the number of pretest respondents exceeds nine members of the public, FDA will submit the pretest instruments for review under this generic clearance.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

As appropriate, each program will consult with FDA statisticians or outside contractors in developing, designing, conducting, and analyzing customer service surveys. FDA will include the names and contact information of persons consulted in the specific information collection requests submitted under this generic clearance.

1 For example, collections that collect PII in order to provide remuneration for participants of focus groups and cognitive laboratory studies will be submitted under this request. All privacy act requirements will be met.

2 As defined in OMB and agency Information Quality Guidelines, “influential” means that “an agency can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions.”

3 In some cases, FDA sponsorship will not be made known to respondents prior to data collection out of concern for the potential introduction of bias to study results. In such cases, FDA sponsorship will be made known after the data are collected.

4 Burden estimates of less than one hour are expressed as a fraction of an hour in the format “[number of minutes per response]/60”.

5Bureau of Labor Statistics, May 2009 National Occupational Employment and Wage Estimates, United States, NAICS 115200; http://www.bls.gov/oes/current/oes_nat.htm#29-0000.