Supporting Statement Part B Final Retail Food Stores 6-19-2015

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The respondent universe will be 2,400 respondents as follows:

Respondent Description	Number
Person in charge of a randomly selected healthcare facilities	400
Person in charge of randomly selected schools (K-12)	400
Person in charge of randomly selected retail food stores	400
Program director (or designated individual) of the respective regulatory authority over the randomly selected establishments	1,200
Total	2,400

A geographical information system (GIS) database containing a listing of businesses throughout the United States will be used as the establishment inventory for the data collections. The data were purchased from the Environmental Systems Research Institute (Esri). The healthcare facility data, school data, and retail food store data are the partial of Esri’s USA Business Locations and Business Summary. It is updated annually. The data is stored as GeoDataBase.

The Esri list contains data for the establishment name, location, franchise code, both versions of industry classification codes (SIC system and NAICS), number of employees, and estimated sales volume (expressed in thousands of dollars).

Esri extracts its business data from a comprehensive list of businesses licensed from Dun & Bradstreet®. Dun & Bradstreet utilizes its exclusive DUNSRight™ process that harvests information from a mix of data sources to collect, maintain, and verify information on individual establishments. This process leverages Dun & Bradstreet's proprietary databases, customer-generated information, and publicly available sources such as business registries, Internet/web mining, news and media reports, telephone directories, court and legal filings, company financials, banking information, directory assistance, industry trade data, and telephone interviews.

We also combined Esri’s datasets with the Homeland Security Infrastructure Program (HSIP) Gold datasets. HSIP Gold is a unified homeland infrastructure geospatial data inventory assembled by National Geospatial-Intelligence Agency (NGA) in partnership with the Department of Homeland Security (DHS) for official use by the Homeland Security and Homeland Defense (HLS/HD) communities.

The addresses of the establishments are geocoded to assign latitude and longitude coordinates. The quality of the local address system varies. Address matching is better in urban areas that use street-level address system than in rural areas. Establishments that cannot be assigned to a census block group are assigned to a census tract or county. We use the geographic codes to do spatial sampling for the risk factor study.

FDA will perform a three-tiered filtering process to ensure establishments are correctly classified into the appropriate facility type described in Table 4 and considered eligible to participate in the survey. The filter types include: the subclass the establishment belongs to, the name of the establishment, and keywords. For school establishments, we also use additional filters: school enrollment and whether the school end-grade is 1 or greater than 1. The term “eligible” in this context means that the establishment is contained in the geographic areas from which it is being sampled. Any establishment in the geographic areas can be selected.

Description of the Facility Types Included in the Survey

Industry Segment	Facility Type	Description
Healthcare	Hospitals	Foodservice operations that provide for the nutritional needs of inpatients, by preparing meals and transporting them to the patient’s room and/or serving meals in a cafeteria setting (meals in the cafeteria may also be served to hospital staff and visitors).
	Long-Term Care	Foodservice operations that prepare meals for residents in a group care living setting such as nursing homes and assisted living centers
Schools (K-12)	Base Kitchen	School foodservice facility where meals are fully prepared in the on-site kitchen. Some meals are served to students on-site; other meals are shipped to other locations (including multiple locations within the same school.
	On-site Kitchen	School foodservice facility where all meals are prepared and serviced on-site.
	Combination Kitchen	School foodservice facility in which some meals are prepared and served on-site; but some meals are fully prepared or partially prepared in a central or base kitchen
Retail Food Stores	Deli Department / Operation	Areas in a retail food store where foods, such as luncheon meats and cheeses, are sliced for the customers and where sandwiches and salads are prepared on-site or received from a commissary in bulk containers, portioned, and displayed. Parts of the deli department/operation may include: Salad bars, pizza stations, and other food bars managed by the deli department manager, Areas where meat and poultry products are cooked and offered for sale as ready-to-eat and are managed by the deli department manager.
	Seafood Department / Operation	Areas in a retail food store where seafood is cut, prepared, stored, or displayed for sale to the customer.
	Produce Department / Operation	Areas in a retail food store where produce is cut, prepared, stored, or displayed for sale to the consumer. A produce operation may include salad bars or juice stations that are operated under the same manager who has responsibility for the produce department.

An effort will be made to exclude establishments from the study that only handle pre-packaged food items or conduct low-risk food preparation activities.

Approximately 22 FDA Regional Retail Food Specialists (Specialists) will serve as the data collectors for the study. These individuals possess technical expertise in retail food safety and a solid understanding of the operations within each of the facility types to be surveyed. The Specialists are also standardized by FDA Center for Food Safety and Applied Nutrition (CFSAN) personnel in the application and interpretation of the FDA Food Code. The 1,200 data collections will be evenly distributed among all available standardized Specialists.

The Specialists are located near major metropolitan areas (i.e. population centers) across the contiguous United States. Population centers usually contain a large concentration of state and local regulatory jurisdictions.

Eligible establishments are randomly selected from among all eligible establishments located within a 150-mile radius of each of the Specialists’ home locations (zip codes). Using the 150 mile radius sampling zones provides a relatively good cross section of urban and rural areas from which to sample the eligible establishments. It also represents a good mix of small, medium, and large regulatory entities having jurisdiction over the eligible establishments. Lastly, it reduces overnight travel and therefore reduces travel costs incurred by the Agency to collect data.

In the interest of cost efficiency, we apply one caveat to the sampling zones as follows. The actual driving distance to a few of the selected establishments may exceed 150 miles to geographic barriers of one form or another.  Since travel time is not included in the Specialists’ work plan hours and excessive overnight travel would be financial burden to the Agency, if an establishment on the inventory list exceeds a 150 mile driving distance from the Specialist’s home, as confirmed via Google Maps, the Specialist has the option of requesting a substitute establishment.  Specialists are encouraged to still conduct data collections at establishments that may exceed the 150 mile radius by only a few miles (or where travel time is not significantly impacted by the extra distance). When requesting a substitute establishment based on driving distance exceeding 150 miles, the Specialist is to include the Google Map showing the mileage distance from their home to the establishment.

The total number of healthcare facilities in the database is 56,974, and the total number within the 22 sampling zones is 36,322. This means that the 22 sampling zones contain approximately 64% of all healthcare establishments in the contiguous U.S. The total number of schools (including both public and private school) in the database is 129,487, and the total number within the 22 sampling zones is 76,915. This means that the 22 sampling zones contain approximately 59% of all school establishments in the contiguous U.S. The total number of retail food stores in the database is 129,314, and the total number within the 22 sampling zones is 83,830. This means that the 22 sampling zones contain approximately 65% of all retail food store establishments in the contiguous U.S. If additional FTEs are utilized in the data collection, then an even greater percentage of establishments would be contained within approximately 22 sampling zones. All analysis reports will clearly indicate that the sample drawn was purposeful and that estimates generated from the study cannot be generalized to the U.S. as a whole.

The following map and table illustrate the location of FDA’s 22 currently standardized Specialists and the corresponding 150 mile radius.

F IGURE 1. Location of FDA’s 22 Currently Standardized Regional Retail Food Specialists’ home zip code and the surrounding 150 mile radius (restricted by FDA region boundaries)

The following is a current list of Regional Retail Food Specialists who will serve as data collectors by region, home city, state, and zip code:

NORTHEAST REGION
Mary Leong – Brooklyn, NY 11249
Steven Nattrass – Avon, CT 06001
Al Pistorio – Manchester, NH 03103

SOUTHEAST REGION
Diane Kelsch – Orlando, FL 32803
Dan Redditt – Snellville, GA 30078
Chris Smith – Alpharetta, GA 30004
Donna Wanucha – Mocksville, NC 27028

CENTRAL REGION
Greg Abel – Minneapolis, MN 55405
Tracynda Davis – Middleton, WI 53562
Barbara Kitay – Southhampton, PA 18966
Kris Moore – Georgetown, IN 47122
Akeila Randle – Chicago, IL 60661

SOUTHWEST REGION
Scott Krause – Sachse, TX 75048
Cindy Kunkel – Kansas City, MO
Celeste Parker – Dallas, TX 75218
Mario Seminara – Lakewood, CO 80228

PACIFIC REGION
David Engelskirchen – Gig Harbor, WA 98329
Katey Kennedy – Brush Prairie, WA 98606
John Marcello – Gilbert, AZ 85233
Richard Ramirez – San Clemente, CA 92673
Brad Tufto – Colbert, WA 99005
Lisa Whitlock – Loma Rica, CA 95901

Based on the number of entry refusals from the 2013-2014 Risk Factor Study in the restaurant facility types, we estimate a refusal rate of 2 percent in the institutional foodservice and retail food store facility types.

Substitute establishments will be selected in cases when an establishment is misclassified, closed, or otherwise unavailable, unable, or unwilling to participate. The inventory of substitute establishments will remain with the FDA CFSAN Biostatistics Branch until needed by a Specialist to replace an ineligible establishment that was included on his or her original list.

2. Procedures for the Collection of Information

Calculation of the sample size

In order to obtain a sufficient number of observations to conduct statistically significant analysis, FDA has determined, based on the previous 1998-2008 risk factor study, that approximately 400 data collection inspections of each facility type are needed. This sample size was calculated to provide for sufficient observations to be 95% confident that the compliance percentage is within 5% of the true compliance percentage.

The rationale for this calculation follows. The previous study that was designed prior to the initial 1998 data collection did not take into account any effect of intracluster correlation (ICC). A random selection mechanism including all establishments in a geographic area was not used since, at the time, we did not have GIS technology. Instead a comparison set list approach was utilized.

During the 1998 data collection period, each Specialist developed five Comparison Set Establishment Lists for each of the facility types. In most cases, the comparison set lists were comprised of between 10 and 20 establishments located in the same geographical area. Establishments were listed in alphabetical order. In order to maintain data reliability and to ensure confidentiality of the selected establishments, the comparison set lists, as well as the inspectional observations, were retained in a central database by number rather than by establishment name or location.

In order to maintain consistency between data collection periods, the Specialists used the 1998 Comparison Set Establishment Lists in 2003 and in 2008. Selection bias was prevented by using a random number table to choose the establishments that were to be inspected.

Only one establishment was inspected from each comparison set list during the data collection. In addition, an establishment on a comparison set list could only be selected once for inspection. For instance, if in 2003, a Specialist randomly picked an establishment that had already been inspected in 1998, the Specialist would have had to draw another random number until an establishment on the comparison set list that had not been inspected was chosen.

The data from the previous study was used to estimate the intra cluster correlation and to estimate the variance for the sample size computation. This would seem to be a conservative approach since the comparison set list approach would be much more likely to produce geographic correlation than the new design that employs GIS technology and establishment lists that contain all establishments contained in a particular geographic region.

In the previous study there were 42 data items comprising six risk factor areas. If a data item was applicable in the establishment being surveyed and it was observed by the specialist it was marked either IN compliance or OUT of compliance. For each facility type an IN compliance percentage was calculated by summing all of the IN compliance observations and dividing this number by the number of observations IN compliance plus the number of observations OUT of compliance. The baseline IN compliance percentage was calculated in 1998. Data collections in 2004 and 2008 utilized the same 42 data items and the IN compliance percentages for the three data collection periods were then used to track trends over time. Within each facility type the risk factors and individual data items were also analyzed and compared over time. As we dug deeper into the data and looked at risk factors and data items, the sample size became smaller and fewer inferences were made.

Although many different population parameter estimates will be made using this survey data, the sample size was calculated to ensure that the primary goal of the study was achieved. The required sample size was calculated based on the ten primary data items. Each of the ten primary data items should have a response (IN compliance or OUT of compliance) based on the information statements which are contained within each data item. We expect that all or almost all of the data items will have a response (see B.3). We will have a compliance percentage for the ten primary data items which will simply be the total number of IN compliance observations divided by the total number of IN compliance observations plus the total number of OUT of compliance observations. Therefore, each of the 400 establishments will have 10 observations that will be used to compute the IN compliance percentage for the facility type.

Using data from the previous study, the “effective sample size” was calculated as follows:

Health Care Facilities

Where m = 180 responses (10 per establishment) in a geographical area, k= number of geographic areas, and DE is the design effect.  .

JMP 10 was utilized to calculate the ICC in EMP-results obtained by the Measurement Systems Analysis platform.

In order to calculate the sample size we needed an estimate of the variance of the proportion, a confidence level. Utilizing the ESS calculated above and estimates for the IN compliance percentages from the previous study, the precision was estimated as follows:

Where Z_.025 = 1.96, P=.82, Q = .18 and n = 441 and e is the margin of error. Solving for e gives 3.586%.

Once the data is collected, the observed sample ICC and variance will be used when reporting the results. We feel that the sample size will be sufficient to have a margin of error of less than 5% of the estimated proportion of IN compliance observations.

Schools

Where m = 180 responses (10 per establishment) in a geographical area, k= number of geographic areas, and DE is the design effect.  .

JMP 10 was utilized to calculate the ICC in EMP-results obtained by the Measurement Systems Analysis platform.

In order to calculate the sample size, we needed an estimate of the variance of the proportion, a confidence level. Utilizing the ESS calculated above and estimates for the IN compliance percentages from the previous study, the precision was estimated as follows:

Where Z_.025 = 1.96, P=.84, Q = .16 and n =371 and e is the margin of error. Solving for e gives 3.731%.

Once the data is collected, the observed sample ICC and variance will be used when reporting the results. We feel that the sample size will be sufficient to have a margin of error of less than 5% of the estimated proportion of IN compliance observations

Retail Food Stores

Where m = 180 responses (10 per establishment) in a geographical area, k= number of geographic areas, and DE is the design effect.  .

JMP 10 was utilized to calculate the ICC in EMP-results obtained by the Measurement Systems Analysis platform.

In order to calculate the sample size, we needed an estimate of the variance of the proportion, a confidence level. Utilizing the ESS calculated above and estimates for the IN compliance percentages from the previous study, the precision was estimated as follows:

Where Z_.025 = 1.96, P=.78, Q = .22 and n =402 and e is the margin of error. Solving for e gives 3.584%.

Once the data is collected, the observed sample ICC and variance will be used when reporting the results. We feel that the sample size will be sufficient to have a margin of error of less than 5% of the estimated proportion of IN compliance observations.

Preparatory Training

Each Specialist will attend a training webinar prior to initiating the data collection. The training will be provided by members of the FDA National Retail Food Team that have been responsible for the design and assessment of all the Retail Food Risk Factor Study elements. The training will cover all the study components with particular emphasis on the data collection protocol and marking instructions for the data collection form.

The training will cover all the following study components:

• Data collection protocol (Attachment A)

• Marking instructions for the data collection forms (Attachment B)

• Data collection forms (Attachment C)

• Industry Introductory Letter (Attachment D)

The data collection forms (Attachment C) are specific to each industry segment. Each form is divided into three sections: Section 1 - Establishment Information; Section 2 - Jurisdiction with Regulatory Authority Information; and Section 3 for tabulating the Specialists’ observations of (a) the food employees’ behaviors and practices related to personal hygiene and food storage, preparation, and service, (b) the industry food safety management being employed, and (c) the frequency of food employee hand washing.

Verification of Eligibility of Randomly Selected Establishments

Specialists will receive a set of health care, school, and retail food store facilities within their primary area of responsibility that have been randomly selected for the study by the FDA CFSAN Biostatistics Branch. Prior to conducting the data collection, Specialists will contact the state or local jurisdiction that has regulatory responsibility for conducting retail food inspections for the selected establishment. Specialists will verify that each facility has been properly classified (in the correct facility type category) for the purposes of the study and is still in operation. Specialists will ascertain whether the selected facility is under legal notice from the state or local regulatory authority. If the selected facility is under legal notice, the Specialists will not conduct a data collection in that establishment. The Specialists will remove the establishment from their sample inventory and select another establishment from their list of substitute establishments provided by the FDA CFSAN Biostatistics Branch.

Information Collection Involving Regulatory Authorities

As part of the initial contact with the regulatory authority, Specialists will obtain information from the jurisdiction pertaining to the items listed under the heading “Information on the Regulatory Authority” on the applicable data collection form for the facility type (Attachment C). The data collection fields in this section are the same for all three industry segments (healthcare; schools; retail food stores). At that time, Specialists will collect the information for the following data collection fields:

• Name of Jurisdiction with Regulatory Oversight.

• Enrolled in FDA Retail Food Program Standards.

• Jurisdiction Meets Standard 1.

• Dates of the Two Most Recent Regulatory Routine Inspections.

• Jurisdiction Uses a Grading System.

• Type of Grading System.

• Jurisdiction’s Program Includes Public Reporting of Inspection Results.

• Jurisdiction Has a Mandatory Food Protection Manager Certification Requirement.

• Scope of Food Protection Manager Certification Requirement.

• Jurisdiction Requires Food Handler Cards.

Guidance for completing these data collection information fields is included on pages 14 – 21 in the FDA Retail Food Program, Foodborne illness Risk Factor Study - Marking Instructions for the Data Collection Form (Attachment B).

Specialists will extend an invitation to the state or local regulatory authority to accompany him or her on the data collection visit. Should the regulatory authority accept and accompany the Specialist, the Specialist should strongly recommend that the state or local regulatory authority refrain from conducting a regulatory compliance inspection during the data collection visit.

Calibration of Temperature Measuring Devices

Specialists will ensure that thermometers used for each data collection are accurate by verifying their accuracy prior to each establishment data collection visit.

Conducting the Data Collection

Each data collection visit is to be unannounced. The intent is to observe the operation in its normal mode, without special preparation to accommodate the data collection visit.

Upon arrival to the establishment, Specialists will explain to the owner the purpose of the visit. An introductory letter (Attachment D) that explains the purpose of the data collection visit and the study must be used in addition to a verbal explanation. If entry into the selected establishment is denied by the owner or person in charge, Specialists will not conduct a data collection. Specialists will select a new establishment from the substitute establishment list provided by the FDA CFSAN Biostatistics Branch.

After discussing the purpose of the data collection and developing a rapport with the person in charge, Specialist will conduct a quick (two to three minute) walk-though of the establishment’s kitchen as described in the Study Protocol (Attachment A) to determine inspection priorities and flow.

Primary Data Items

Specialists will then make every effort to observe procedures and practices related to data items 1 through 10 (primary data items) in Attachment C. Each of the primary data items has been placed under the appropriate FDA foodborne illness risk factor category which will be used as the key indicators for FDA’s statistical analysis for the study:

• Risk Factor –Poor Personal Hygiene

#1 – Employees practice proper handwashing

#2 – Food Employees do not contact ready-to-eat foods with bare hands

• Contaminated Equipment / Protection from Contamination

#3 – Food is protected from cross-contamination during storage, preparation, and display

#4 – Food contact surfaces are properly cleaned and sanitized

• Improper Holding / Time and Temperature

#5 – Foods requiring refrigeration are held at the proper temperature

#6 – Foods displayed or stored hot are held at the proper temperature

#7 – Foods are cooled properly

#8 – Refrigerated, ready-to-eat foods are properly date marked and discarded within 7 days of preparation or opening

• Inadequate Cooking

#9 – Raw animal foods are cooked to required temperatures
#10 – Cooked foods are reheated to required temperatures

Comprehensive guidance for marking observations of primary data items is provided on pages 59 – 79 of the FDA Retail Food Program, Foodborne Illness Risk Factor Study – Marking Instruction for the Data Collection Form (Attachment B).

Other Areas of Interest – Data Items

Specialists will also collect information on data items 11 through 19 are listed under the heading “Other Areas of Interest” in Attachment C. These food safety practices and procedures directly support active managerial control of the foodborne illness risk factor areas addressed under the primary data items:

• Data Item #11 – Handwashing facilities are accessible and properly maintained

• Data Item #12 – Employees practice good hygiene

• Data Item #13 – Consumers are properly advised of risks of consuming raw or undercooked animal foods

• Data Item #14 – Time alone is properly used as a public health control

• Data Item #15 – Facilities have adequate equipment and tools for ensuring food temperature control and sanitization of food contact surfaces

• Data Item #16 – Special processes are conducted in compliance with issued variance / HACCP Plan, when required

• Data Item #17 – Food is received from safe sources

• Data Item #18 – Toxic materials are identified, used and stored properly

• Data Item #19 – Management and food employees are trained in food allergy awareness as it relates to their assigned duties

Comprehensive guidance for marking observations of data items listed under the “Other Areas of Interest” is provided on pages 80 – 93 of the FDA Retail Food Program, Foodborne Illness Risk Factor Study – Marking Instructions for the Data Collection Form (Attachment B).

Information Statements

The Specialists will also collect information related to the “information statements” under most of the data items. These information statements are preceded by a letter for organization purposes and describe a specific observation (food safety practice) associated with the overarching data item under which it is listed.

Documenting Observations of Food Safety Practices

Using the current version of the FDA Food Code, the data collector will determine whether the observations made of the employee food safety practices or behaviors contained in the information statements were IN Compliance, OUT of Compliance, Not Observed (NO), or Not Applicable (NA). The recorded markings of the information statements are then used to determine the compliance status of the corresponding data item.

An observation is based on an evaluation of one or more occurrences of a data item or information statement at an establishment. Specific instructions for marking each data item and information statement are provided in the FDA Retail Food Program, Foodborne Illness Risk Factor Study – Marking Instruction for the Data Collection Form (Attachment B). The four marking options are defined as follows:

• IN – means that all observed occurrences were IN Compliance with the appropriate FDA Food Code provision for the data item or information statement.

• OUT – means that one or more of the observations made were OUT of Compliance with the appropriate FDA Food Code provision for the data item or information statement. An explanation of the specific criteria used for determining OUT of Compliance for each data item is to be recorded by the data collector on the data collection form.

• NO – means the data item or information statement was Not Observed during the inspection. The NO marking is used when an information statement is a usual practice in the food establishment, but the practice is NOT observed during the time of the inspection.

• NA – means the data item or information statement is Not Applicable. The NA marking is used when a data item or information statement is NOT a function of the food establishment.

Quantitative measurements will be made with calibrated thermocouples, heat sensitive tape or maximum registering thermometers, and chemical test strips. Quantitative temperature measurements will be recorded in the food temperature charts provided on the data collection form. Sanitization measurements will be recorded in the comment section for the specific data item observed.

Recording Food Product Temperatures

Specialists will record all food product temperatures measured during the data collection in the charts provided under data items that contain specific product temperature critical limits.

The database that will be used to record the data has been designed to provide a drop down menu for the Food Code Critical Limits for each temperature-based data item. Using the food product temperature entered by the Specialist, the database has been programmed to automatically calculate the difference between the food product temperature recorded by the Specialist and the Food Code critical limit. The database system will then use this information to automatically enter the correct totals in the summary of product temperatures table depicted below. The Specialist will not have to manually complete the product temperature summary tables.

Handwashing Frequency Assessment

Specialists will record all handwashing observations during the regular data collection using the “Handwashing Frequency Assessment” located under data item #1 – Employees practice proper handwashing on the Data Collection Form (Attachment C). Over the course of the data collection visit, the Specialist will record a tally of each time an employee is observed doing the following.

• Washing hands properly and when required.

• Washing hands improperly.

• Failing to wash hand when required.

Assessment of Food Safety Management Systems

In addition to collecting information on compliance with the FDA Food Code, Specialists will obtain information on the extent to which food establishments have developed and implemented food safety management systems. FDA will use this information to examine the correlations, if any, between the degree to which management systems are in place and the control of foodborne illness risk factors.

The Food Safety Management System Assessment will be conducted during the same establishment visit but independent from the determination of Food Code compliance for individual data items.

The assessment of food safety management systems will focus on systems related to the control of the four key foodborne illness risk factors associated with the ten primary data items.

Each randomly selected establishment will have a management system assessment conducted for ONE of the four foodborne illness risk factor areas described above. The FDA CFSAN Biostatistics Branch will randomly select the risk factor area for which a food safety management system assessment is to be conducted for each establishment.

Specialists will evaluate the presence of three key food safety management system elements (procedures, training, and monitoring) for each of the primary data items listed under the assigned risk factor:

• Procedures – A defined set of actions adopted by food service management for accomplishing a task in a way that minimizes food safety risks.

• Training – Management informs employees what the procedures are and teaches the employees how to carry them out.

• Monitoring – Routine observations and measurements made by management to determine if procedures are being followed and maintained.

For each of these three food safety management system elements, the data collector will determine if the information provided by the establishment management adequately addresses the essential critical limits for the assigned risk factor area. A food safety management system assessment questionnaire has been developed for each of the foodborne illness risk factor areas. The questionnaire for each of the risk factor areas requires the Specialist to enter a YES or NO response for the following four statements:

• Management is able to describe the critical limits for (the specific risk factor procedure or practice) as they apply to their establishment.

• Management is able to describe the steps / tasks (how and when) that are performed to ensure the identified critical limits for (the specific risk factor procedure or practice) are achieved.

• Management is able to identify specific employees that have been assigned the responsibility to correctly perform the (the specific risk factor procedure or practice).

• Management is able to produce written materials (SOPs; posters; wall charts; wallet cards; etc.) that support the implementation of their (the specific risk factor procedure or practice) within their establishment

Using the food safety management system assessment tool, the data collector will add up the total number of “YES” responses for each of the management system elements (Procedures, Training, and Monitoring). The number of “YES” responses on the assessment tool will determine how to mark the Procedures, Training, And Monitoring sections for the data item on the data collection form.

Comprehensive guidance for marking the food safety management system assessment (Procedures; Training; and Monitoring) for selected risk factor areas is provided on pages 52 – 55 of the FDA Retail Food Program, Foodborne Illness Risk Factor Study – Marking Instruction for the Data Collection Form (Attachment B).

Establishment Information

During the course of the data collection, Specialists will interview the owner/person in charge and make observations to collect information to complete the Establishment Information sections of the Data Collection Form (Attachment D). These sections include:

• Establishment Information such as size, capacity and level of activity.

• Manager Certification.

• Employee Health Policy.

• Foodservice for Highly Susceptible Populations (only in healthcare facilities).

Guidance for completing the information fields associated with these sections of the data collection form is provided on pages 1 – 13 and 22 – 37 of the FDA Retail Food Program, Foodborne Illness Risk Factor Study – Marking Instructions for the Data Collection Form (Attachment B).

Corrective Actions – Observations that Pose a Significant Public Health Risk

Though industry participation in the Study is voluntary, correction action is to be obtained for observations that pose a significant public health risk. If conditions observed during the data collection visit pose a significant public health risk, the Specialist will discuss the situation with the person in charge and seek to obtain voluntary corrective action. FDA’s experience from data collections performed as part of its previous study indicate that in all but a few instances, industry responded in a cooperative and responsible manner to alleviate potential public health risks.

Should an instance occur where an observation during the data collection poses a significant public health risk and corrective action cannot be voluntarily obtained, the Specialist will contact the appropriate regulatory authority to ensure appropriate corrective actions are taken.

Exit Briefing with Person in Charge

Upon completion of the data collection, the Specialist will conduct an exit briefing with the owner or person in charge to discuss significant findings and answer any questions. No written report is left with the establishment.

Capturing the Data

During the 2015-2016 data collection, FDA will begin the transition process from manual data entry to the use of hand held technology. As part of an agency-wide initiative to assess the use of hand held technology, FDA will have five Specialists conduct their data collections using hand held tablets. The information from the hand held tablets will be transferred “real-time” into the FoodSHIELD database via a web browser.

All of Specialist will enter their data into a web-based database platform located in FoodSHIELD using secure FDA computers.

3. Methods to Maximize Response Rates and Deal with Non-response

The expected response rate is 98%. The study design includes assignment of substitute establishments to a Specialist when the originally selected establishment is misclassified, closed, or otherwise unavailable, unable, or unwilling to participate. The inventory of substitute establishments will remain with the FDA CFSAN Biostatistics Branch until needed by a Specialist to replace an ineligible establishment that was included on his or her original list.

4. Test of Procedures on Methods to be Undertaken

The procedures were pilot tested prior to the 2013-2014 data collection in risk factor study in restaurants. No additional pilot testing will be needed for the 2015-2016 study in retail food stores, schools, and healthcare facilities.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Sampling and Statistical Methods/Data Analysis:

Marc Boyer

Mathematical Statistician

FDA/CFSAN/OFDCER/DPHB

5100 Paint Branch Parkway

College Park, MD 20740

(240) 402-1686

Marc.Boyer@fda.hhs.gov

Dr. Guilan Huang

Regulatory Information Specialist

FDA/CFSAN/Office of Food Safety/Retail Food Protection Staff

5100 Paint Branch Parkway

College Park, MD 20740

240-402-2904

Guilan.Huang@fda.hhs.gov

NORTHEAST REGION

Alfred P. Pistorio

New England District Office

One Montvale Avenue

Stoneham, MA 02180

(781) 587-7427

Fax: (781) 587-7558

Alfred.Pistorio@fda.hhs.gov

Steven Nattrass

Hartford Resident Post

135 High Street; Room 230
Hartford, CT 06103

(860) 240-4289 ext. 18

Fax: (860) 240-4313

Steven.Nattrass@fda.hhs.gov
  

Mary Leong

158-15 Liberty Ave., 5th Floor

Jamaica, NY 11433-1034

(718) 662-5536

Fax: (718) 662-5434

Mary.Leong@fda.hhs.gov

CENTRAL REGION

Greg Abel

MN District Office

250 Marquette Avenue

Suite 600

Minneapolis, MN 55401

(612) 758-7199

Fax: (612) 334-4134

Greg.Abel@fda.hhs.gov

Kris Moore

Louisville Resident Post

9600 Brownsboro Road, Suite 210

Louisville, KY 40241

(502) 425-0069, Ext. 1013

Fax: (502) 425-0450

Kris.Moore@fda.hhs.gov

Barbara Kitay

Regional Field Office

900 US Customhouse

200 Chestnut Street, Rm 902

Philadelphia, PA 19106

(215) 717-3700

Fax: (215) 597-5798

Barbara.Kitay@fda.hhs.gov

Akeila Randle

CE-FO/CE-SPS

Burrell Building – Rm 510

20 N. Michigan

Chicago, IL 60602

(312) 596-6514

Fax: (312) 886-1682

Akeila.Randle@fda.hhs.gov

Tracynda Davis

Pittsburgh Resident Post                    

7 Parkway, Suite 250                                            

Pittsburgh, PA 15220     

           (412) 644-3394, Ext. 15             

Fax: (412) 644-5496

Tracynda.Davis@fda.hhs.gov

SOUTHEAST REGION

J. Daniel Redditt

Southeast Regional Office

60 8th Street, N.E.

Atlanta, GA 30309

(404) 253-1265

Fax: (404) 253-2257

joseph.redditt@fda.hhs.gov

Diane Kelsch

555 Winderly Place, Suite 200

Maitland, FL 32751

(407) 475-4747

Fax: (407) 475-4770

Diane Kelsch@fda.hhs.gov

Christopher Smith

Southeast Regional Office

60 8th Street, N.E.

Atlanta, GA 30309

(404) 253-1264

Fax: (404) 253-2257

Chris.Smith@fda.hhs.gov

Donna Wanucha

Charlotte Resident Post

5701 Executive Center Dr.

Suite 104

Charlotte, NC 28212

(704) 344-6116

Fax: (704) 344-6402

Donna.Wanucha@fda.hhs.gov

SOUTHWEST REGION

Scott Krause

Southwest Regional Office

4040 N. Central Expressway

Suite 900

Dallas, TX 75204

(214) 253-4948

Fax: (214) 253-4960

Scott.Krause@fda.hhs.gov

Mario Seminara

Denver District Office

Denver Federal Center

Building 20

P.O. Box 25087

Denver, CO 80225

(303) 236-3026

Fax: (303) 236-3551

Mario.Seminara@fda.hhs.gov

Cindy Kunkel

Kansas City District Office

11630 West 80^th Street

Lenexa, KS 66214

HFR-SW36

(913) 752-2401

Fax: (913) 752-2487

Cynthia.Kunkel@fda.hhs.gov

Celeste Parker

Southwest Regional Office

4040 N. Central Expressway

Suite 900

Dallas, TX 75204

(214) 253-4946

Fax: (214) 253-4960

Celeste.Parker@fda.hhs.gov

PACIFIC REGION

Lisa Whitlock

Pacific Regional Office

Oakland Federal Building

1301 Clay Street

Suite 1180N

Oakland, CA 94612

(510) 287-2700

Fax: (510) 637-3976

Lisa.Whitlock@fda.hhs.gov

Richard Ramirez

Los Angeles District Office

19701 Fairchild, Suite 1179

Irvine, CA 92612-2506

(949) 608-4475

Fax: (949) 608-4498

Richard.Ramirez@fda.hhs.gov

Kathryn Kennedy

9708 SW Nimbus Avenue, Building 16

Beaverton, OR 97008

HFR-PA3515

(503) 671-9711 ext. 16

Fax: (503) 671-9445

Kathryn.Kennedy@fda.hhs.gov

John Marcello

Phoenix Resident Post

51 West Third Street, Suite 265

Tempe, AZ 85281

(480) 829-7396 ext. 35

Fax: (480) 829-7677

John.Marcello@fda.hhs.gov

Bradley Tufto

Spokane Resident Post

621 N. Argonne, Suite 102

Spokane Valley, WA 99212

(509) 353-2554

Brad.Tufto@fda.hhs.gov

David H. Engelskirchen

Puget Sound Resident Post

1000 2^nd Avenue Suite 2400

Seattle, WA 98104

(206) 340-8237

Fax: (206) 553-7020

David.Engelskirchen@fda.hhs.gov

22