National_Survey_of_Employers_Supporting_Statement_20100809 (clean)

SUPPORTING STATEMENT

Department of Defense National Survey of Employers

A. JUSTIFICATION

A.1. Need for Information Collection

In accordance with 10 United States Code, Section 2358 (Attachment 1), the Under Secretary of Defense for Personnel and Readiness (USD[P&R]) is required to conduct research of interest to the Department of Defense (DoD). This research of interest to the DoD requires the collection and dissemination of information to non-military U.S. employers of Selected Reserve and National Guard members who are absent for more than 30 days to serve in the military. In a directive-type memorandum issued 21 March 2003 (Attachment 1), the USD(P&R) required military departments to implement a Civilian Employment Information (CEI) Program. The memorandum states the CEI is to have ready employment-related information for mobilization planning purposes of the Services concerned, and this information shall also be used on a recurring basis to help the Department accomplish its employer outreach purposes.

The Uniformed Services Employment and Reemployment Rights Act (USERRA) (Attachment 1) protects Service members’ reemployment rights when returning from a period of service. The Act requires persons who serve or have served in the Armed Forces, Reserves, National Guard or other “uniformed services:” (1) are not disadvantaged in their civilian careers because of their service; (2) are promptly reemployed in their civilian jobs upon their return from duty; and (3) are not discriminated against in employment based on past, present, or future military service.

The Act covers members of the Uniformed Services, the commissioned corps of the Public Health Service, any other category of persons designated by the President in time of war or national emergency, and their government and civilian employers. It is the responsibility of the Employer Support of the Guard and Reserve (ESGR) to promote cooperation and understanding between Reserve component members and their civilian employers and to assist in the resolution of conflicts arising from an employee's military commitment. The Department of Defense National Survey of Employers is being conducted on a statistically random basis to determine best practices in supporting employers of Reserve and Guard members and to evaluate the effectiveness of ESGR and DoD programs. The information collected is used for overall program evaluation, management, and improvement.

Primary Objectives. The primary objectives of this research are to determine employer burden when National Guard and Reserve employees are absent serving in the military for more than 30 days, to evaluate the impact of DoD policy (including the switch to the Operational Reserve and Contiguous Pre-Mobilization Training) on employer operations, and to gain insight into unmet employer needs. This research will examine the general attitudes toward Guard and Reserve employees. The research will also assess the contributions of Guard and Reserve employees to employers; and examine knowledge of and desired supports for compliance with USERRA. The research will examine these objectives by number of employees at the sampled establishments, industry, state, and employment of National Guard and Reserve members, and generalize them to the universe of eligible employers. A detailed content summary can be found in Attachment 1. This summary highlights the relationships between the research objectives and the specific items found on the questionnaire.

Reasons the Information is Necessary. This information collection is necessary because previous research in this area is limited, outdated, and/or not specific to the objectives of this information collection request.

A.2. Use of Information

The sponsor of the Department of Defense National Survey of Employers is the Office of the Under Secretary of Defense (P&R), and the users of the data will be ESGR, the Office of the Secretary of Defense, Reserve Affairs, other DoD senior staff and administrators, and DMDC.

The respondents for this information collection are U.S. employers. Sampling efforts have focused on providing representative samples of employers that include National Guard and Reserve members among their employees and those that do not.

In general, the survey results will be used to inform DoD about the impact on employers when their employees are absent to serve in the military and to help shape policy and program initiatives, pre-mobilization planning efforts, and ESGR outreach efforts to employers. More specifically, results will also be used by ESGR to identify areas where the training and assistance it provides can be improved; i.e., ESGR will use the results for overall program evaluation, management, and improvement.

The survey will ask about general employment policies including experience with National Guard and Reserve employee absences for military duties and the challenges resulting from these absences. Survey items will also ask respondents how these challenges have been managed in the past and to suggest ways to manage these challenges in the future. Survey questions will also ask about the respondents’ (or the establishments’) experiences with ESGR programs pertaining to communicating with and supporting employers, and about their knowledge of USERRA.

To assist with its research and program evaluation objectives, ESGR will compare the survey results with the information collected previously by the Department of Reserve Affairs in 1999 and by the Institute for Defense Analyses (IDA) in June 2008. When IDA conducted its survey, only 478 usable responses were collected and IDA was thus unable to make broad generalizations about the impact on employers. The scope of the IDA analysis was also much narrower, for example, IDA examined only those employers who employed a mobilized Reservist.

The paper-and-pencil survey instrument used to conduct the Department of Defense National Survey of Employers, the informed consent text, the Agency Disclosure Statement, and the Privacy Advisory Statement are included in Attachment 3. The final web-based survey instrument has not yet been programmed, but the web-based survey will contain the same text as the hard copy version, and mock-ups of the introductory Web screens the respondents will see prior to accessing the actual instrument are included in Attachment 7.

A.3. Improved Information Technology

Respondents will be able to complete the survey on the Web via the survey operations contractor’s secure Web site. However, a paper-and-pencil option will also be made available because it is expected that some businesses will not have Web-access available for their employees to use to complete an external survey. To access the Web site, respondents will be provided with an individual access code; i.e., a unique Ticket number, and will be asked to create a PIN number for added security. Respondents opting to complete a paper-and-pencil survey will receive a hard copy instrument with their unique Ticket number located on the back of the survey instrument. The purpose of the unique Ticket number is to ensure only persons at eligible businesses can access the survey Web site, to ensure respondents complete the Web survey instrument only once, and to determine if a completed paper version of the instrument has been submitted. For further details on protection of data for the Web-based survey see section A11.

A. 4. Efforts to Identify Duplication

Reserve Affairs conducted the “Survey to Determine Economic Costs to and Impact on Employers of Mobilized Reserve Component Members” in the fall 2006. This information is obsolete and cannot be used for understanding the impact on the complete range of employers, especially small employers, since the Reserve Affairs survey was conducted only of Fortune 500 companies. The United States Air Force was considering conducting its own survey of employers which would have had the same stated purpose of as the Department of Defense National Survey of Employers. However, after consulting with ESGR, USAF decided against their own survey in favor of using the results that would be generated from the current survey.

A. 5. Methods Used to Minimize Burden on Small Entities

The number of survey questions has been kept to a minimum to minimize the burden on all employers, especially small entities who have fewer resources to devote to completing surveys. The survey intentionally does not ask for any detailed policy, financial, or other administrative record information in order to reduce burden. The DoD does not anticipate any selected employers, including small entities, will experience adverse impacts due to the effort or cost of reporting the requested information. In addition, a screening question at the start of the survey minimizes the questions asked of those businesses not employing any Reserve or Guard members within a given time frame; that is, during the past 36 months, and many of those will be small businesses. In addition, it will be small businesses, who likely have few employees to spare, that have the greatest potential to benefit from the improvements that ESGR intends to make to its programs as a result of the survey.

A. 6. Consequences of Not Collecting the Information

This is a single request for information. Not collecting this information will preclude DoD’s ability to: 1) benchmark employment conditions of Reservist and National Guard personnel and; 2) respond to employers’ needs that may jeopardize the DoD’s outreach efforts. By creating significant lag times between the identification and response to unmet employer needs, the DoD’s ability to gain and maintain employer support may be limited and its planning efforts possibly impeded. In addition, no information collection may result in insufficient numbers of enlistments and reenlistments in the Selected Reservists which are critical components of DoD’s ability to maintain sufficient forces to service national security requirements. The collection information will allow informed evaluation of policy, program, and legislative efforts to establish and maintain employer support, and will help the DoD to formulate its recommendations to Congress. In addition, ESGR hopes to conduct a similar survey approximately every other year but using a shorter instrument and limiting the sample to only those businesses that have National Guard and Reserve employees as identified by the CEI database. The results from the current Department of Defense National Survey of Employers would provide ESGR with the critical information it would need to develop a useful and valid data collection instrument for these future surveys as it continues its mission of maintaining employer support for National Guard and Reserve service, increasing awareness of the relevant laws impacting both employer and employee, and resolving, through mediation, potential conflict that may from an employee's military commitment.

A. 7. Special Circumstances

There are no special circumstances. This collection will be conducted in a manner consistent with guidelines contained in 5 CFR 1320.5(d)(2).

A. 8. Agency 60-Day Federal Register Notice and Consultations Outside the Agency

An agency 60-Day Federal Notice was published in Vol. 74, No. 215, Monday, November 9, 2009, Federal Register, page 57664, as required by 5 CFR 1320.8(d). A copy of the 60-Day Federal Notice is included in Attachment 4. No public comments were received in response to the notice.

DMDC has consulted with Dr. Fran Featherston (703.521.3439) and Dr. Jock Black (703.292.7802) of the National Science Foundation (NSF) regarding the proposed sampling and data collection methodology. NSF advised DMDC based in part on their experience with the 2008 Business R&D and Innovation Survey. An outcome of this consultation was the decision to reduce the sample size from the original number of 250,000 employers to the current 80,000 employers in order to place more financial and field resources into the relatively costly telephone screening and telephone reminders calls. The result was a more efficient sample design while maximizing the response rates, and therefore maintaining the ability to calculate reliable statistical estimates once all the data were collected.

DMDC, along with ESGR, briefed Secretary Dennis M. McCarthy, the Assistant Secretary of Defense for Reserve Affairs, on the survey methodology, and incorporated his feedback in the design. An outcome of this meeting was the decision not to attempt to contact Reservists’ supervisors directly for the survey in order to maintain the privacy of the employed Reservists. DMDC has worked closely with ESGR as the survey instrument was being developed, and ESGR had the responsibility for signing off on the final version. Also, throughout the planning and survey development process, ESGR consulted with a number of other agencies including the Society for Human Resource Management and the U. S. Air Force. A full list of the individuals consulted by ESGR for the proposed data collection is included in Attachment 5. Finally, DMDC and ESGR also consulted with a number of volunteer employers who reviewed drafts of the survey instrument resulting in edits to make the instrument more accessible to employers.

Coordinations were obtained from Ms. Cindy Allard, OSD/JS Privacy Office, WHS/ESD, 703.588.2386, and Ms. Andrea Zucker, Exempt Determination Official for the Office of the Deputy Under Secretary of Defense (Program Integration), Human Research Protection Program, DHRA, 703.696.7178.

A. 9. Payments to Respondents

The ESGR will provide no payments or gifts to respondents.

A. 10. Assurance of Confidentiality

The information collection does not ask respondents to submit propriety or trade secret information to DoD. Though DMDC cannot promise confidentiality to this population, respondents will be told that the information they provide will be kept private to the extent permitted by law.

A. 11. Sensitive Questions

The data collection instrument contains no questions of a sensitive nature. The survey will be non-intrusive and respondents will be informed that their participation is voluntary. The survey does not collect personally identifiable information and survey responses are not retrieved by personal identifier. Therefore, the information collected is not subject to the Privacy Act of 1974, as amended. DMDC will only report results in the aggregate; that is, in the form of statistical summaries.

On the paper-and-pencil version of the instrument, the Agency Disclosure Notice (ADN) is located on the inside cover immediately before the Privacy Advisory Statement. The Privacy Advisory Statement is followed by the informed consent information on the inside front cover of the survey document. The informed consent information includes the instructions "Returning this survey indicates your agreement to participate in this research."

On the Web version of the instrument, the ADN is located on a screen the sample members must read before continuing to the Privacy Advisory Statement and Informed Consent Information screen. The Privacy Advisory Statement and Informed Consent Information screen is one the sampled members must read before taking the survey (Attachment 7). The informed consent screen includes the instruction "Click 'Continue' if you agree to do the survey" and informed consent is indicated by clicking the "Continue" button and answering the survey questions.

Because the data are collected on a Web site, the Web site is required to include Security Protection Advisory (Attachment 7) information according to the Office of the Secretary of Defense Policy for Establishing and Maintaining a Publicly Accessible Department of Defense Web Information Service (dated July 18, 1997; updated January 9, 1998). The Advisory for the survey program informs visitors to the Web site no information on the person's computer or Internet connection is collected in a way it can be associated with the person or the survey responses. The Security Protection Advisory page is accessed via a link from the Opening page.

The data collection procedures are not expected to involve any risk to participants. Names are used only in communicating with employers. These names are kept securely by the survey operations contractor and are not linked to response data. The datasets sent to DMDC contain no names or addresses. Access to full detailed confidential data is limited to DMDC analysts and contractors under their direct supervision.

DMDC’s and the survey contractor’s network sites are secure and password protected. Security is strictly enabled by using physical and software access restrictions. All servers are physically located in locked rooms with access permitted only to Technical Services staff through the use of a security card system. Access to the network is allowed only through a login account and password. In addition, employees use password protected screen savers at workstations to protect their systems while they are away from their desks. At DMDC, the network is accessed through the use of Common Access Card (CAC) readers and utilizes Public Key Infrastructure (PKI) security. Logging on to the network requires both physical possession of the CAC and a separately issued Personal Identification Number. All computer systems comply with current Federal Information Security Management Act security standards.

The survey contractor makes daily backup tapes that are stored for five years in fire proof vaults located within a security-card protected area, and all provisions dealing with the protection of human subjects and data security are in force for as long as the contractor retains any protected data.

A. 12. Estimates of Annual Response Burden and Labor Cost for Hour Burden to the Respondent for Collection of Information.

a. Response Burden:

Total annual respondents: 24,000

Frequency of response: 1

Total annual responses: 24,000

Burden per response: 25 minutes (average across respondents)

Total burden hours: 600,000 minutes/60 = 10,000 hours

b. Explanation of How Burden was Estimated. The anticipated number of respondents is based on conservative estimates of the response rates (30%) across the employer databases (CEI and Non-CEI) from which the total sample was constructed. The burden per response of 25 minutes is a weighted average: It is expected that 75% of the completed surveys will be submitted by those employers who employ (or have employed) a Reserve or National Guard member, and these will take approximately 30 minutes to complete. The remaining 25% of the completed surveys will be submitted by those employers who do not employ (or did not employ) a Reserve or National Guard member, and these will take approximately 10 minutes to complete.

c. Labor Cost to Respondent.

Total annual respondents: 24,000

Frequency of response: 1

Total annual responses: 24,000

Burden per response: 25 minutes

Average cost per response: $8.37 ($19.92 *.42 (25/60 minutes))

Total cost per response $200,880 (Average cost per response ($8.37) * number of responses (24,000)).

d. Explanation of How Labor Cost to Respondent was Estimated. The estimated hourly wage used to calculate the average cost per response is the 2010 GS-9/1 hourly rate of $19.92 (excluding any locality adjustment) given that the survey would likely be completed by a Human Resources representative or a line/site manager.

A. 13. Estimates of Other Cost Burden for the Respondent for Collection of Information.

a. Total Capital and Start-up Cost. There are no capital/startup costs.

b. Operation and Maintenance Cost. There are no operation and maintenance costs. No outside resources, consultations or record retrieval are required to answer the survey questions. Any computer costs borne by the establishment will be minimal.

A. 14. Estimates of Cost to the Federal Government.

a. Federal Labor Costs.

GS Grade/Step	Annual Rate	25% Fringe	Monthly	FTE Months	Cost
12/1	$74,872	$93,590	$7,799	6	$46,795
13/1	$89,033	$111,291	$9,274	4	$37,097
14/1	$105,211	$131,514	$10,959	3	$32,878
Total Cost					$116,770

b. Explanation of How Cost was Estimated. Federal labor costs were estimated using the GS Salary Table for 2010-DCB which includes a locality payment of 24.22% for the Washington, Baltimore, and Northern Virginia area. An additional estimated 25% fringe benefit cost was added based on research available from the U.S. Bureau of Labor Statistics.¹

c. Additional Costs

Survey Contractor Operations and Maintenance Cost $651,397

(Includes contractor labor to produce surveys and letters, screening and reminder telephone calls, all data collection costs, materials and freight, data storage, and postage.)

On-Site Contractor Support $174,277

(Costs for support contracts are based on negotiated rates for similar services.)

Government Staffing Cost $ 116,770

(Includes sampling and weighting, analysis of basic data set, creation of tab volume, statistical methods reports, contractor technical oversight, contract administration, consults with ESGR, preparation of the all final internal documents.)

Total Cost $942,444

A. 15. Changes in Burden

This is a new collection. There have been no appreciable changes in burden.

A. 16. Publication Plans/Time Schedule.

There are currently no plans to publish the results outside the Department of Defense.

A. 17. Approval Not to Display Expiration Date

This approval is not being requested.

A. 18. Exceptions to the Certification Statement

No exceptions to the Certification Statement are being requested.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B. 1. Description of Activity

Employer Support of Guard and Reserve (ESGR) will conduct a national survey of employers. The populations of interest will consist of establishments that employ or employed Guard or Reserve members on their workforce since June 2007 (CEI) and establishments without Guard or Reserve members on their workforce (Non-CEI). DMDC will conduct the sampling and weighting.

The first population was built from a DMDC maintained database, Civilian Employer Information system (CEI). For the purpose of this study, only establishments in the 50 states and the District of Columbia who employ or employed Guard or Reserve members on the workforce since June 2007 as full or part-time paid employees were included. Self-employed, students and volunteers were excluded. This frame represents all distinct establishments with unique DUNS numbers (a number assigned by Dun & Bradstreet that identifies unique establishments), North American Industry Classification System (NAICS) codes, and usable state information. The final sampling frame (total number of unique establishments) for the CEI population totaled to 134,138 establishments. The second population (Non-CEI) of establishments without Guard or Reserve employees was built from the ‘U.S. Census Bureau: ‘the 2005 Statistics of U.S. Businesses’ and the ‘2007 Census of Governments.’ The final sampling frame for the Non-CEI population totaled to 7,830,378 establishments.

The primary sampling unit (PSU) for this study is the establishment. For each population, the establishments were grouped by their NAICS code within the 50 states and the District of Columbia. The total number of NAICS codes was condensed into 18 groups (Table 1). The Public Administration group consists of federal, state, and local establishments.

For each population the list of establishments was stratified by state and NAICS groups. For each stratum the sample size was determined by the Power Allocation Method (Table 2). Power allocation permits a focus on total estimates at the national and state levels while accounting for states’ dynamic differences in size (number of establishments). Alternate sampling methods were discussed, but excluded because if a proportionate-to-size allocation strategy was used, then the sample from smaller states would be much smaller (e.g., Alaska vs. California, Table 3), resulting in very little survey input from states and NAICS groups classified as small. If an equal allocation strategy was adopted across states and NAICS groups, a considerable design effect would be expected as the selection probabilities would vary greatly.

Power allocation is a compromise method between equal and proportionate-to-size allocation. Under power allocation the sample allocates proportional to x^λ, where x is the measure of size and λ can take on values between 0 and 1.

If we let n denote the total sample size, n_h be the sample allocated to stratum h, and N_h be the number of establishments in stratum h, then the sample allocated to stratum h is given as:

, 0< λ <1

For λ = 0.5 get,

In the above formulas, λ=0 corresponds to equal allocation, and λ=1 corresponds to proportionate-to-size allocation. A square root allocation (λ =0.5) was used for this study. Sample size for both populations by state is shown in Table 3.

Although the interest is in obtaining estimates at the state level from both populations combined (CEI and Non-CEI) and in NAICS group estimates for each population separately, we will stratify each state by the NAICS groups to assure appropriate representation of business types within states for each population. A single-stage stratified sample of approximately 80,000 establishments will be selected of which approximately 60,000 establishments from CEI and 20,000 from the Non-CEI population. The sample is allocated to permit simultaneous representation of the largest States and NAICS as well as other sizes of states and NAICS nationwide. Sample contact information is provided by Dun & Bradstreet. DMDC will confirm and update the CEI information as required.

Sampling for the Department of Defense National Survey of Employers will be conducted as follows: Since interest is in state total and in total NAICS groups estimates, the sample was stratified by states and NAICS groups (A random sample based on power allocation was selected from each combination of state and NAICS groups). States total estimates were based on samples from both the CEI and the Non-CEI samples. NAICS estimates were derived at the national level and for the CEI and the Non-CEI populations individually. Every business falls in a certain NAICS group. Every business, whether employing Guard or Reserve members or not, was represented within each state. For each population, Dun & Bradstreet will select the sample independently within each stratum based on detailed specifications. The estimated total number of respondents is approximately 24,000.

Table 1: NAICS Groups

NAICS Code	Industry
11, 21, 22, 562	Agriculture, Mining, Utilities, Waste Management
23	Construction
31-33	Manufacturing
42	Wholesale trade
44-45	Retail trade
48-49	Transportation and warehousing
51	Information
52, 55	Finance and insurance/ Management of companies and enterprises
53	Real estate and rental and leasing
54	Professional, scientific, and technical services
561	Administrative and support services
61-(6111-6117)	Educational services
62 (minus 621910)	Health care and social assistance
71	Arts, entertainment, and recreation
72	Accommodation and food services
81	Other services (except public administration)
922120, 922160, 621910	First Responders
92	Public Administration

Table 2: CEI and Non-CEI Populations and Power Allocation Samples by NAICS Groups

NAICS Code	Industry	CEI Population	CEI Sample	Non-CEI Population	Non-CEI Sample
11, 21, 22, 562	Agriculture, Mining, Utilities, Waste Management	2,857	1,347	88,079	258
23	Construction	7,183	3,307	811,452	2,136
31-33	Manufacturing	11,881	5,271	331,355	833
42	Wholesale trade	5,968	2,676	434,464	1,049
44-45	Retail trade	15,919	7,169	1,123,629	2,902
48-49	Transportation and warehousing	6,951	3,066	219,806	580
51	Information	3,472	1,501	143,779	357
52, 55	Finance and insurance/ Management of companies and enterprises	4,989	2,198	558,734	1,432
53	Real estate and rental and leasing	2,491	1,078	380,138	940
54	Professional, scientific, and technical services	9,993	4,309	867,556	2,131
561	Administrative and support services	7,949	3,437	363,043	909
61-(6111-6117)	Educational services	6,817	3,022	74,402	189
62 (minus 621910)	Health care and social assistance	10,682	4,751	780,310	1,964
71	Arts, entertainment, and recreation	1,794	802	125,222	316
72	Accommodation and food services	7,820	3,568	632,489	1,621
81	Other services (except public administration)	5,981	2,663	744,252	1,921
922120, 922160, 621910	First Responders	5,715	2,601	48,132	141
92	Public Administration	15,676	7,109	103,536	322
Total	Total	134,138	59,877	7,830,378	20,000

Table 3: CEI and Non-CEI Populations and Power Allocation Samples by State

State	CEI Population	CEI Sample	Non-CEI Population	Non-CEI Sample
United States	134,138	59,877	7,830,378	20,000
Alabama	3,043	1,356	108,015	366
Alaska	602	602	20,575	160
Arizona	2,914	1,327	143,772	423
Arkansas	1,895	1,070	70,420	296
California	10,574	2,529	895,787	1,055
Colorado	2,348	1,192	160,547	447
Connecticut	1,341	900	94,595	343
Delaware	770	682	25,943	180
District of Columbia	933	751	20,940	161
Florida	7,147	2,079	524,398	807
Georgia	4,137	1,582	233,655	539
Hawaii	1,208	855	33,335	204
Idaho	783	688	48,954	247
Illinois	4,146	1,583	334,495	645
Indiana	3,006	1,348	157,307	442
Iowa	2,597	1,253	86,416	328
Kansas	2,005	1,101	82,217	320
Kentucky	1,553	969	96,086	346
Louisiana	2,039	1,110	105,784	363
Maine	859	721	44,048	234
Maryland	2,247	1,166	141,689	420
Massachusetts	3,212	1,394	178,063	471
Michigan	2,586	1,250	240,017	546
Minnesota	2,825	1,307	156,699	441
Mississippi	1,650	999	63,717	281
Missouri	3,538	1,463	158,264	444
Montana	817	703	39,693	222
Nebraska	1,479	946	56,012	264
Nevada	1,334	898	63,126	280
New Hampshire	895	736	40,584	225
New Jersey	2,217	1,158	245,857	553
New Mexico	682	642	48,101	245
New York	5,059	1,749	524,177	807
North Carolina	3,900	1,536	229,897	535
North Dakota	896	736	24,864	176
Ohio	5,656	1,849	276,324	586
Oklahoma	2,414	1,208	94,801	343
Oregon	1,622	990	115,357	379
Pennsylvania	5,020	1,742	313,115	624
Rhode Island	731	665	30,587	195
South Carolina	2,843	1,311	109,190	368
South Dakota	1,097	814	28,426	188
Tennessee	2,414	1,208	139,652	417
Texas	9,963	2,454	529,041	811
Utah	1,604	985	72,598	300
Vermont	580	580	23,672	172
Virginia	5,055	1,748	201,506	501
Washington	3,183	1,387	186,774	482
West Virginia	990	774	41,817	228
Wisconsin	3,378	1,429	147,765	429
Wyoming	351	351	21,704	164

B. 2. Procedures for the Collection of Information

Field Procedures. The following data collection procedures will be undertaken after receipt of OMB approval.

1. Initial telephone screening calls will be made to the selected establishments prior to the start of data collection in order to identify, and preferably to speak directly with, the individuals who are most appropriate to be contacted for the survey, e.g., the Human Resources staff person in charge of Guard and Reserve programs. Information to be collected at the time of the screening call includes the contact person’s full name and job title, telephone number, email address, and postal mailing address. Another purpose of the screening call is to determine if the contact person is able to access external Web sites at their establishment in order to complete the web-based survey or if a paper version of the survey will be required or preferred. The entire telephone screening call phase will take place over a period of 4 – 5 weeks and will conclude as close to the start of data collection as possible. It is assumed that 25% of the establishments will require a second phone call to identify the correct individual and to collect the necessary information. The telephone script for making the screening calls is included in Attachment 2.

2. A notification letter will be postal mailed to all establishment contacts identified during the screening telephone calls (or will be addressed to the appropriate job title and/or office if the telephone screening did not successfully identify a specific individual) to coincide with the time the Web site is to be activated for survey data collection. There will be two versions of the announcement letter, one for those contacts who stated they are able and willing to complete an on-line instrument, and one for those contacts who are unable to complete an on-line instrument, or would prefer a paper-and-pencil instrument. This second version of the letter would include the paper-and-pencil instrument. Establishments not successfully reached during the phone screening, will receive the version of the letter that includes the paper-and-pencil instrument. Both versions of the letter would describe the purpose and sponsor of the survey, a toll-free phone number to call with questions, the Web address, and a unique ticket number (Attachment 2).

3. On the day the Web survey opens, an announcement email will also be sent to all establishment contacts identified during the screening telephone calls for whom email addresses have been collected. Similar to the announcement postal letters, the announcement e-mail will describe the purpose and sponsor of the survey and will provide a Web address for the survey and a unique ticket number (Attachment 2).

4. Approximately two (2) weeks after the e-mail announcement and the postal notification mailing, the first of three (3) reminder e-mails will be sent to all non-respondents for whom email addresses are available (Attachment 2). At points during the remainder of the field period, the second and third e-mail reminders will be sent to non-respondents for whom there are good email addresses. The survey operations contractor will maintain a control system which will be updated daily for Web receipts and paper receipts to determine non-respondents.

5. Interspersed with the e-mail reminders, a total of three (3) reminder telephone calls will be made to the contacts at the non-responding establishments. DMDC will determine which non-responding establishments will receive the reminder phone calls based on tracking reports received from the survey operations contractor.

6. When respondents log on to the Web site using their unique Ticket number to complete the on-line version of the instrument, they will be directed to a set of Web screens starting with the Welcome screen. From there they can view the Frequently Asked Questions and the Security Protection Advisory. Then the respondents are directed first to the screen where they create their own PIN number, and then to the screen entitled, “Privacy Act Statement and Informed Consent Information.” Mock ups of these screens are included in Attachment 7.

Businesses completing a paper-and-pencil version of the survey will use the postage-paid envelope to mail the completed survey to the survey operations contractor who will first log the received instruments into the control system and then review them and conduct any necessary cleaning and editing before data entry. The cleaning process includes looking for any errors in skip logic and cleaning difficult-to-read hand written entries. The hard copies are scanned into the database and the data are then merged with the data coming from the Web-based survey responses. Once all data are collected, cleaned, and merged, SAS databases are then created and shipped to DMDC for weighting and analysis.

B. 3. Non-response, Maximization to Response Rates, and Accuracy and Reliability

To encourage participation and maximize the response rate, the telephone screening calls are intended to identify those individuals who can best, and most easily, provide reliable and valid answers to the survey questions. These individuals may also have a vested interest in the survey given its purpose includes informing DoD about the impact on employers when their employees are absent to serve in the military and helping reduce the burden on employers by shaping policy and developing program initiatives and outreach efforts. In addition, it is expected that having specific individuals identified to contact at the establishments via the screening telephone calls will help maximize response rates because the e-mail reminders, and especially the telephone reminders, would be personalized and sent directly to these individuals. The Telephone Call Center staff will be trained and prepared to answer questions about the survey effort that may come up during the reminder telephone call and in that way help reduce the number of refusals as well. Ideally, the option of conducting a CATI interview right at the time of the reminder phone call would be made available to the respondent, but that option was determined to be cost prohibitive. The decision was made to spend the available funds on placing a larger number of short reminder telephone calls than on making fewer, but much longer and more costly, telephone calls to conduct CATI interviews.

Using its automated case control system, the survey operations contractor will submit both daily and cumulative returns throughout the field period allowing DMDC to closely track production. In this way, the telephone reminder effort can be systematically planned out throughout the field period. Since the analysis plan includes producing statistical estimates separately by state, industry, and size of establishment (i.e., number of employees), it will be particularly important to monitor returns by these three sample unit characteristics. Tracking returns in this way will be possible because the sample DMDC provides to the contractor will include these variables in the record of each establishment. Telephone reminders can be made to those states or types of businesses with relatively low returns earlier in the field period while there is still time to make a difference in the effort to reach the production goals.

Independent of this effort, ESGR plans on mounting an extensive media campaign to increase its visibility within the community of employers at large and to inform employers that DoD is conducting a survey in which they may be asked to participate. The campaign would coincide with the timing of the survey, and would include such things as radio Public Service Announcements (PSAs), Web-based media, notices in large employer publications, and press releases in large cities. It is anticipated that the media campaign would increase the legitimacy of the survey, and would help in the efforts to maximize the response rate.

DMDC based the sample design on a conservative expected response rate of 30%. If the actual response rate is 30%, the sample design will still allow DMDC to produce reliable statistical estimates of percentages with margins of error less than five percentage points. However, we anticipate that the data collection methodology, particularly the use of extensive telephone screening calls and the reminder calls, will result in a response rate greater than 30%; i.e., at least 60%. In addition, DMDC and ESGR will examine the actual response rates (by state, industry, and size of business), as a guide when designing the field procedures for the surveys of businesses ESGR plans on conducting in the future which are referred to in Section A. 6: Consequences of Not Collecting the Information.

Response rates will be calculated using the following, fairly strict, formula whereby a completed survey is one that has at least 50% of the applicable questions completed, and where e(Unknown Eligibility) is an estimate of the non-responding businesses with an unknown eligibility status.

Completed /

[Completed + Partial Interviews) + (Non-interviews) +e(Unknown Eligibility)]

To handle unit survey nonresponse, DMDC will employ logistic regression models to predict the probability of eligibility for the survey (known eligibility vs. unknown eligibility). A second logistic regression model will be used to predict the probability of response among eligible sample members (complete response vs. nonresponse). Predictors will include business size, type (NAICS code), and geographic location (state). Next, DMDC will post-stratify the survey estimates to match known population totals. This procedure reduces bias unaccounted for by the previous weighting adjustments. Post-stratification cells will include the same variables.

DMDC’s sampling, weighting, and estimation plans will result in estimates of percentages with margins of error less than five percentage points. Separate estimates will be made by business size, type, and geographic location.

Analysis of Survey Nonresponse

Both DMDC and OMB are concerned with general declining response rates in the survey industry. To address this concern, OMB issued standards and guidelines for federal statistical surveys requiring that, for any survey with a response rate below 80 percent, survey agencies conduct a nonresponse analysis. For the National Survey of Employers, DMDC plans to use administrative data available on the sampling frame to compare survey respondents and nonrespondents. For this analysis, DMDC plans to use the following characteristics (variables) of the sample establishments:

1. Industry classification – represented by 2-digit NAICS code (DMDC has augmented these codes to discern between Federal State, and local governments)

2. Business size

   1. Number of employees at establishment

   2. Number of employees in the business

3. Establishment geography

   1. State (from D&B file)

   2. Metropolitan vs. non-metropolitan statistical area

4. Sampling frame (CEI file vs. non-CEI component of D&B)

5. D&B confidence code (variable is scored 1 to 10 by D&B based upon quality of contact name, phone number, postal address, and other variables)

6. Number of years in business

   1. Establishment

   2. Business

7. Success of screening interviews

   1. Call outcome (successful vs. non-successful)

   2. Number of calls required

   3. Access to online completion of survey, etc.

8. Success of reminder phone calls

   1. Call outcome (successful vs. non-successful)

   2. Number of calls conducted (0 to 3)

DMDC will compute response rates for each of these groups. If response rates differ within a group, DMDC will consider use of the identified variable for post-survey statistical adjustments including nonresponse adjustments to survey weights. In addition, DMDC will analyze the responses to the survey by the eight groups above and by survey mode (web versus paper). These analyses and subsequent statistical adjustments should reduce potential nonresponse bias to the National Survey of Employers and inform the DoD on methods for improving future surveys of businesses, including maximizing survey response and the effectiveness of both telephone screeners and telephone nonresponse follow-up.

B. 4. Tests of Procedures

DMDC utilizes best practices in its design of Web-based surveys; e.g., visual presentation of questions and response options, usability and interactive elements, use of color, font style and size, and screen layout. These features of Web-based data collection, in addition to automated skip logic, serve to ease the burden on the respondents, increase data quality, and minimize response error. Similarly, DMDC has incorporated a number of randomized experiments into its surveys over the past several years testing such conditions as number and wording of letters and emails, timing of respondent contacts, presence or absence of a brochure, sponsorship, and subject line text. As a result of these experiments, DMDC has developed field procedures that are grounded in experience and empirical findings.

ESGR consulted with several of their employer volunteers, who regularly review early drafts of ESGR products, on the content of the current Department of Defense National Survey of Employers. The current survey reflects their feedback. In addition, ESGR examined the results of the 2008 IDA survey and worked with DMDC on using the results of the IDA survey to inform the content of the current survey.

B. 5. Personnel/Organizations Responsible for Design, Collection or Analysis of the Information

DMDC Principal Investigator: Dr. Tim Elig (703-696-5858)

DMDC Analysts: Kimberly Hylton, Dr. Kenneth Matos, Laverne Wright, Kristin Williams

DMDC Survey Reviewer: Dr. Robert Simmons

DMDC Survey Statisticians: Dr. Fawzi Al Nassir, Eric Falk, Dorothy Kester Jackman, and David McGrath

ESGR Principal Investigators: Virginia Hyland and Samantha Walker

NSF Consultants: Dr. Jock Black (703.292.7802) and Dr. Fran Featherston (703.521.3439);

Survey Operations Contractor: Data Recognition Corporation (DRC) (800-826-2368): Contact person: Valerie Waller (763.268.2166)

1 An estimate of 25% as the cost of fringe benefits was taken from a review of two papers available from the U.S. Bureau of Labor Statistics website; 1) Report on the American Workforce, U.S. Department of Labor, Elaine L. Chao, Secretary, 2001, and, 2) “The Growth of fringe benefits: implications for social security” by Yung-Ping Chen.

23