1205-0421 ONET Supporting Statement FINAL 08-18-2015

This Supporting Statement is a request and justification for a 3-year clearance from the Office of Management and Budget (OMB) to continue, with minor revisions, the Occupational Information Network (O*NET®) Data Collection Program. The O*NET Data Collection Program continually operates to populate and maintain a current database on the detailed characteristics of workers, occupations, and skills. The program uses an occupational taxonomy, the O*NET-SOC, which is based on the 2010 version of the Standard Occupational Classification (SOC) mandated by OMB for use by all federal agencies collecting occupational and labor market information (LMI). The O*NET-SOC comprises occupations at the most detailed level of the SOC and includes additional occupational specificity as needed. In addition, new and emerging occupations in high-growth sectors of the economy have been identified and added to the taxonomy. Data have been published for 940 O*NET-SOC occupations, 509 of which have undergone more than one update. The O*NET Data Collection Program received initial OMB clearance in 1999 for a pretest and five subsequent clearances that have allowed main study data collection to continue without interruption since June 2001. Our current clearance expires June 30, 2015. This request is to continue to update occupations that reflect older data as well as to collect data on selected high-growth and new and emerging occupations for 3 more years (July 2015–June 2018), subject to annual budget levels.

The continued population of the O*NET database is important because the O*NET database is the most current and comprehensive standard source of descriptive occupational information in the United States. The O*NET Data Collection Program remains at the center of an extensive network of occupational and skill information used by a wide range of audiences, including individuals making career decisions; public agencies, such as workforce boards and American Job Centers, making training investment decisions; educational institutions preparing a future workforce; and employers making staffing and training decisions.

This program provides a common language and framework to facilitate communication about industry skill needs among business, education, and the workforce investment system. The resulting O*NET database also is used to develop industry competency models and occupational competency profiles. O*NET data include information about transferable skills that are used for skills gap analysis, facilitating a mobile workforce responsive to changing economic needs. The O*NET database and companion O*NET Career Exploration Tools are used by many private companies and public organizations to tailor applications to their needs and those of their customers. The broad utility of the O*NET tools contributes to U.S. competitiveness in a global, 21st-century economy.¹

The O*NET Data Collection Program employs a multiple-method approach to updating the O*NET database. The primary method involves a two-stage sample design to survey establishments and workers in those establishments. When necessary, this method may be supplemented with a sample selected from additional sources, such as professional and trade association membership lists, resulting in a dual-frame approach. An alternative method, based on sampling from lists of identified occupation experts, is used for occupations for which the primary method is inefficient. This method is reserved for selected occupations, such as those with small employment scattered among many industries and those for which no employment data currently exist on which to base a sample, such as new and emerging occupations. The O*NET survey instruments are used with all methods.² The rest of Part A describes the O*NET Program and reviews statutory and regulatory information.

What Is the O*NET Program?

The O*NET Program is a comprehensive system for collecting and disseminating information on occupational and worker requirements. The O*NET data supersede the U.S. Department of Labor’s (DOL’s) Dictionary of Occupational Titles (DOT) and provide additional occupational requirements not available in the DOT. The DOT is no longer updated or maintained by DOL.

As shown in Exhibit 1, the O*NET Program uses a data structure, the Content Model, to organize occupational information and to provide a common language of standardized and defined occupation descriptors and measures for use by all audiences. The O*NET Content Model is the result of extensive research, and its development is fully documented (Peterson, Mumford, Borman, Jeanneret, & Fleishman, 1995, pp. 2–6; Peterson, Mumford, Borman, et al., 1997; Peterson et al., 2001). It comprises worker-oriented and job-oriented characteristics at both an occupation-specific level and across occupations, as the exhibit illustrates.

Each of the six domains of the Content Model groups information hierarchically. For example, the Worker Characteristics domain contains four types of information: Abilities, Occupational Interests, Work Values, and Work Styles. From these four, the Abilities domain, in turn, contains four types of abilities: Cognitive, Psychomotor, Physical, and Sensory. Each of these types of abilities contains further levels of detail. For example, the Psychomotor type includes Fine Manipulative, Control Movement, and Reaction Time and Speed. Finally, Fine Manipulative contains three specific descriptors: Arm-Hand Steadiness, Manual Dexterity, and Finger Dexterity. Hierarchies are a useful means of both organizing occupational information and allowing for its access at different levels of specificity. By organizing occupational information hierarchically, the O*NET taxonomies of information and SOC-based occupations facilitate the use of a common language to describe the world of work.

Exhibit 1. O*NET Content Model

The O*NET Data Collection Program takes the best knowledge about both work content and work analysis methodology from more than 70 years of research, since the first DOT. The descriptors and rating scales for O*NET data were developed through extensive research, drawing primarily from job analysis in industrial/organizational psychology and human resource management (Peterson et al., 1995). The descriptors in the O*NET Program are meant to be comprehensive. The primary sources of data are job incumbents and occupation experts. The SOC system is used as the basis for classifying occupations. The use of questionnaires and rating scales reflects the most widely accepted approach to job analyses conducted across settings, occupations, or positions. The scales used for the O*NET ratings are Importance, Level, and Frequency. Each descriptor in the O*NET questionnaires may use one or more scales. For example, the O*NET Work Activities descriptor Monitoring and Controlling Resources is rated on both a 5‑point Importance scale and a 7-point Level scale. For the complete set of O*NET questionnaires, which include O*NET descriptors, see Appendix A.

Exhibit 2 summarizes the number of descriptors and scales in the O*NET Data Collection Program questionnaires. Descriptors are identified from O*NET Content Model domains. Data are collected by means of 239 descriptors that include 400 scales (e.g., Importance, Level, and Frequency). Currently, to collect ratings for the Abilities and Skills domains, trained occupational analysts review updated information (e.g., Tasks, Generalized Work Activities) provided by job incumbents.³ No data collection is planned for the Workforce Characteristics domain. Information for it is provided through links to the employment, wage, and long-term projections databases produced by the U.S. Bureau of Labor Statistics (BLS), the state employment security agencies, and other agencies.

Exhibit 2. O*NET Data Collection Program Questionnaires

*ONET Data Collection Program Questionnaire**	Number of Descriptors	Number of Scales per Descriptor	Total Number of Scales	Data Source
Skills	35	2	70	Analysts
Knowledge	33	2	66	Job incumbents
Work Styles^a	16	1	16	Job incumbents
Education and Training^a	5	1	5	Job incumbents
Generalized Work Activities	41	2	82	Job incumbents
Work Context	57	1	57	Job incumbents
Abilities	52	2	104	Analysts
Tasks^b	Varies	2	Varies	Job incumbents
Total (not including Tasks)	239	NA	400	NA

Notes: Occupation experts use the same questionnaires as job incumbents for those occupations whose data collection is by the Occupation Expert Method. NA = not applicable.

^a The Knowledge Questionnaire packet also contains the Work Styles Questionnaire and the Education and Training Questionnaire.

^b All job incumbents are asked to complete a Task Questionnaire in addition to the domain questionnaire.

Versions of the O*NET Database

The first version of the O*NET database released to the public was O*NET 98. O*NET 98 moved from the DOT’s more than 12,000 occupations to a more user-friendly 1,122 occupations based on the BLS Occupational Employment Statistics (OES) codes. The O*NET 98 database contained 306 descriptors and 684 scales. A review of O*NET 98–specific scales and descriptors during the preparation for pretest data collection led to some consolidation and refinement of descriptors and scales to reduce burden on the public and increase employee response rate.⁴

The O*NET 98 database was first replaced with the O*NET 3.1 database and has been updated 15 times as new data have been collected and analyzed. The current database, O*NET version 19.0, contains the same descriptors used in O*NET 98; however, the occupations have been restructured and coded to encompass the most detailed level of the 2010 SOC, with more occupational specificity added as needed. Research is ongoing to identify additional new and emerging occupations in high-growth industries. New occupations emerge because of changes in technology, society, law, business practices, and markets. As these new and emerging occupations are identified and their data are collected, they will be integrated into the O*NET-SOC occupation classification and database.

O*NET 19.0 has a Web-based accessing application, O*NET OnLine, which is available to the public at no cost at http://online.onetcenter.org/. An electronic version of the 19.0 database can be downloaded at http://www.onetcenter.org/. The O*NET 19.0 database has been restructured to incorporate improvements made to the O*NET data collection instruments and is the structure currently being offered to developers.

Data in the O*NET database include the mean ratings on each of the items (or descriptors) in the O*NET questionnaires. Ratings have been standardized to facilitate interpretation and comparison. In addition to mean rating data on Level and Importance for various questionnaire items, text information is also included on occupational definitions, descriptor definitions, scale anchors, and task descriptions.

The Foundation for the O*NET Program

The DOT was first published by DOL in 1939 and provided the occupational classification and descriptions required for operation of the public employment service. Information for the DOT was collected primarily through observation and interview by trained occupational analysts. These collection techniques were applied to a small number of job incumbents from what can best be characterized as a convenience sample. The DOT has not been maintained or updated since 1991; many of its observations were conducted during the 1970s.

In the late 1980s the Employment and Training Administration (ETA) of DOL began a review of the DOT program. The purpose was to address concerns about the cost and difficulty of maintaining the DOT with existing methods and to address the need for additional and more current information. In 1990, as part of the DOT review, the Secretary of Labor appointed the Advisory Panel for the Dictionary of Occupational Titles. In response to its charge to advise on improvements to the DOT, this panel called for development of a new database of occupational information, a concept that evolved into the O*NET database. An important theme throughout the panel’s recommendations was the development of a common language about jobs, occupations, and skills. The panel’s complete report is presented in Appendix C.

The O*NET Data Collection Approach

The O*NET Data Collection Program is key to the continued effort to update the O*NET database to reflect changing skills requirements of occupations with the advent of new technologies and the changing world of work, with special emphasis on high-growth sectors of the economy and new and emerging occupations. In the research leading to the O*NET Data Collection Program, various sources and methods for collecting occupational information were examined, including collection of data from job incumbents and supervisors and development of ratings by occupation experts and occupational analysts. On the basis of this work, the O*NET team determined that the preferred source of data for most domains (Generalized Work Activities, Work Context, Knowledge, Education and Training, and Work Styles) is job incumbents. Other occupation experts, such as supervisors and trainers, may be used where access to job incumbents proves difficult or where the sampling of business establishments is inefficient.

Previous studies comparing various sources of job analysis ratings suggest that incumbents are able to provide information across a variety of descriptor domains (Fleishman & Mumford, 1988; Peterson, Owens-Kurtz, Hoffman, Arabian, & Whetzel, 1990). In addition, “large samples of knowledgeable job incumbents are available, which should contribute to the reliability of the resulting descriptive system” (Peterson, Mumford, Levin, Green, & Waksberg, 1999, pp. 2–6). Furthermore, the world of work is constantly changing, and technological advancements are occurring so rapidly that an efficient, effective way to remain current and accurate is to obtain the information directly from those performing the work.

By contrast, occupational analysts, who are provided with updated information from job incumbents, are preferred for the Abilities domain, which tends to be more abstract than the other domains. The Skills domain, whose variables are somewhat abstract, is a strong candidate for either source of collection; it is now updated by occupational analysts. A study conducted in 2006 found no clear evidence that one source of raters provides more valid or accurate data than the other for the Skills domain (Tsacoumis & Van Iddekinge, 2006).⁵ Consequently, considerations of relative practicality, such as cost, informed the decision to proceed with analyst ratings of both Abilities and Skills.

As part of a random sample, workers selected to participate in the O*NET Data Collection Program are asked to rate the requirements of their own jobs as defined by the O*NET questionnaire items. The responses are tabulated into statistics, such as mean ratings for each scale. Collecting information from job incumbents presents many challenges; among them is determining the best method for identifying a representative sample of job incumbents in each occupation. The Advisory Panel for the Dictionary of Occupational Titles had recommended using sampling techniques that would ensure the representativeness and accuracy of the occupational data. Sampling allows an estimate of the population. By contrast, no systematic sampling methods were used in the development of the original DOT. The O*NET Program is concerned, in particular, with identifying sampling approaches that minimize burden on employers and the public, achieving broad coverage of the workers in each occupation, ensuring acceptable response rates, and supporting overall cost-efficiency.

Three types of sampling frames are available for identifying samples of workers in each occupation: (1) lists of individual workers identified through professional and trade associations, licensing agencies, and unions; (2) households; and (3) employer establishments.

Identifying sampling frames of workers through professional and trade associations and unions retains the advantage of lower response burden because contacts with a sample of employers are replaced with contact with one or a few associations. Although it adds the cost of soliciting and maintaining association cooperation, this procedure also removes the cost of soliciting and maintaining employer cooperation. However, a major limitation of using special sampling frames of professional and trade associations and unions is coverage: rarely does association membership encompass a broad coverage of employment in the occupation. In addition, the membership of many associations consists of people in multiple occupations, retired individuals, and other interested parties. Moreover, few associations keep occupational information on their membership, causing the identification of job incumbents in a specific occupation to be problematic. Where coverage of employment in an occupation by association membership is significant and members of the occupation in the association can be identified, a special frame can sometimes be used to supplement the use of employer establishments in a dual-frame sample design.

Persons in various occupations could also be identified and interviewed using a sample of households. The primary disadvantage of this approach is cost. Because it is impossible to know what occupations would be represented by the employed members of a household, many households would have to be sampled and screened in order to identify and interview persons in specific occupations. As an example, consider that there are approximately 70,000 veterinarians in the United States and about 122 million households. This means that more than 1,700 households, on average, must be screened to find just one veterinarian. Some occupations are much rarer and would require screening thousands of households to locate and interview the required number for the O*NET survey. The cost of this approach would be prohibitive, and other, more economical options are available.

The Establishment Method (using employer establishments to identify occupational samples, as described further in Section A.1.3) provides the advantages of lower response burden and cost than a household survey, as well as good coverage for the large majority of occupations. Response burden and costs are lower for two primary reasons: (1) there are more workers per employer than per household, so fewer contacts are required to identify workers; and (2) employer contacts can be minimized by focusing on those most likely to employ workers in each occupation for which the sample is required. Efficient sample design is possible because the distribution of employment in an occupation is usually a function of the industry of the employer. Employment by occupation by industry is measured by the federal-state OES program national estimates provided by BLS.

The Establishment Method provides good coverage of wage and salary employment so long as an acceptable employer sampling frame is available. Coverage of self-employment is more difficult, although the sampling frame used in O*NET surveys includes many establishments operated by self-employed workers.

The Establishment Method, pretested in 1999 and 2000,⁶ remains the primary way to update the O*NET database; most data are currently collected this way. Achieving high response rates with the Establishment Method can be challenging, however, because the method requires cooperation at two levels: that of the employer and that of the sampled worker. Nonetheless, high levels of cooperation have been attained to date, and this method has proved successful. Although the resulting response rates have been acceptable, the O*NET team continually works to enhance response rates.⁷

An alternative method for collecting occupational information, involving occupation experts, is used to optimize the use of burden hours and resources because some occupations are difficult to sample efficiently. This situation occurs when it is difficult to locate industries or establishments with occupation incumbents; when employment is low; or when employment data are not available, as is the case for many new and emerging occupations. With the Occupation Expert (OE) Method, persons considered experts in the target occupation are surveyed. These experts include supervisors and trainers, as well as experienced job incumbents. The limitation of the OE Method is that locating experts can be difficult. For some occupations, identifying a professional association proves difficult; in other cases the association may lack membership information sufficient to identify experts for a specific occupation.

Using the most appropriate sources of information (e.g., workers, occupation experts, analysts) and a multiple-method approach, the O*NET Data Collection Program efficiently collects and yields high-quality occupational data.

Summary of the O*NET Data Collection Process

The O*NET data collection process is broadly summarized here and detailed later in separate sections of this Supporting Statement.

Sample Design

The O*NET Program sampling approaches are designed to create and update the O*NET database in a highly cost-efficient and timely manner while maximizing the reliability of the information in it. The primary method for collecting this information is the Establishment Method, a survey of workers employed in a national probability sample of establishments. Data collection for approximately 75% of occupations is completed by the Establishment Method. The method uses a stratified two-stage design. At the first stage, a sample of businesses is selected from a national database, provided by Dun & Bradstreet (D&B), of nearly 17 million establishments. The sample is selected with probability proportional to the expected number of employed workers in the specific occupations being surveyed. At the second stage, a sample of workers is selected in the occupations within the sampled businesses.

For selected occupations that are difficult to complete and for which additional observations are required, a special frame, such as a professional or trade association membership list, is sometimes used to supplement the D&B sample. The sample selection procedures vary across associations, depending on the type of information available on association members. In general, association lists are sampled with a single-stage, stratified, simple random sampling approach. Stratification by geographic location and occupation subspecialty is considered if it is appropriate for the occupation.⁸

The OE Method is considered for use when the Establishment Method would likely be problematic because the target occupations have very low employment rates, or because they are new or emerging, lack industry employment data, or are populated by incumbents in remote or difficult-to-access locations. The OE Method can be used only if the occupation is well represented by one or more professional or trade associations that are willing and able to identify experts in the target occupation. For this method, stratified samples of experts are selected from lists of potential respondents. These potential experts are questioned to determine whether they meet the program-specified criteria to serve as occupation experts for their respective occupations. Data collection for approximately 25% of the O*NET-SOC occupations is completed by the OE Method.⁹

Data Collection

Data collection operations are conducted by RTI International at its Operations Center in Raleigh, North Carolina, and at its Survey Support Department, also located in Raleigh. For the Establishment Method, the Operations Center’s Business Liaisons (BLs) contact sample business establishments, secure the participation of points of contact (POCs), and work with the POCs to carry out data collection in the target occupations. The data are provided by randomly selected employees in the occupations of interest. All within-establishment data collection is coordinated by the POCs; the BLs do not contact employees directly. After a POC agrees to participate, informational materials and questionnaires are mailed to the POC, who distributes the questionnaires to the sampled employees.

As noted above, for difficult-to-complete occupations, the D&B sample may be supplemented with a sample of workers selected from a professional or trade association membership list. Similarly, when the OE Method is used, occupation experts are also selected from professional or trade association lists. In both situations, the workers or occupation experts are contacted directly by the BLs, without involvement of a sampled establishment or a POC.

Survey support staff mail materials to POCs, job incumbents, and occupation experts, and they receive and process completed questionnaires returned by respondents. Both the telephone operations of the BLs and the mailing and questionnaire-receipt operations of the survey support staff are supported by a Case Management System (CMS). Data-entry software supports the keying and verification of incoming survey data.

Three domain questionnaires are used to collect data from sampled workers: Knowledge (including Work Styles and Education and Training), Generalized Work Activities, and Work Context. Each sampled worker is randomly assigned one of the three questionnaires. The workers are also asked to provide basic demographic information and to complete a brief task inventory for their specific occupations. By contrast, the occupation experts are asked to complete all three domain questionnaires, as well as basic demographic questions and a task inventory for the occupation of interest.

Workers may complete the paper questionnaire and return it by mail, or they may choose to complete the questionnaire online at the project Web site. Occupation experts have the same options for completing their questionnaires. Questionnaires are available in Spanish for selected O*NET-SOC occupations. Data for two domains, Abilities and Skills, are provided by trained analysts because of the more abstract nature of the questions.^¹⁰

Data Cleaning; Identification and Evaluation of Anomalous Cases

Data cleaning procedures eliminate completely blank questionnaires and insert consistent analysis codes for legitimate skips, blank items, and invalid responses. Anomalous cases are identified so respondents may be removed whose responses either suggest that they are not working in the occupation of interest or are highly inconsistent with those of the others responding for the occupation. Unusable cases are identified according to prescribed eligibility criteria, such as percentage of items completed. Cases with certain questionable characteristics are flagged for further analysis. These include cases with response patterns deviating from those of other cases in the occupation and cases with write-in job titles that do not appear to match the occupation. Responses judged invalid by expert reviewers are excluded from the analysis file.¹¹

Weighting and Estimation

Estimates generated from O*NET survey data are computed with sampling weights that compensate for the unequal probabilities of selecting establishments, occupations within establishments, and employees within each selected occupation. In addition, these base weights are adjusted to further compensate for multiple sub-waves of sampling, sample adjustment, population under- and over-coverage caused by frame imperfections, and nonresponse at both the establishment and the employee levels.

These weight adjustments can lead to weights that are very large or very small compared with the weights for other sample units. Such weight variability may increase the standard error estimates. When the variation in the weights is large, it is desirable to trim the weights to reduce the variation. For the O*NET estimates, the weighting process involves a weight trimming procedure in which extremely large or small weights are truncated to fall within a specified range. Although trimming weights can introduce bias in the estimates, the variance reduction it achieves usually offsets the potential bias, resulting in estimates with smaller net mean squared errors.

Based on a pooled sample of all completed waves, final estimates are produced. Estimates are computed by summing the weighted observations and dividing by the sum of the weights. Standard errors are estimated with the first-order Taylor series approximation of deviations of estimates from their expected values. These design-based variance estimates are computed with SUDAAN^® software (RTI International, 2004). These estimates properly account for the combined effects of clustering, stratification, and unequal weighting—all of which are present in the O*NET data. In addition, estimates with questionable precision are flagged “recommended for suppression” in the O*NET database.¹²

Analysis

Nonresponse is analyzed at multiple levels. Establishment nonresponse in the O*NET Data Collection Program can occur at the verification, screening, recruiting, or sampling stages of selection. Employee-level nonresponse occurs when a selected employee fails to complete and return a questionnaire. Because these same characteristics of establishments and employees were used to adjust the analysis weights, compensation is made to further reduce the size and impact of any nonresponse bias. In addition, item-level nonresponse is assessed, and item response rates are high—generally higher than 90% for Likert scale items, with most exceeding 95%.

For each O*NET-SOC occupation, the degree of interrater reliability (the co-variation among ratings) and the level of inter-rater agreement (the absolute difference among ratings) is calculated annually. The results of the analyses are used to examine the potential sources of variability across respondents in a specific occupation. As part of a continuous improvement process, these results also inform an evaluation of the O*NET-SOC occupational taxonomy, Content Model descriptors, and scales.¹³

Summary of Response Rate Experience to Date

Establishment Method

Data collection for the O*NET Data Collection Program began in June 2001 and has been in continuous operation since then. Exhibit 3 shows our cumulative response rate experience as of December 31, 2014.

Exhibit 3. Establishment Method Data Collection Results

Sampled establishments	260,467
Eligible establishments	204,544
Participating establishments	155,304
Establishment response rate (participating establishments/eligible establishments)	75.9%
Eligible employees	294,808
Participating employees	191,573
Employee response rate (participating employees/eligible employees)	65.0%

As indicated, more than 155,000 establishments and 191,000 employees have responded to the survey request, resulting in an establishment response rate of 76% and an employee response rate of 65%. Although it is difficult to make response rate comparisons between the O*NET Data Collection Program and other establishment surveys, these rates compare favorably with those of other establishment surveys.

Some of the design characteristics that affect the response rate of the O*NET Program follow:

Voluntary rather than mandatory participation. While past experiments conducted by the U.S. Bureau of the Census suggest that simply informing respondents that their response is mandatory adds about 20 percentage points to overall response (Tulp, Hoy, Kusch, & Cole, 1991; Worden & Hoy, 1992), more recent analyses conducted on data from the American Community Survey found that response rates on mandatory surveys would typically be a more modest 5.8 percentage points higher than that of their voluntary counterparts (Navarro, King, & Starsinic, 2011). Because the survey is voluntary, O*NET response rates would be expected to be 6 to 10 percentage points lower than those for the average federally mandated survey.
No direct personal contact with the survey respondents by the survey organization conducting the data collection. The O*NET Program requirement of respondent anonymity means that participation at the employee level relies exclusively on the interactions between the POC and the employee. The survey organization is not allowed to speak to the employee to respond to questions, motivate responses, or follow up on noncompliance. In their review of establishment mail survey response rates, Paxon, Dillman, and Tarnai (1995) found that establishment surveys featuring anonymous mailings typically have lower response rates (by as many as 30 percentage points) than surveys featuring direct personal contact with the respondents.
Participation required at three stages of response—establishment level, point of contact level, and employee level. The typical establishment survey requires participation at only one or two levels: the establishment level and, in some cases, the POC level. By contrast, for the O*NET Program three often distinct entities must agree to participate: the establishment administration, the POC identified in the screening interview, and the employee who is asked to complete the questionnaire. Because very few surveys incorporate such a design, survey methods literature is essentially devoid of examples on which to base a reasonable response rate expectation for the O*NET Data Collection Program. However, it is possible to compare O*NET response rates at each stage with other establishment surveys that incorporate these stages either separately or in combination. For example, the O*NET establishment-level response rate can be compared with other mail establishment surveys having only one response stage at the establishment level. In addition, the O*NET employee-level response rate can be compared with the response rate of other establishments’ self-conducted employee surveys.

A high level of nonresponse is a threat to the accuracy of the estimates. For this reason, the O*NET data collection protocol (described in Sections ‎B.2 and B.3 in Part B) was designed to achieve the highest response rates possible within respondent burden and survey cost constraints. Tarnai & Paxon (2004) acknowledge the historically low response rates to business surveys; while responses can fall as low as 10% of the organizations surveyed, the typical data collection process achieves response rates between 40% and 50%. Their study of 2,626 businesses on survey mode preference yielded a response rate 48.6%. Baruch and Holtom (2008) analyzed the response rates for 463 studies published across 17 first- and second-tier journals in 2000 and 2005. Among the studies examined, 117 of them were organization-level surveys whose average response rate was 35.7%.

In a more recent review of four major, voluntary, establishment-based surveys conducted by BLS, initial unweighted data collection response rates (defined as the percentage of sampled establishments that agreed to provide any of the requested data) ranged from 66% to 87% (Petroni, Sigman, Willimack, Cohen, & Tucker, 2004). Additionally, the Census Bureau’s Monthly Retail Sales Survey and Monthly Wholesale Survey, both of which are voluntary mail surveys with a telephone follow-up, show cooperation rates ranging between 66% and 80% (W. Davie, personal communication, March 23, 2011). The 2001 Survey of Respirator Use in Private Sector Firms, a voluntary mail survey of 40,002 establishments sponsored by the National Institute of Occupational Safety and Health, used a two-tiered sampling process similar to the one used on O*NET and achieved a business-level response rate of 75.5% (U.S. Department of Labor, Bureau of Labor Statistics, and U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, 2003). The above results for establishment surveys, summarized in Exhibit 4, are highly comparable to the O*NET establishment-level response rate of 76%.

Exhibit 4. Studies of Establishment-Level Response Rates

Study	Response Rate
Tarnai & Paxton (2004)	49%
Baruch & Holtom (2008)	36%
Petroni et al. (2004)	66%–87%
W. Davie (2011)	66%–80%
National Institute of Occupational Safety and Health (2003)	76%
*ONET**	76%

Comparisons of the O*NET employee response rate can be made with surveys that directly sample employees within an establishment. Because federally sponsored surveys of employees within organizations are rare, the literature on their response rates is sparse. Most surveys of this type are employee satisfaction surveys. For example, one well-documented, government-sponsored survey of employees is the Public Service Employee Survey, administered to about 258,000 employees of the Public Service of Canada and conducted by Statistics Canada in 2008 (Treasury Board of Canada Secretariat, 2009). A questionnaire was delivered to each employee by a government agent who personally requested that the employee complete the questionnaire and return it by mail. Multiple follow-ups of non-respondents were made by e-mail and interoffice mail to maximize the response rate. No incentive was used, however; because all sample members were also employees of the organization conducting the survey and could fill out the survey on government time, the use of an incentive was thought to be unnecessary. The final overall response rate for the survey was 66%.

The Office of Personnel Management (OPM) developed the Organizational Assessment Survey (OAS) and has encouraged all federal agencies to survey their employees in order to evaluate organizational performance, benchmark best practices, and align performance with important and measurable outcomes. The experience OPM has had in implementing these surveys in numerous federal agencies provides some evidence of response rates for employee surveys conducted by the U.S. government. The OAS design closely resembles the Canadian Public Service Employee Survey design. The surveys are self-administered and are conducted by each agency for its own employees. Furthermore, like the Public Service Employee Survey, the OAS request to participate is personalized and made directly to the employee by his or her employer. The features of the design offer a significant advantage over the O*NET survey design, as previously noted.

Although the results of the OAS surveys are not publicly available, an official at OPM was able to provide some general information regarding OAS response rates (C. Simons, personal communication, March 21, 2002). According to OPM, response rates for OAS surveys vary considerably by agency, from 30% to 80%. However, the average response across all agencies is approximately 57%.

The Employee Viewpoint Survey (L. Pollack, personal communication, March 23, 2011), a biennial census administered to hundreds of thousands of full-time federal employees across U.S. government agencies, most recently yielded response rates of 49.3% in 2011, 46.1% in 2012, and 48.2% in 2013 (U.S. Office of Personnel Management, 2010).

Additionally, Anseel, Lievens, Schollaert, and Choragwicka (2010), who conducted a meta-analysis of 2,037 surveys in the field of industrial and organizational psychology, management, and marketing from 1995 through 2008, found that surveys implementing many of the response-rate-enhancing features of the O*NET survey had an average response rate of 52%. In a study examining implications of employees’ mode preference in completing a survey, Cole, Bedeian, and Feild (2006) sampled 8,598 employees across 50 countries who worked in the manufacturing industry and who varied in age, job tenure, and job functions. The study offered targeted respondents the same survey mode options as are given in O*NET: a paper-and-pencil version or a Web-based option. The overall response rate of 57% suggests that the O*NET employee response rate of 65% exceeds expectations for mail surveys of employees within their organizations. The above results for employee surveys are summarized in Exhibit 5.

Exhibit 5. Studies of Employee-Level Response Rates

Study	Response Rate
Treasury Board of Canada Secretariat (2009)	66%
OPM (2002)	57%
OPM (2013)	46%–49%
Anseel et al. (2010)	52%
Cole et al. (2006)	57%
O*NET	65%

Although results from the survey methods literature and from other federal surveys summarized in Exhibits 4 and 5 suggest that a 76% establishment response rate and a 65% employee response rate exceed expectations for federal surveys that use similar data collection approaches, methods for further improving response rates will continue to be explored.¹⁴

Occupation Expert Method

The OE Method is a much smaller but still important component of the O*NET Data Collection Program protocol. Exhibit 6 shows our response rate experience with this method as of December 31, 2014.

Exhibit 6. Occupation Expert Method Data Collection Results

Sampled occupation experts	15,092
Eligible occupation experts	11,448
Participating occupation experts	8,944
Occupation expert response rate (participating occupation experts/eligible occupation experts)	78.1%

As indicated, 8,944 of 11,448 eligible OEs have participated, for a response rate of 78%. This response rate is higher than the employee response rate for the Establishment Method and may be due to factors such as personalized direct contact, greater total incentive, and generally higher education levels of the eligible population.

Statutory and Regulatory Information

Although the O*NET name is not referenced specifically in statutes, it is cited at least eight times in the U.S. Code of Federal Regulations (C.F.R.), and the DOT—which O*NET largely replaced—is cited 24 times in the C.F.R. (the DOT was formerly cited in the statutes, but there are no current statutory references to it). Furthermore, O*NET information is the foundational common language for fulfilling the statutory and regulatory responsibilities regarding information on skills discussed in this section.

Section 308 of the Workforce Innovation and Opportunity Act (WIOA; Public Law 113-128 amended section 15 of the Wagner-Peyser Act to require the Secretary of Labor to oversee the “development, maintenance, and continuous improvement of a nationwide workforce and labor market information system,” which shall include, among other components, “skill trends by occupation and industry.” See 29 U.S.C. 49l-1. The O*NET program is the primary means for collecting skills information across all occupations in the economy. Updating the entire O*NET database is a critical component of the nationwide workforce and labor market information system to support employer, workforce, and education information needs.

The WIOA contains numerous references that identify the need for information on the skill requirements of jobs. The word “skill” is used more than 130 times and there are three references to “knowledge, skills, and abilities” and two to “knowledge, skills, and competencies.” For example, Section 102 requires the Unified State Plan to include an economic analysis of “(i) existing and emerging in-demand industry sectors and occupations and (ii) the employment needs of employers, including a description of the knowledge, skills, and abilities, needed in those industries and occupations.” See 29 U.S.C. § 3112. Section 134 requires the provision of “information on job skills necessary” and on “skill requirements” for obtaining jobs listed for the local occupations in demand. See 28 U.S.C. 3174 § (c)(2)(A)(vi)(ii).

Other WIOA references address the need for information on the skills of individuals. For example, Section 134 provides for assessment services to identify “the skill levels and service needs of adults and dislocated workers.” Section 129 allows funds to be used to provide youth with an assessment which “shall include a review of basic skills, occupational skills, prior work experience, employability, interests, [and] aptitudes.” See 29 U.S.C. § 3164(c)(1)(A). The O*NET Career Exploration Tools, including the O*NET Interest Profiler and O*NET Work Importance Locator, are such assessment tools, designed specifically to relate a person’s interests and work values to the information on education and skill requirements for occupations in the O*NET database.

WIOA Section 303 on the Public Labor Exchange Services System amends the Wagner-Peyser Act to add “The Secretary, in consultation with States, is authorized to assist the States in the development of national electronic tools that may be used to improve access to workforce information for individuals….” The suite of O*NET Web sites (such as O*NET OnLine and My Next Move) and O*NET Web services used to disseminate O*NET occupational information and related workforce and labor market information are such national electronic tools designed to improve access to information for individuals.

Finally, the predecessor to the O*NET database, the DOT, is frequently cited as a source of occupational information in support of federal programs. The 27 citations of the DOT in the C.F.R. include references to determining disability, administering DOL programs, and administering immigration, civil rights, and labor standards law. DOL officials responsible for the O*NET Program work with various federal users of the DOT, some of whom have made the transition either in regulatory changes or in practices and procedures, while others are exploring potential transition to O*NET information or uses of O*NET data. These include State Department officials responsible for visas, the U.S. Office of Apprenticeship, and the Office of Foreign Labor Certification. The specific C.F.R. citations appear in Exhibit 7.

Exhibit 7. O*NET Citations in Code of Federal Regulations

1.	20 CFR 651.10 - Definitions of terms used in parts 651-658. [PDF 101 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Part 651: GENERAL PROVISIONS GOVERNING THE FEDERAL-STATE EMPLOYMENT SERVICE SYSTEM. Tuesday, April 1, 2014. ... Occupational Information Network (O*NET) means the online reference database...work styles, and work values. O*NET-SOC means Standard Occupational...calculating and disseminating data. DOL uses O*NET-SOC titles and codes for the... More Information Historical editions have been hidden from results. Show all editions.
2.	20 CFR 656.3 - Definitions, for purposes of this part, of terms used in this part. [PDF 97 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart A: Purpose and Scope of Part 656. Tuesday, April 1, 2014. ...and Nationality Act, as amended. O*NET means the system developed by the...levels associated with occupations. O*NET is based on the Standard Occupational...system. Further information about O*NET can be found at... More Information Historical editions have been hidden from results. Show all editions.
3.	22 CFR 42.33 - Diversity immigrants. [PDF 112 KB] Code of Federal Regulations (annual edition). Title 22: Foreign Relations. Subpart D: Immigrants Subject to Numerical Limitations. Monday, April 1, 2013. ...and Diversity Visa Programs occurring in subsequent fiscal years, consular officers must use the Department of Labor's O*Net On Line to determine qualifying work experience. (4) Limitation on number of petitions per year. No more... More Information Historical editions have been hidden from results. Show all editions.
4.	20 CFR 655.1308 - Offered wage rate. [PDF 91 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart N: Labor Certification Process for Temporary Agricultural Employment in the United States (H-2A Workers). Tuesday, April 1, 2014. ...experience that are generally required as described in the O*NET Job Zones. (3) Level III wage rates are assigned...educational degrees that are at the higher ranges indicated in the O*NET Job Zones would be an indicator that a Level III... More Information Historical editions have been hidden from results. Show all editions.
5.	20 CFR 653.103 - MSFW job applications. [PDF 90 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart B: Services for Migrant and Seasonal Farmworkers (MSFWs). Tuesday, April 1, 2014. ...ES-511). Additional Occupational Informational Network (O*NET) codes or keywords shall be assigned, where appropriate...does not require completion of secondary cards, additional O*NET-SOC codes shall be noted on the primary application... More Information Historical editions have been hidden from results. Show all editions.
6.	34 CFR 668.6 - Reporting and disclosure requirements for programs that prepare students for gainful employment... [PDF 93 KB] Code of Federal Regulations (annual edition). Title 34: Education. Subpart A: General. Monday, July 1, 2013. ...to enter, along with links to occupational profiles on O*NET or its successor site. If the number of occupations related...entering the program's full six digit CIP code on the O*NET crosswalk at... More Information Historical editions have been hidden from results. Show all editions.
7.	20 CFR 656.17 - Basic labor certification process. [PDF 103 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart C: Labor Certification Process. Tuesday, April 1, 2014. ...the occupation and must not exceed the Specific Vocational Preparation level assigned to the occupation as shown in the O*NET Job Zones. To establish a business necessity, an employer must demonstrate the job duties and requirements bear a... More Information Historical editions have been hidden from results. Show all editions.
8.	34 CFR 600.2 - Definitions. [PDF 96 KB] Code of Federal Regulations (annual edition). Title 34: Education. Subpart A: General. Monday, July 1, 2013. ...Occupational Classification (SOC) code established by the Office of Management and Budget or an Occupational Information Network O*NET-SOC code established by the Department of Labor and available at http://online.onetcenter.org or its... More Information Historical editions have been hidden from results. Show all editions.
9.	20 CFR 404.1560 - When we will consider your vocational background. [PDF 91 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...may use the services of vocational experts or vocational specialists, or other resources, such as the “Dictionary of Occupational Titles” and its companion volumes and supplements, published by the Department of Labor, to obtain... More Information Historical editions have been hidden from results. Show all editions.
10.	20 CFR 404.1566 - Work which exists in the national economy. [PDF 87 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...available from various governmental and other publications. For example, we will take notice of— (1) Dictionary of Occupational Titles, published by the Department of Labor; (2) County Business Patterns, published by the Bureau... More Information Historical editions have been hidden from results. Show all editions.
11.	20 CFR 220.13 - Establishment of permanent disability for work in regular railroad occupation. [PDF 90 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart C: Disability Under the Railroad Retirement Act for Work in an Employee's Regular Railroad Occupation. Monday, April 1, 2013. ...employer on the appropriate form set forth in appendix 3 of this part, and consult other sources such as the Dictionary of Occupational Titles and the job descriptions of occupations found in the Occupational Disability Claims Manual, as provided... More Information Historical editions have been hidden from results. Show all editions.
12.	20 CFR 220.135 - Exertional and nonexertional limitations. [PDF 90 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational Considerations. Monday, April 1, 2013. ...Board uses the classification of jobs by exertional levels (strength demands) which is contained in the Dictionary of Occupational Titles published by the Department of Labor, to determine the exertional requirements of work which exists... More Information Historical editions have been hidden from results. Show all editions.
13.	20 CFR Appendix 2 to Subpart P of... - Medical-Vocational Guidelines [PDF 113 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart P: Determining Disability and Blindness. Tuesday, April 1, 2014. ...various functional levels (sedentary, light, medium, heavy, and very heavy) as supported by the “Dictionary of Occupational Titles” and the “Occupational Outlook Handbook,” published by the Department of Labor; the “County Business... More Information Historical editions have been hidden from results. Show all editions.
14.	20 CFR 416.967 - Physical exertion requirements. [PDF 87 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...sedentary, light, medium, heavy, and very heavy. These terms have the same meaning as they have in the Dictionary of Occupational Titles, published by the Department of Labor. In making disability determinations under this subpart, we... More Information Historical editions have been hidden from results. Show all editions.
15.	20 CFR 655.730 - What is the process for filing a labor condition application? [PDF 97 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart H: Labor Condition Applications and Requirements for Employers Seeking To Employ Nonimmigrants on H-1b Visas in Specialty Occupations and as Fashion Models, and Requirements for Employers Seeking To Employ Nonimmigrants on H-1b1 and E-3 Visas in Specialty Occupations. Tuesday, April 1, 2014. ...occupational classification for which the LCA is being submitted and shall state: (i) The occupation, by Dictionary of Occupational Titles (DOT) Three-Digit Occupational Groups code and by the employer's own title for the job; (ii)... More Information Historical editions have been hidden from results. Show all editions.
16.	20 CFR 220.131 - Work which exists in the national economy. [PDF 90 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational Considerations. Monday, April 1, 2013. ...Board determines that unskilled, sedentary, light and medium jobs exist in the national economy: (1) Dictionary of Occupational Titles, published by the Department of Labor. (2) County Business Patterns, published by the Bureau... More Information Historical editions have been hidden from results. Show all editions.
17.	20 CFR 220.134 - Medical-vocational guidelines in appendix 2 of this part. [PDF 87 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational Considerations. Monday, April 1, 2013. ...Vocational Considerations § 220.134 Medical-vocational guidelines in appendix 2 of this part. (a) The Dictionary of Occupational Titles includes information about jobs (classified by their exertional and skill requirements) that exist in... More Information Historical editions have been hidden from results. Show all editions.
18.	20 CFR 416.960 - When we will consider your vocational background. [PDF 91 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...may use the services of vocational experts or vocational specialists, or other resources, such as the “Dictionary of Occupational Titles” and its companion volumes and supplements, published by the Department of Labor, to obtain... More Information Historical editions have been hidden from results. Show all editions.
19.	20 CFR 655.730 - What is the process for filing a labor condition application? [PDF 97 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart H: Labor Condition Applications and Requirements for Employers Seeking To Employ Nonimmigrants on H-1b Visas in Specialty Occupations and as Fashion Models, and Requirements for Employers Seeking To Employ Nonimmigrants on H-1b1 and E-3 Visas in Specialty Occupations. Tuesday, April 1, 2014. ...occupational classification for which the LCA is being submitted and shall state: (i) The occupation, by Dictionary of Occupational Titles (DOT) Three-Digit Occupational Groups code and by the employer's own title for the job; (ii)... More Information Historical editions have been hidden from results. Show all editions.
20.	20 CFR 220.131 - Work which exists in the national economy. [PDF 90 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational Considerations. Monday, April 1, 2013. ...Board determines that unskilled, sedentary, light and medium jobs exist in the national economy: (1) Dictionary of Occupational Titles, published by the Department of Labor. (2) County Business Patterns, published by the Bureau... More Information Historical editions have been hidden from results. Show all editions.
21.	20 CFR 220.134 - Medical-vocational guidelines in appendix 2 of this part. [PDF 87 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational Considerations. Monday, April 1, 2013. ...Vocational Considerations § 220.134 Medical-vocational guidelines in appendix 2 of this part. (a) The Dictionary of Occupational Titles includes information about jobs (classified by their exertional and skill requirements) that exist in... More Information Historical editions have been hidden from results. Show all editions.
22.	20 CFR 416.960 - When we will consider your vocational background. [PDF 91 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...may use the services of vocational experts or vocational specialists, or other resources, such as the “Dictionary of Occupational Titles” and its companion volumes and supplements, published by the Department of Labor, to obtain... More Information Historical editions have been hidden from results. Show all editions.
23.	20 CFR 416.966 - Work which exists in the national economy. [PDF 91 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...available from various governmental and other publications. For example, we will take notice of— (1) Dictionary of Occupational Titles, published by the Department of Labor; (2) County Business Patterns, published by the Bureau... More Information Historical editions have been hidden from results. Show all editions.
24.	20 CFR 655.10 - Determination of prevailing wage for temporary labor certification purposes. [PDF 97 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Prefiling Procedures. Tuesday, April 1, 2014. ...DBA, or SCA wage available for the job opportunity; (B) The job opportunity was not listed in the Dictionary of Occupational Titles (DOT) and is not listed in the Standard Occupational Classification (SOC) system, or if the job... More Information Historical editions have been hidden from results. Show all editions.
25.	41 CFR 60-3.15 - Documentation of impact and validity evidence. [PDF 109 KB] Code of Federal Regulations (annual edition). Title 41: Public Contracts and Property Management. Subjgrp: Documentation of Impact and Validity Evidence. Monday, July 1, 2013. ...s) from U.S. Employment Service's Dictionary of Occupational Titles. (5) Criterion measures. The bases...the United States Employment Service's dictionary of occupational titles. (6) Selection procedure. The... More Information Historical editions have been hidden from results. Show all editions.
26.	20 CFR 220.132 - Physical exertion requirements. [PDF 90 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational Considerations. Monday, April 1, 2013. ...medium”, “heavy”, and “very heavy.” These terms have the same meaning as they have in the Dictionary of Occupational Titles, published by the Department of Labor. In making disability determinations the Board uses the... More Information Historical editions have been hidden from results. Show all editions.
27.	20 CFR Appendix 2 to Part 220 - Medical-Vocational Guidelines [PDF 113 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Part 220: DETERMINING DISABILITY. Monday, April 1, 2013. ...various functional levels (sedentary, light, medium, heavy, and very heavy) as supported by the “Dictionary of Occupational Titles” and the “Occupational Outlook Handbook,” published by the Department of Labor; the “County Business... More Information Historical editions have been hidden from results. Show all editions.
28.	20 CFR 416.969 - Listing of Medical-Vocational Guidelines in appendix 2 of subpart P of part 404 of this chapter. [PDF 87 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...Listing of Medical-Vocational Guidelines in appendix 2 of subpart P of part 404 of this chapter. The Dictionary of Occupational Titles includes information about jobs (classified by their exertional and skill requirements) that... More Information Historical editions have been hidden from results. Show all editions.
29.	20 CFR 416.969a - Exertional and nonexertional limitations. [PDF 91 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...how we use the classification of jobs by exertional levels (strength demands) which is contained in the Dictionary of Occupational Titles published by the Department of Labor, to determine the exertional requirements of work which exists... More Information Historical editions have been hidden from results. Show all editions.
30.	29 CFR 553.103 - “Same type of services” defined. [PDF 87 KB] Code of Federal Regulations (annual edition). Title 29: Labor. Subpart B: Volunteers. Monday, July 1, 2013. ...the duties and other factors contained in the definitions of the 3-digit categories of occupations in the Dictionary of Occupational Titles in determining whether the volunteer activities constitute the “same type of services” as the... More Information Historical editions have been hidden from results. Show all editions.
31.	28 CFR 50.14 - Guidelines on employee selection procedures. [PDF 171 KB] Code of Federal Regulations (annual edition). Title 28: Judicial Administration. Part 50: STATEMENTS OF POLICY. Monday, July 1, 2013. ...s) from U.S. Employment Service's Dictionary of Occupational Titles. (5) Criterion measures. The bases...the United States Employment Service's dictionary of occupational titles. (6) Selection procedure. The... More Information Historical editions have been hidden from results. Show all editions.
32.	20 CFR 404.1567 - Physical exertion requirements. [PDF 91 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...sedentary, light, medium, heavy, and very heavy. These terms have the same meaning as they have in the Dictionary of Occupational Titles, published by the Department of Labor. In making disability determinations under this subpart, we... More Information Historical editions have been hidden from results. Show all editions.
33.	29 CFR 1607.15 - Documentation of impact and validity evidence. [PDF 109 KB] Code of Federal Regulations (annual edition). Title 29: Labor. Subjgrp: Documentation of Impact and Validity Evidence. Monday, July 1, 2013. ...s) from U.S. Employment Service's Dictionary of Occupational Titles. (5) Criterion measures. The bases...the United States Employment Service's dictionary of occupational titles. (6) Selection procedure. The... More Information Historical editions have been hidden from results. Show all editions.
34.	20 CFR 404.1569 - Listing of Medical-Vocational Guidelines in appendix 2. [PDF 91 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...Vocational Considerations § 404.1569 Listing of Medical-Vocational Guidelines in appendix 2. The Dictionary of Occupational Titles includes information about jobs (classified by their exertional and skill requirements) that... More Information Historical editions have been hidden from results. Show all editions.
35.	20 CFR 404.1569a - Exertional and nonexertional limitations. [PDF 90 KB] Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subjgrp: Vocational Considerations. Tuesday, April 1, 2014. ...how we use the classification of jobs by exertional levels (strength demands) which is contained in the Dictionary of Occupational Titles published by the Department of Labor, to determine the exertional requirements of work which exists... More Information Historical editions have been hidden from results. Show all editions.
36.	29 CFR 553.30 - Occasional or sporadic employment-section 7(p)(2). [PDF 90 KB] Code of Federal Regulations (annual edition). Title 29: Labor. Subjgrp: Other Exemptions. Monday, July 1, 2013. ...the duties and other factors contained in the definitions of the 3-digit categories of occupations in the Dictionary of Occupational Titles (except in the case of public safety employees as discussed below in section (3)), as well as... More Information Historical editions have been hidden from results. Show all editions.

The search for O*NET and the DOT in the most recent editions of the C.F.R. was performed on July 21, 2014, using the U.S. Government Printing Office’s Federal Digital System at www.gpo.gov/fdsys/search/home.action. Eight O*NET citations (the first eight rows of the table) were identified in the C.F.R., along with 28 DOT citations (the remaining rows).

Secretary of Labor’s Strategic Plan

The Secretary of Labor’s Strategic Plan 2014–2018 encompasses O*NET within Strategic Goal 1: Prepare workers for better jobs. O*NET also addresses the following supporting Strategic Objectives under Strategic Goal 1.

Strategic Objective 1.1: Advance employment opportunities for U.S. workers in 21^st century demand sectors and occupations using proven training models and through increased employer engagement and partnerships.
Strategic Objective 1.2: Provide marketable skills and knowledge to increase workers’ incomes and help them overcome barriers to the middle class through partnerships among business, education, labor, community organizations, and the workforce system.

The purpose of the proposed data collection effort is to keep O*NET information on occupational skills and other requirements and characteristics current, which is key to supporting the needs of customers through the wide variety of public- and private-sector products that depend on the availability, quality, and currency of O*NET information. O*NET supplies the common and comprehensive language on skills, knowledge, abilities, and other work requirements and characteristics for all occupations throughout the economy.

Uses, Products, and Services Based on the O*NET Program

The O*NET Program provides essential tools and services for numerous critical federal and state workforce investment functions. These tools also serve job seekers, businesses, and educational institutions across the country. The common language used in O*NET occupational and skill descriptions facilitates communication among the various user groups. This facilitation contributes to our nation’s talent development and promotes U.S. competitiveness in the global 21st-century economy.

Users of the O*NET database can access and use the information in various ways. The data are available to the public free of charge through the O*NET OnLine, My Next Move, and My Next Move for Veterans Web sites; through the O*NET Web Services application programming interface (API); or by downloading the database, which is done by developers who provide applications targeted to specific communities or audiences. The O*NET database is now the foundation for many programs serving the American workforce, providing information to build transferable skills, skills gap analyses, and competency profiles and to facilitate cross-functional team building. These examples are just a few of the ways the O*NET Program supports activities critical to maintaining a mobile workforce responsive to changing regional and national economic needs.

The O*NET Database, O*NET OnLine, My Next Move, O*NET Career Tools, O*NET Training Academy, and O*NET Code Connector

The O*NET Database

With the July 2014 version of the database, 940 occupations have been comprehensively updated through the O*NET Data Collection Program; 509 of these occupations have had more than one update. The O*NET database currently provides:

detailed occupational and skill information for more than 940 occupations;
information on standardized descriptors of skills, abilities, interests, knowledge, work values, education, training, work context, and work styles; and
occupational coding based on the 2010 SOC.

Since September 2007, there have been seven database updates as listed in Exhibit 8.

Exhibit 8. Database Updates

Update	Number of Occupations Updated
June 2008	108
June 2009	117
June 2010	120
July 2011	107
July 2012	108
July 2013	105
July 2014	126

Recent enhancements of the O*NET database include identification of and information about “green” and new and emerging occupations.¹⁵ Recent database updates are significantly increasing coverage of these occupations.

O*NET OnLine

The O*NET database is provided free of charge to the public through O*NET OnLine, a Web-based application at http://www.onetonline.org/. O*NET OnLine offers users multiple ways to search for O*NET occupations and related data. O*NET OnLine includes:

regularly updated occupational information;
tiered search algorithms that maximize successful results of searches by keyword (title), occupational code, or partial code;
O*NET equivalents of occupations listed in other occupational systems, such as the Military Occupational Classification or the Classification of Instructional Programs;
the ability to browse by O*NET descriptor (this search enables users to make cross-occupational comparisons by viewing an occupation’s rank order on a selected knowledge, skill, ability, work activity, interest, or work value variable);
the ability to browse groups of similar occupations, including bright outlook¹⁶; career clusters; green economy sectors; industry; or science, technology, engineering, and mathematics (STEM) disciplines;
a sample of reported job titles for each occupational report, providing the user with a broader understanding of the O*NET-SOC;
report display options (besides the default Summary Report, users can choose to view a comprehensive Details Report or build a Custom Report);
inclusion of wages and employment trends information (derived from BLS data) and links to Web sites of professional associations, giving the user access to additional information on related specialties, industries, and education and training resources; and
occupation-specific links to find training, certifications, licensing, and apprenticeship information.

My Next Move

My Next Move (http://www.mynextmove.org/) is a new, easy-to-use, easy-to-read Web-based interactive tool for new job seekers, students, and other career explorers to learn more about their career options. Users can explore more than 900 different careers and see important information about them, including skills, tasks, salaries, and employment outlook. They can also look at related apprenticeships and training and search actual job openings. They can find careers by searching keywords, by browsing industries, or by using the O*NET Interest Profiler (see below). Additionally, users can browse occupations by bright outlook, green occupations, apprenticeship, and amount of job preparation required.

Career reports in My Next Move feature the most important knowledge, skills, and abilities needed to perform the work, explained in language that is easy to understand. Outlook and education sections let users find salary information, job postings, and training opportunities. The visual design enables users to identify a career’s key points or explore a career in depth.

Visitors interested in specific careers can start exploring quickly with an intuitive keyword search; more than 900 career options are only a few keystrokes away. Users looking for a broader range of opportunities can explore more than a dozen different industries; each features a range of careers to choose from, including those in the green economy and with a bright outlook for job opportunities.

Also at the site is a Web-based version of the popular O*NET Interest Profiler, a tool designed to assess an individual’s vocational interests. The Web-based version of the tool features 60 items that, along with information about the user’s education and work experience, guide users to careers they may enjoy. More information about the O*NET Interest Profiler–Short Form can be found at http://www.onetcenter.org/IPSF.html.

My Next Move for Veterans

My Next Move for Veterans (http://www.mynextmove.org/vets/) is a Web-based interactive tool for U.S. veterans to learn more about their career options. As on the My Next Move site, users can explore over 900 different careers and see important information including skills, tasks, salaries, and employment outlook on easy-to-read career reports. They can also look at related apprenticeships and training and search actual job openings. They can find careers by searching keywords; by browsing industries; or by using the O*NET Interest Profiler, a tool that offers customized career suggestions based on a person’s interests and level of work experience. Additionally, users can browse occupations by bright outlook, green occupations, apprenticeship, and amount of job preparation required.

Unique to this My Next Move site for veterans is a feature that allows veterans to search civilian occupations related to their military training. Veterans indicate their branch of service and either the military code or title of their training to receive a list of civilian occupations that best match their military training. Where available, the results returned will indicate how closely their military duties match the civilian occupations, along with the minimum military rank and length of military service associated with the civilian occupation.

Mi Próximo Paso

Mi Próximo Paso (http://www.miproximopaso.org/) is a Web-based interactive tool for Spanish-speaking job seekers, students, and other career explorers to learn more about their career options. Mi Próximo Paso includes all the features of the English-language site, My Next Move.

O*NET Web Services

O*NET Web Services (http://www.onetcenter.org/dev_web.html) is an API that developers can use to display O*NET information in their applications. Features include the following:

Keyword searches. The same occupation search technology used in O*NET OnLine, My Next Move, My Next Move for Veterans, and O*NET Code Connector is available for use by developers. The REST Web services API returns occupations matching a word, phrase, title, or full or partial O*NET-SOC code. The results include the code and title of each matching occupation.
Career reports. These reports offer details from My Next Move. Key tasks, knowledge, skills, and abilities are available for over 900 occupations. The APIs also provide bright outlook and green information, job outlook, and more.
Occupation reports. These reports offer more detailed occupational descriptions and data available in the O*NET OnLine Summary and Details reports.
Military searches. The military transition search used in My Next Move for Veterans is also available through the Web services API. The search returns relevant O*NET-SOC occupations based on full or partial codes and titles from the Army, Navy, Air Force, Marine Corps, and Coast Guard classification systems.
Spanish keyword searches. The Spanish-language keyword search used in Mi Próximo Paso is part of the Web services API. Occupation titles are returned in Spanish, matching a Spanish word or phrase. A wide variety of features from Mi Próximo Paso is also available, including detailed career reports and Interest Profiler questions and scoring.
Interest Profiler. This useful career tool can be included in the user’s own site with the IFrame Widget. After adding a simple block of HTML code, users can take the O*NET Interest Profiler without leaving their career resources. For tighter integration, the REST Web services API is offered to provide scoring services and career results from among the O*NET-SOC occupations.

O*NET Web services were introduced to O*NET customers in January of 2014. Although still in its infancy, within the first 3 months over 200 user accounts were established and the number of requests had grown to over 3 million a month.

O*NET Career Exploration Tools

The Career Exploration Tools are based on a “whole person” concept and are designed for career counseling, career planning, and career exploration. They include the Ability Profiler, the Interest Profiler, and the Work Importance Locator. They also include electronic versions of the Interest and Work Importance assessments—the Computerized Interest Profiler and the Work Importance Profiler. These electronic versions can be downloaded onto a computer system and used at no cost by schools, American Job Centers, and others. Most recently, with the release of the My Next Move Web site, users can choose to complete a 60-item Web-based version of the Interest Profiler. Users exploring careers may use the Ability Profiler to discover what they do well, the Interest Profiler to identify the types of work they may like to perform, and the Work Importance Locator to determine which occupations will likely be satisfying according to their values and needs. The tools enable users to discover important information about themselves and use the information to explore the world of work. Workers may use these tools as aids in exploring career options, in considering career transitions, and in preparing for career change. The assessments also are designed for use by students exploring a school-to-work transition.

O*NET Academy

Training support for the O*NET system can be found on the Web at http://www.onetacademy.org. The O*NET Training Academy provides various user communities (workforce development professionals, employers, educators, students, workforce boards, job seekers, etc.) with access to O*NET support materials tailored to their needs. Through podcasts and O*NET user spotlights, customers can gain new insights into user needs and real-world applications of O*NET data. Through the O*NET Academy, O*NET users gain electronic access to recorded webinars, self-paced training courses, and best-practices tutorials on using the O*NET system and applying it on the job. The Academy site also provides access to an array of O*NET tools (e.g., O*NET OnLine, O*NET Questionnaires).

O*NET Code Connector

The Code Connector (http://www.onetcodeconnector.org/) was developed to assist workforce professionals needing to code jobs. American Job Centers, other government workforce agencies, and college career services offices are the most prevalent users of the Code Connector. The Code Connector uses the O*NET database to help users determine the correct occupational code for their job orders. To access an occupation, the user may type in a keyword or select an occupational group from the home page. On subsequent pages the user is able to refine the search to select a specific occupation. The final report contains information to help determine whether the selected occupational code is the best match for the particular job order. This information includes the O*NET-SOC description, Tasks, Related Occupations, Occupation Family, and Detailed Work Activities.

O*NET Web Site Statistics

Use of O*NET products has increased dramatically over the past few years. O*NET OnLine currently averages over 3.4 million visits per month, three times as many as the reported average 3 years ago. The O*NET Resource Center (http://www.onetcenter.org) averages 670,000 visits per month—more than twice the number of visitors from 3 years ago. The recently developed My Next Move sites average over 800,000 visits per month. In addition, use of career information systems (CISs), Web site linkages, user certifications, and O*NET product downloads is widespread; by design, the primary dissemination strategy of the O*NET Program is for the private sector to build O*NET-based products that are tailored to specific audiences or user needs.

Career Information Systems

Private and governmental online CISs using O*NET data and career tools reach millions of customers annually. For example, XAP Corporation (http://www.xap.com/), an industry leader in developing and providing students and adults with tools to explore careers and explore post-secondary education options, embeds O*NET data into its products. XAP's tools are available in over 8,000 middle and high schools in North America. IntoCareers (http://intocareers.org), another CIS provider, includes O*NET occupational data that provide O*NET-driven career exploration. IntoCareers powers more than 35% of the nation's state-sponsored CISs. Nationally, CIS programs are accessed at more than 87,000 sites by over 37 million users a year, according to a 2009 survey done by the National Career Development Association. The O*NET Program is designed to be accessible to multiple users, in keeping with the directives from the Advisory Panel for the Dictionary of Occupational Titles.^¹⁷ The O*NET Program encourages these and other service providers and product developers to create applications that deliver O*NET information to the public.

Internet Linkages

According to an exploratory search conducted in April 2014:

more than 18,100 sites link to O*NET OnLine,
more than 900 sites link to the O*NET Code Connector,
more than 2,800 sites link to the O*NET Resource Center,
more than 3,600 sites link to My Next Move,
more than 750 sites link to My Next Move for Veterans, and
more than 580 sites link to Mi Próximo Paso.

Many different types of organizations and professionals are linked to the O*NET Web sites, including:

libraries and career centers based in higher education;
higher education institutions’ schools of business, labor and industrial relations, psychology, education, and counseling;
federal, state, and local government agencies;
public libraries (especially those offering career and job search assistance programs);
career counselors, coaches, and recruiters (mostly private-sector vendors);
providers of career exploration or job search assistance (both private and public sectors);
public school systems, educational associations, and secondary schools (often recommending the O*NET database as a resource for faculty, parents, and high school juniors and seniors);
human resources management organizations;
law firms specializing in immigration law;
vocational rehabilitation or occupational medicine and health centers; and
international sites in Turkey, Canada, the United Kingdom, Australia, New Zealand, Holland, Japan, Bangladesh, and elsewhere.

User Certifications

When individuals or organizations download the O*NET database or intend to use all, some, or even part of one of the O*NET Career Exploration Tools to develop value-added products, they are asked to voluntarily register their use by completing a certification form. As of March 31, 2014, the certification database holds 4,127 entries. Because registration is voluntary, this number likely reflects only a portion of those actually using O*NET products for their own applications. An overview of those user groups that have submitted certification forms for their use of O*NET products appears as Exhibit 9.

Exhibit 9. Main Organization Types Submitting O*NET Certifications

Organization Type	Percentage of Registrations
Education services	23
Government/public administration	10
Computer systems design; programming services	9
Employment services	8
Human resources and executive search consulting	4
Internet publishing	4
Vocational rehabilitation services	4
Health care	3
Software publishers	3
Individual and family services	2
Research and development, social services and the humanities	2
Legal services	1
Military	1
Temporary health services	1
Other	25
Total	100

O*NET Product Downloads

From January 2002 through March 2014, downloads of O*NET products totaled 1,185,610 (Exhibit 10).The use of O*NET products and tools continues to increase. The O*NET Program, through continuous improvement efforts based on user needs and advancing technology, works to efficiently develop products that meet customer demands in both the public and private sectors.

Exhibit 10. O*NET Product Downloads

Product	Number of Downloads
Database		110,169
Career Exploration Tools
Ability Profiler	227,400
Interest Profiler	316,895
Work Importance Locator	141,820
Computerized Interest Profiler, Work Importance Profiler software	200,769
Total Career Exploration Tools		886,884
Other (e.g., Toolkit for Business)		188,557
*Total ONET Products**		1,185,610

Examples of O*NET Data and Products in Use

The O*NET Program provides comprehensive, up-to-date occupational information used directly by the public through access to the O*NET Web sites, as well as indirectly through value-added products delivered by both governmental and private developers. O*NET-based products benefit the public through product and service development by:

federal and state government agencies,
public workforce investment systems and workforce boards ,
assessment and CIDSs,
educational and research institutions,
U.S. armed forces,
private companies and commercial product developers, and
international users.

The following recent examples of O*NET uses are taken from the publication O*NET Products at Work (http://www.onetcenter.org/paw.html).

Federal and State Government Agencies

Federal and state agencies are using O*NET products to achieve their goals of service to employers and the public. Business development specialists, human resources personnel, and others rely on the common language of O*NET products to build connections required for a strong workforce. Provided below are examples of O*NET products used by federal and state agencies.

The Alabama Department of Rehabilitation Services incorporated O*NET information into its Employment Institute, a two-part training program required for everyone who provides direct services to clients. Trainees are shown how to use O*NET OnLine to address the vocational goals of clients, looking at tasks, skills required, work environments, and potential barriers. Training also shows counselors how to use the O*NET Code Connector to link a client’s past job title to an appropriate O*NET occupation (http://www.rehab.alabama.gov/).

Discover Arkansas, the state’s LMI delivery system, uses O*NET data in its skills-matching program to match job seekers to occupations in the state’s growth industries. The system provides O*NET skills, work activities, tasks, work values, and interests within occupational profiles linked to targeted jobs in the state’s 10 local workforce investment areas. Within each area, a job seeker begins by selecting an industry and education career cluster, an occupational group or “pathway,” and then a specific occupation. The occupation page furnishes the top O*NET skills, tasks, work activities, work values, and interests, as well as alternate job titles and the Job Zone-based level of preparation associated with the occupation. With the aid of O*NET data, the Discover Arkansas system enables job seekers to match their education levels to employers in their chosen geographic areas and industries and to determine the worker and job characteristics associated with jobs offered by those employers (http://discoverarkansas.net/).

CareerOneStop, sponsored by ETA, offers job seekers, students, businesses, and workforce professionals the most comprehensive set of career resources and workforce information on the Web. O*NET products are integrated throughout the site, including the O*NET database, O*NET Career Exploration Tools, and O*NET Tools and Technology (T2; http://www.careeronestop.org/). In addition, mySkills myFuture, a new DOL-sponsored site designed to assist dislocated workers by helping them to identify occupations that would use transferable skills from their previous work, also relies on O*NET information as the basis for the transferable skills matching algorithm.

ETA supports a Projections Managing Partnership (PMP) consortium that operates and manages the nationwide program of state and local occupational employment projections. These data are among the most requested and used datasets by a wide variety of workforce information customers and private-sector businesses. These datasets are used for multiple purposes, including the preparation of workforce investment area state plans for submission to DOL. The PMP develops and makes available to all states a standard employment projections software suite to facilitate the release of standardized estimates and output reports across states.

In addition to tools to produce both short-term and long-term employment projections, this software suite also includes a skills-based projections (SBP) tool, particularly designed to support workforce planning, policy, and decisions. The SBP tool integrates O*NET Knowledge, Skills, and Generalized Work Activities data with a state’s or territory’s short- and long-term occupational projections to identify current skill supply, projected demand, potential skill gaps, and replacement needs.

The Connecticut Department of Labor used O*NET data to respond to the workforce investment area requirements that states assess current and future job opportunities in the state, the skills necessary to obtain these jobs, and the skills necessary to meet the economic development needs of the state. To meet these requirements, Connecticut collected and analyzed LMI and published an extensive report. O*NET skills and skill descriptions were used in the report sections describing skills necessary for Connecticut’s high-demand occupations and industry sectors important for state economic development (http://www.doleta.gov/programs/ONET/ct.cfm).

The Maine Department of Labor uses O*NET information in its effort to attract new business to the state. Businesses provide descriptions of the jobs they need to fill, and the Maine Department of Labor searches O*NET data to match occupations to the knowledge, skills, and abilities provided in the job descriptions. In addition, small businesses are helped to develop their employees through training strategies based on O*NET occupational skills information.

O*NET information has also been used by the state of Maine to help dislocated workers make informed decisions about employment opportunities after a layoff. On the basis of a comparison of knowledge, skills, and abilities, the jobs the workers were leaving were first linked to O*NET occupations to identify likely opportunities requiring similar skills. O*NET information was then supplemented with state-level wage information, job openings, and information about retraining possibilities so workers could choose re-employment opportunities best suited to them.

Public Workforce Investment Systems and Workforce Investment Boards

State workforce investment systems have always been among the primary users of O*NET products. Serving employers and the public through state-sponsored online CISs and American Job Centers, O*NET products are responding to the demand for information about high-growth demand industry sectors and occupations and helping to build the connections needed for a strong workforce.

The Dislocated Worker System, part of Alabama's Comprehensive Labor Market Information System (ACLMIS), is a Web-based system providing assistance to individuals who have lost their jobs because of permanent layoffs or plant closings. Its skill survey questionnaire is based on O*NET data (abilities, knowledge, skills, and work activities). The system uses the O*NET occupational classification system for easy linkage to Alabama’s LMI. The ACLMIS system allows users to research occupations, analyze their skills to find suitable occupations, or find occupations similar to their current jobs (http://www.doleta.gov/programs/ONET/alabama.cfm).

The Employment Training Division in Humboldt County, California, uses O*NET occupational information to assist people with disabilities and those on public assistance. O*NET Career Exploration Tools help clients get a comprehensive picture of their fit with the world of work. Employers and potential employees are encouraged to consider the many facets of work other than basic skills. The division presents classes to employees of social service branches, describing how to use O*NET OnLine to serve their clients (http://co.humboldt.ca.us/HHS/SSB/EmploymentTraining.asp).

Texas CARES (Career Alternative Resource Evaluation System) is a Web site providing career exploration and occupational information. Created by the Texas Workforce Commission, the site offers versions of the O*NET Work Importance Locator and the O*NET Interest Profiler, as well as Best Match, a skills transferability program that matches users to O*NET occupations on the basis of their current skill sets. Occupational information for the many features available on the site is supported by the O*NET database (http://www.texascaresonline.com/).

The Texas Workforce Commission has also employed the O*NET-SOC AutoCoder on one of its Web sites. This Web tool provides high-quality occupational classifications for job orders, unemployment insurance claims, and résumé searches. The AutoCoder also interprets job titles written in Spanish and returns relevant O*NET-SOC codes. The O*NET-SOC AutoCoder performs by splitting the text of a job opening, résumé, or unemployment insurance claim into its individual words and phrases and matches the words and phrases in the O*NET database. Analysts have weighted the words in the database so that the most important words for an occupation are given more importance in the match calculation. The O*NET-SOC AutoCoder provides the means to make consistent and high-quality occupational code assignments (http://autocoder.lmci.state.tx.us:8080/jc/onetmatch).

The Florida Agency for Workforce Innovation uses O*NET information for job matching, re-employment analysis, and business recruitment. O*NET data are embedded in FRED, the Florida Research and Economic Database. One key application is preparing labor supply studies for business recruitment. O*NET also plays a large role in Florida’s Re‑employment Labor Market Information One-Stop Workshop. The workshop trains workforce development professionals to use the appropriate O*NET data sets and complementary tools for re-employment analysis: selecting the right tools, evaluating current skills, discovering transferable skills, studying related occupations, and finding growth industries. Workshop participants learn how to overlay pertinent local LMI on the data from these dimensions of occupational information. The workshop enables staff members to help job seekers, employers, and workforce professionals to meet their respective challenges (http://fred.labormarketinfo.com/).

Assessment and Career Information Systems

Organizations routinely rely on O*NET products as the basis for their assessment and CIDSs. Community colleges, university career centers, and many higher education Web sites also have integrated O*NET products into their career services to students. From nonprofit organizations to high-powered consulting firms, specific examples provided below indicate the variety of organizations using O*NET products.

The American Foundation for the Blind (AFB) uses O*NET data to help the visually impaired expand their employment possibilities. CareerConnect™ is a free employment planning resource for persons who are blind or visually impaired. Sponsored by the AFB, the program helps these individuals learn about the varied occupations available in the labor market. It also provides mentors and information about assistive technology that can help them do the work. This practical, user-friendly resource incorporates O*NET data to supply essential information for career exploration and to expand the universe of jobs for persons with visual impairments (http://www.afb.org/Section.asp?SectionID=7).

AZCIS is a career information system offered by the Arizona Department of Education. The Web site provides education, career, and occupation information in both English and Spanish to middle school, high school, college, and adult students. The O*NET career tools and database are essential components of the career exploration process offered by AZCIS. Spanish translations of the O*NET Interest Profiler and the O*NET Work Importance Profiler have been available on this Web site since fall 2010 (http://www.azcis.intocareers.org/).

CareerScope, developed by the nonprofit Vocational Research Institute, enhances career and educational planning processes, including career development portfolios, transition plans, individualized educational plans, and employee training and retention plans. Users include secondary schools, community and technical colleges, state and community vocational rehabilitation agencies, Veterans Affairs medical centers, and state adult and juvenile correctional facilities. Written at a fourth-grade reading level, the valid and reliable interest and aptitude assessment is self-administered and easy to complete. Accommodation options include audio delivery, English with Spanish text, and untimed delivery. CareerScope’s comprehensive reports incorporate O*NET occupational data and provide career recommendations based on the client’s interest and aptitude results (http://www.vri.org/).

Cengage Learning has developed a product called Career Transitions that uses the O*NET Interest Profiler and the O*NET database to help job seekers. Cengage’s Web site states that career research is the number one use of public libraries today, sometimes stretching staff to their limits. Career Transitions helps public libraries manage the number of people seeking job search help by providing a self-paced application that walks job seekers through the entire process from assessing strengths and interests to exploring new job opportunities (http://www.gale.cengage.com/careertransitions/).

A case manager for Michigan Works! has found My Next Move, with access to information on more than 900 O*NET occupations, to be a very useful tool for her customers. She works with the 55+ Program, a demonstration grant program that gives specialized assistance to individuals who are 55 or older, unemployed, and seeking employment. Her clients, the 55+ Program Navigators, are using the comprehensive, easy-to-use Web site as part of their career exploration and job search activities (http://mynextmove.org/).

The National External Diploma Program (NEDP) is a program of the Comprehensive Adult Student Assessment Systems (CASAS), which provides assessment and evaluation systems for adult education and workforce programs in the public and private sectors. The NEDP is a unique way for adult students to earn a high school diploma. Students who have acquired many of their high-school-level abilities through work, family, and community experiences can complete a high school diploma, opening the way to postsecondary education and improved employment options. In July 2010, the NEDP selected the O*NET Career Exploration Tools as the only official occupational tools for use by its students. Students using the tools receive results correlated to the up-to-date, detailed, and comprehensive occupational information provided by the O*NET database (https://www.casas.org/).

Pathway Builder, a division of Profiles International Educational Services Division, provides participants with an interactive, comprehensive, online experience to plan and monitor their progress from high school through postsecondary education and into the workforce. The Pathway Builder ePortal provides communication tools, educational modules, e-learning programs, site-approved Web site connections, comprehensive reporting, and career exploration tools. The assessments in the career planning ePortal solution provide users with a link to their O*NET occupational matches. Additional features include job board technology and an e-mail platform to assist with an education-training-workforce initiative to engage students and workers with career success (http://www.pathwaybuilder.com/).

Women Employed, a national advocate for women’s economic advancement, developed Career Coach, a free online career development tool that helps adults explore career options, define career goals, identify education and training resources, and make step-by-step plans to reach their goals. The Web-based program provides users with occupational data available from the O*NET database. It also provides an online version of the O*NET Interest Profiler (http://womenemployed.org/).

Higher education institutions are among the most prevalent types of users of O*NET products for assessment and career information systems. A Web search shows O*NET products at work in most state educational systems. A few examples include AlabamaMentor.org, Arizona State University CRESMET, CaliforniaColleges.edu, Cascadia Community College (Washington), University of Central Florida, CTMentor.org (Connecticut), IllinoisMentor, Middle Tennessee State University, MississippiMentor, Normandale Community College (Minnesota), PennsylvaniaMentor, TexasMentor, and Texas State University.

Educational and Research Institutions

With the help of the occupational information contained in the O*NET database, colleges are developing educational curricula and research organizations are conducting a broad array of research projects. Presented here are a few examples.

In a study titled The Hidden STEM Economy (Rothwell, 2013), the Metropolitan Policy Program at the Brookings Institution used O*NET knowledge data to redefine the STEM economy and its constituent workforce. Among the study’s key findings were that about 20% of all U.S. jobs require a high level of knowledge in any one STEM field, and that only about 20% of all federal STEM funding supports training at the associate’s degree level and below. To define STEM fields, the researchers grouped six O*NET knowledge descriptors into four fields. The O*NET knowledge descriptors biology, chemistry, and physics were chosen to represent the science field; computers and electronics to represent the technology field; engineering and technology to represent the technology field; and mathematics to represent the mathematics field. For each occupation, O*NET descriptor-level ratings were used to arrive at an average level score per STEM field. The author concluded that greater federal support was needed for training in STEM jobs requiring an associate’s degree or less and that greater coordination was needed between workforce development and state and local education resources.

College for America, a competency-based college degree program, cites the O*NET program as a source of information for their profiles of six front-line and nonclinical health care occupations with high projected growth. The report, Rise of the Medical Assistant (Alssid & Goldberg, 2013), lists O*NET Detailed Work Activities common to all six occupations, as well as those specific to each one. The report emphasizes the value of work activities at this level of detail for development of college and training curricula, which can ensure curriculum relevance over time while enhancing student employability (http://collegeforamerica.org/latest/entry/rise-of-the-medical-assistant).

A study by the Federal Reserve Bank of New York, titled Workforce Skills Across the Urban-Rural Hierarchy (Abel, Gabe, & Stolarick, 2012), used O*NET skills data to identify important differences in occupational skill requirements across the United States and across areas of differing population density. They concluded that urban areas tend to specialize in skills-based clusters involving scientists, technicians, engineers, and executives, with social skills and complex problem-solving skills as fundamental to these groups’ effectiveness. By contrast, rural areas feature the skills-based clusters machinists, makers (carpenters, tool and dye makers, and others), and laborers. Use of O*NET skills data enabled these researchers to expand their urban-rural comparison beyond the typical education-level comparisons to provide a more functional comparison of these areas (http://www.newyorkfed.org/research/staff_reports/sr552.pdf).

A study led by researchers at the Idaho State Department of Education explains how O*NET helps persons with disabilities to cross the gap between their postsecondary goals and their current skills and knowledge. In particular, the study illustrates the ways students can use the O*NET database to research their postsecondary goals and determine the knowledge and skills necessary to meet those goals. The authors observe that the O*NET database allows students to realistically assess their current goals and to continue to follow their dreams by developing alternative related goals. This process of comparing goals with concrete knowledge and skill characteristics to one’s existing knowledge, skills, and abilities is known as transition planning, which is critical not only to life planning by persons with disabilities but to the mandate of the Disabilities Education Improvement Act of 2004 (Lowenthal & Bassett, 2012).

Educational Testing Service used O*NET data to develop a competency model to meet the demands of 21st-century employers. In their report Identifying the Most Important 21^st Century Workforce Competencies: An Analysis of the Occupational Information Network (O*NET), Burrus et al. (2013) describe the process by which they analyzed O*NET Abilities, Work Styles, Skills, and Knowledge data in order to develop a model of competencies that addresses what they describe as a “skills mismatch” between job seeker attributes and those sought by employers. Whereas several prominent groups had already developed competency models in response to this need, the authors emphasize that these models were derived rationally, rather than statistically, and that the O*NET database provides a data-driven alternative to this approach with its large-scale collection of data on a broad range of occupations. Comparing the data-driven O*NET model to the consensus of three rationally driven models, the authors identified five competency components: problem solving, fluid intelligence, teamwork, achievement/innovation, and communications skills. The authors compared their resulting competency model to the competency deficits cited by employers and found substantial overlap (http://www.ets.org/research/policy_research_reports/publications/report/2013/jrkj).

The University of Illinois at Urbana-Champaign and Center for Regional Economic Competitiveness research team created a Web tool to help policy makers and practitioners better understand their local economies and compare these economies with others in terms of industry function, required occupational knowledge, and commuting. The tool’s 12 occupational knowledge clusters are based on O*NET knowledge areas and include O*NET data on level of education or training needed. The tool enables comparison of the more than 3,100 U.S. counties on key statistics such as population, annual growth, employment, and average earnings. Cross-industry comparisons may be made overall and by occupational knowledge cluster and industry diversity (distribution of an occupation’s employment across industries) (http://economicdiversityinappalachia.creconline.org/).

In Workforce Skills and the Changing Knowledge Economy in Massachusetts, Renski and Wallace (2012) at the University of Massachusetts describe a study using O*NET data to investigate the hybrid skill sets needed in the changing knowledge economy in Massachusetts. The study, conducted jointly with the Federal Reserve Bank of Boston, uses O*NET occupational data on education, experience, training, and skill requirements. The study observes that, although the Massachusetts economy has become more technology intensive, success in growing industries like health care depends on combining technical skills with social, communication, and learning skills. With the aid of O*NET skills data, the study concludes that skills related to the acquisition, processing, and dissemination of new knowledge are essential to the Massachusetts economy’s ability to meet the changing demands of the 21st century.

U.S. Armed Forces

The U.S. military has recognized the value of O*NET data and career tools in its various transition programs, recruiting activities, and human systems development projects. Presented here are a few examples of the range of O*NET products being put to work in the armed forces.

As described in Section A.2.1, My Next Move for Veterans is designed for U.S. veterans who are current job seekers. This interactive tool helps veterans learn about their career options. The site has tasks, skills, salary information, job listings, and more for over 900 different careers. Veterans can find careers through keyword search, by browsing industries that employ different types of workers, or by discovering civilian careers that are similar to their jobs in the military. Veterans can also take advantage of the O*NET Interest Profiler, a tool that offers personalized career suggestions based on a person’s interests and level of work experience.

A report prepared for the Air Force Personnel Center details how O*NET assists the process of selecting the candidates most likely to succeed as either remotely-piloted aircraft (RPA) pilots or sensor operators. Specifically, the O*NET Content Model provides a framework for organizing relevant skills, abilities, and other characteristics required by these jobs. The report’s authors stipulate that they require a widely used framework that leads to differentiable and measurable individual differences. The authors explain that the O*NET Content Model allows them to “reduce redundancy across constructs and to ensure broad coverage of several different domains of individual differences.” Their use of the Content Model resulted in two different options for selection batteries for each of the critical positions of RPA pilot and sensor operator (Paullin, Ingerick, Tripp, & Wasco, 2011).

The Transition Assistance Program at the Office of Wounded Warrior Care and Transition Policy is offering a variety of virtual learning opportunities to accommodate the needs of the National Guard, reserves, and wounded warrior populations and their families. The virtual aspect of these learning modes opens up much-needed educational opportunities often not available to service members because of their mobility, varying time zones, accessibility, and stage of demobilization and integration. All that is required for attendance is a broadband Internet connection and a telephone. Course registration is accessed via www.TurboTAP.org. The series of 11 webinar course offerings includes one on decoding military skills for civilian employment, which prominently features both O*NET OnLine and My Next Move as tools for service members to facilitate a successful transition to civilian employment.

The Navy Manpower Analysis Center uses O*NET data in its work to develop occupational standards, which serve as a basis for training and career development in areas such as formal school curricula, onboard training, development of personnel advancement requirements, and development of Navy-wide advancement examinations. The O*NET skills taxonomy is used to categorize task statements as part of the process to develop the standards (http://www.npc.navy.mil/AboutUs/NAVMAC/).

The North Carolina Military Foundation teamed with the North Carolina Military Business Center to create a database and interactive Web site that enables businesses to link their needs to the competencies of troops exiting the military. One of the challenges faced by troops and business leaders alike is identifying the knowledge, skills, and abilities shared by military and civilian jobs. Using a keyword related to a job opening, employers are able to search for related military occupations and information on how many military personnel in these occupations are returning annually to civilian jobs. They can view additional information about these occupations, including a list of related civilian job titles. Further exploration is available through a link to the related occupations in O*NET OnLine. This Web site helps employers and transitioning military personnel come together through the common language of the O*NET system (http://www.ncmbc.us/).

The Office of the Secretary of Defense enlisted the RAND National Defense Research Institute to convene a panel of experts to provide assistance in refining the implementation of the Department of Defense human capital strategy. The goal of the strategy is to develop a foundation for military personnel management. A major component of this goal is a competency-based occupational analysis system. In the Final Report of the Panel on the Department of Defense Human Capital Strategy (Hanser et al., 2008), the panel members concluded that O*NET “has the potential to provide a framework for developing much of the common language and functionality desired in a new DoD system.”

The U.S. Department of Defense’s Armed Services Vocational Aptitude Battery (ASVAB) Career Exploration Program offers tools to help high school and postsecondary students learn about career exploration and planning. Developers of the ASVAB Program wanted to change its philosophy to emphasize wider career exploration and decision-making among its participants. Completely redesigned, the program now uses O*NET data to broaden occupational choices for nearly a million ASVAB participants annually at more than 14,000 of America’s high schools. Students are encouraged to explore a variety of career possibilities suited to their interests and learn to make decisions based on information about themselves and about careers, instead of exploring just a few occupations that match their current abilities (http://www.doleta.gov/programs/ONET/asvab.cfm).

Private Companies and Commercial Products

Private companies are using O*NET information for both in-house purposes (human resources functions such as job description writing and employee development) and for commercial product development. The number of products with “O*NET in-it” continues to grow.

CareerNoodle by Transcend Innovation Group provides an online suite of career exploration and planning tools for middle school, high school, and college students. CareerNoodle relies on modern learning theories and human-computer interaction principles so that students are engaged, motivated, and in control of their career exploration process. Interactive activities to measure student work interests and values are adapted from the O*NET Career Exploration Tools. Occupational data come from the O*NET database. Transcend Innovation Group’s David Burns says, “We use O*NET data and the career exploration tools.... Thanks for providing us with such wonderful tools and information!” (http://www.transcendinnovation.com/careernoodle.php)

Corporate Gray Online is a Web site devoted to linking employers with transitioning or former military personnel. The online tool complements the organization’s military-to-civilian career transition books and Corporate Gray job fairs. The Web site provides users with a link to the O*NET OnLine Crosswalk Search. Users enter a military job code or title and may view detailed occupational data on related civilian occupations (http://www.corporategray.com/).

Economic Modeling Specialists, Inc. (EMSI), has developed a system that allows users to target competency gaps through a unique way of visualizing the makeup of occupations. Using O*NET knowledge, skills, and abilities data, EMSI’s Strategic Advantage is a Web-based analysis platform for use by workforce development and education professionals. It provides the data needed to help these professionals make informed decisions necessary for regional growth and development (http://www.economicmodeling.com/ ).

Choices^® Planner by Bridges Transitions provides the tools needed for students to build personalized plans for college and careers. An online CIDS, Choices Planner integrates O*NET data, as well as the O*NET Career Exploration Tools, into their online program. Bridges Transitions has included Spanish translations of all O*NET components so the system can be used by both English and Spanish speakers (http://www.bridges.com/us/prodnserv/choicesplanner_hs/index.html).

iNQUATE Corporation has incorporated O*NET data into its human-capital software application. iNQUATE is an integrated seven-module application that accommodates the development and communication of corporate goals and objectives, automates the performance management process, and engages managers in the best use of human capital while helping them analyze costs associated with maintaining a workforce. iNQUATE can be integrated with onsite databases or information available from external providers through Web services. All seven of iNQUATE’s modules work together to simplify the process and eliminate duplication of information and effort (http://www.inquate.com/).

Manpower, a worldwide provider of high-value staffing services with nearly 1,100 offices in North America and 4,500 offices in 80 countries, provides jobs to 5 million persons every year and services more than 400,000 clients worldwide. The O*NET occupational and skills taxonomy helps Manpower match the right person to the right job. The O*NET system also offers a systematic structure that enhances Manpower’s analysis of the employment marketplace and its tracking of staffing trends. By incorporating O*NET structure into its procedures, Manpower has benefited by being able to:

accurately identify the types of placements each field office makes;
locate field offices where the highest need exists;
more accurately consolidate information for various types of analysis, including marketing analysis; and
begin the process of having Manpower offices in other countries map their occupations to O*NET, enabling Manpower to more efficiently consolidate information for global reporting (http://www.doleta.gov/programs/ONET/Manpower.cfm).

Trustmark Insurance Company uses O*NET OnLine to collect information on job titles, tasks, and skills and to access salary data. Human resources personnel use this information to describe tasks associated with sales support activities and to align managers’ job titles with appropriate occupations across industries. O*NET occupational descriptions and data help to determine appropriate salary ranges for company positions and provide validation of company salary survey data (http://www.trustmarkinsurance.com/).

Valpar International manufactures and develops specialized test instruments and software that measure work-related skills. Their products are used worldwide in education, workforce development, and allied health. Valpar’s Aviator software uses the O*NET database in conjunction with various assessments to aid users in identifying occupations that fit their skills and interests. The following description is found on their Web site.

Aviator’s multifunctional approach encompasses aptitude assessment, two pictorial/audio interest surveys, and two databases—the standard and O*Net databases each with approximately 1000 jobs. Using Valpar’s skills-based assessment foundation, this fast, reliable, valid tool is easy to administer and cost efficient. All subtests and surveys are computer-based and can be completed in 60 minutes (http://www.valparint.com/).

International Users

O*NET data and career tools have quickly gained prominence in government and private industry products around the globe. One example is Australia’s government, which uses the O*NET database as a behind-the-scenes data set linked to its own Standard Classification of Occupations. Human resource professionals in Japan have adopted the O*NET Career Exploration Tools. Chinese researchers have relied on the O*NET database for their occupational studies. European and Central American countries are translating O*NET products for their own populations. O*NET OnLine has received visits from users in over 190 countries. Countries logging hundreds of thousands of hits include Australia, Canada, China, Egypt, the Netherlands, New Zealand, Puerto Rico, Singapore, Germany, Great Britain, Hong Kong, India, Indonesia, South Africa, South Korea, and Taiwan. Each year, the O*NET Web sites log millions of visitors from virtually every geographic region in the world.

E-Career Guidance.Com, Ltd. (ECG), is an Irish company that provides vocational and career counseling services. It developed an online career exploration tool, the CareerDecisionMaker^® (CDM), which assesses a client’s standing on variables from four domains: vocational interests, work values, personality, and skills. The client’s score profile is matched to score profiles for occupations. The occupations having profiles that correspond most closely to the client’s profile are presented as career options that the client might explore. ECG linked the CDM to the O*NET database and its occupations from the SOC. Scores from two of the CDM domains link directly to O*NET data for interests and work values. The CDM skills and personality domains, however, differ from the O*NET Skills and Work Styles domains (i.e., they are based on different taxonomies). ECG contracted with Human Resources Research Organization (HumRRO) to develop skills and personality profiles for O*NET’s SOC occupations that are commensurate with the CDM skills and personality profiles. In addition, HumRRO devised a profile matching algorithm that permits use of the entire four-domain CDM profile to identify promising career options for ECG’s clients. All project activities (e.g., development of stimulus materials for O*NET analysts, summary of personality ratings provided by O*NET analysts, data analysis, development of the profile-matching algorithm) are documented in a technical report (McCloy, Byrum, Munoz, & Tsacoumis, 2006). (http://www.careerdecisionmaker.com/).

The International Labour Organization (ILO), a specialized agency of the United Nations that sets and oversees international labor standards, highlights O*NET in its book, Skills for Green Jobs: A Global View (Strietska-Ilina, Hofmann, Durán Haro, & Jeon, 2011). In coordination with the European Centre for the Development of Vocational Training, the ILO’s Skills and Employment Department launched a worldwide investigation of skills needs for greener economies in 21 countries. In presenting the results of this study, the report frequently features the work of O*NET and includes a special section devoted to describing the O*NET system and taxonomy. It also features O*NET’s report Greening of the World of Work: Implications for O*NET-SOC and New and Emerging Occupations (Dierdorff et al., 2009). Moreover, the report categorizes efforts across the globe to determine whether the green economy is sufficiently accounted for by national occupational classification systems. The United States’ efforts toward the development of the O*NET database are described as the most rigorous and systematic of the countries considered.

The Technical Education and Skills Development Authority (TESDA), a government agency attached to the Philippine Department of Labor and Employment, has used the O*NET Ability Profiler in its aptitude assessments of hundreds of thousands of high school students. In addition, TESDA is distributing the O*NET Computerized Interest Profiler to regional, provincial, and district offices, as well as to their training institutions, as part of its e-Profiling program, targeting out-of-school youth, career shifters, course shifters, and unemployed adults (http://www.tesda.gov.ph).

Examples of the O*NET Program in Published Literature

Presented here are some examples of references to the O*NET Program in publications. For an extensive list of research articles, books, book chapters, technical reports, and presentations referencing the O*NET Program, see Appendix D.

An article in the journal Human Factors describes the use of O*NET work activities, work context, and abilities data to identify job-specific physical work exposure factors relating to carpal tunnel syndrome (CTS). The researchers identified six O*NET descriptors to capture physical job demands conceivably leading to development of CTS. Results showed a strong relationship between CTS and job-related physical exposures, as represented by the six O*NET descriptors. These results, involving secondary job exposure data, were found to be consistent with results of studies involving direct measurement or observation of workers on the job (Evanoff, Zeringue, Franzblau, & Dale, 2014).

An article by Koys (2013) in the Journal of Human Resources Education details the value of O*NET data and Web applications for development of human resources courses. The author describes uses of several O*NET data types in development of course content. Importance ratings on the O*NET knowledge descriptor Personnel and Human Resources help educators emphasize to students the importance of human resources knowledge for nonmajors. Human resources–related tasks linked to business-related occupations help instructors develop human resources courses in an MBA core curriculum. O*NET’s Bright Outlook designation and linked BLS projected-growth ratings inform students of an occupation’s future prospects. Links from O*NET sites to wage data provide students information on potential earnings. The Career One Stop Job Description Writer, which uses O*NET data, teaches students to write a thorough, structured job description. The article also demonstrates the relevance of tasks for human resources-related O*NET occupations to curriculum guidelines established by the Society for Human Resource Management.

In “Doing competencies well: Best practices in competency modeling,” Campion et al. (2011) identify the O*NET Content Model as a valuable resource for competency modeling. The authors explain that the foundational research behind the Content Model distinguishes it from most competency dictionaries in that it ensures a “fairly exhaustive and conceptually independent” set of descriptors to provide the foundation for a competency model.

Researchers at ACT, Inc., have validated the WorkKeys Fit Assessment, a tool that uses occupational interests and work values to measure person-occupation congruence. They used the O*NET database to develop and validate the assessment. Results of their work suggest that the fit between person and occupation predicts positive work attitudes and outcomes. This relationship is a useful concept that can be applied to personnel staffing. Organizations that include an interest-values assessment of person-occupation congruence may increase job satisfaction, job match, and job commitment among their employees (Swaney, Allen, Casillas, Hanson, & Robbins, 2012).

Looking to the Future With Competency Models

DOL has identified industry and occupational competency models as a resource that is key to businesses and industries articulating their workforce needs. The O*NET Content Model and database provide a valuable resource for the development of reliable and valid competency models. In turn, these models form the basis on which curriculum developers and training providers develop objectives and outcomes to ensure that workers have the right skills. The ETA acts as a broker of information in the area of competency models development by promoting, supporting, and disseminating information (http://www.careeronestop.org/competencymodel/). The information gathered through the O*NET Data Collection Program continues to provide the workforce investment system partners with tools necessary for investment in human-capital development. To date, 23 high-growth industry competency models have been developed, and more are in progress.

For some industries, the information available from industry sources is neither comprehensive nor readily available to developers. In these cases, model developers can access occupational information in the O*NET database to identify the critical work requirements, worker skills, and knowledge for occupations in the industry. The O*NET data are an essential source of information, reducing the time and effort required to develop industry competency models. When industry information is available or competency models have been developed, a comparison with the O*NET data validates information from industry sources.

The Tennessee Valley Authority used O*NET information as presented in the Career Pathways tool, which was designed by EMSI, to evaluate workforce competencies in northeastern Mississippi. By showing Toyota that competencies of workers in the declining furniture industry in northeastern Mississippi were 98% compatible with the needs of the auto industry, O*NET data helped bring a Toyota plant to northeastern Mississippi (http://www.economicmodeling.com/).

Uses of Information Technology to Reduce Burden

The O*NET Data Collection Program employs the latest in information technology systems and procedures to enhance the quality of the data, minimize burden on the responding establishments and job incumbents, and reduce the overall cost of the data collection effort.

Web Questionnaires

Electronic versions of the O*NET questionnaires are available via the Internet to sampled job incumbents and occupation experts. Many of the benefits of the paper questionnaires are replicated in the electronic questionnaires. Specifically, users are able to start and stop multiple times without losing data. They can return to a partially completed questionnaire at any time during the survey period and resume where they stopped. A respondent may also review and edit previous answers as necessary. In addition, an on-screen progress meter keeps respondents informed of their movement through the questionnaire.

Advances in Web technologies and security, as well as the increasing prevalence of establishments’ and employees’ access to Web browsers, have made Internet-based data collection both feasible and practical. Internet use continues to accelerate, and the use of the Web questionnaire has increased with it. For 2004, 15.8% of job incumbents used the Web questionnaire, a substantial increase over the 3.7% who responded via the Web during the 1999 pretest. The usage of the Web questionnaire continues to increase over time. In 2013, 19.8% of job incumbents and 47.8% of occupation experts completed the survey online. The paper questionnaire cover and informational materials mailed to respondents continue to highlight and encourage the option of completing the survey on the Web.

The paper and Web versions of the questionnaires were designed to be optimal for their respective modes of administration. The questionnaire design literature suggests that this approach is essential to reduce mode effects. That is, if each questionnaire is designed to minimize measurement error in its particular mode of interview, mode effects are also minimized. For example, in the paper version, multiple questions appear on a single page of the questionnaire. However, in the Web version, the domain questionnaires display only one question per screen (although the respondent can navigate at will through the instrument). The literature on Web survey design (see, e.g., Couper, 2008) suggests that one question per screen for Web surveys reduces measurement error and therefore the effects of administration mode. This difference was the only important one necessary for the Web version, because both instruments are self-administered. In fact, to ensure comparability between the paper and Web responses, the formats and wordings of the questions and response categories for the two versions remain identical.

Project Web Site

An O*NET data collection Web page application has been developed to support and assist with the O*NET Data Collection Program (http://onet.rti.org). This site is divided into two major sections: the public and restricted-access sections. The restricted-access section is further subdivided into two areas: the online questionnaires area and the project management area.

The goal for the public section is to support the establishment-recruiting process. This section is accessible to the public, without restrictions. The public section includes O*NET background information, endorsement letters, frequently asked questions, copies of the questionnaires, and links to other O*NET Program–related Web sites. The purpose of this section is to provide establishments, sampled workers, and occupation experts with readily accessible information about the data collection effort and uses of the data.

The restricted-access section contains sensitive information available only to certain groups, such as survey respondents and project managers. This section is controlled by a user ID and password authentication scheme. The Web server includes a secure socket layer (SSL) certificate to allow encrypted transmission of all information over the Internet. This same technology is used by e-commerce Web sites to secure credit card numbers. In addition, no “cookies” are used. (A cookie is data given to a user’s Web browser so that the browser will return the data to the server or Web site during subsequent requests.) Some Internet users distrust Web sites that deposit cookies on their computers and may even configure their computers to prohibit cookies, so the “cookie-less” techniques ensure that the site will perform as expected, whether or not a user has disabled cookies.

The online questionnaire area of the restricted-access section provides sample members an alternative to completing the paper questionnaire. Only persons who have been selected to participate in the survey have access to this area. Unique user IDs and passwords are assigned to each job incumbent and occupation expert by a central office computer system at the time of selection into the sample; names and other personally identifiable information are not obtained. The ID and password, along with other survey materials, are provided to the sample member. Before allowing access to the online questionnaire area, the Web site confirms the validity of the ID and password and verifies that a completed paper survey form has not already been received.

Having entered this portion of the site, respondents are:

informed that participation in the survey is voluntary,
assured that their survey responses will remain private,
limited to seeing only the questionnaire they have been asked to complete,
permitted to stop at any point and continue responding later,
permitted to skip questions they choose not to answer, and
permitted to review and change previous responses.

On the last page of the survey, respondents confirm that they have completed the questionnaire and are given the option to enter comments about the survey; then they exit the questionnaire area of the site and are thanked for their participation. Their user ID is automatically deactivated at this time. Any further attempts to log in will not be possible, because the system recognizes these users as having completed the survey.

The database containing the survey data is not accessible via the Internet; it resides on a server inside RTI’s firewall. The Web data collection application encrypts and transmits data from the respondent’s computer into the survey database. Only authorized project staff, operating from inside the firewall, have access to the survey database.

The project management area of the restricted-access section contains data collection management reports and information. Login credentials for this area are created for managers of the O*NET Data Collection Program. This portion of the Web site serves as an intranet for the O*NET Program, facilitating communication among RTI staff, program staff at the National Center for O*NET Development, and DOL. Production reports, posted nightly, include summaries of the progress of establishment recruiting, questionnaire shipment and receipt, and overall data collection status. Additional applications include a secure, centralized data file repository used by statisticians, analysts, and others to share results of specialized, nonroutine analyses and reports. The project management area of the Web site reduces the cost of the data collection effort by centralizing and streamlining features used by the project team.

The Case Management System and Data Collection Utilities

The O*NET CMS is a Web-based control system that supports and monitors the data collection activities of the BLs, the mailing of informational materials and questionnaires, and the receipt of completed paper and Web questionnaires. Enhancements have been made to the CMS that allow greater flexibility and effectiveness in the communications between the BLs and the POCs in sample establishments. For example, visual cues (icons) in the CMS allow the BLs to prioritize and customize their approach for certain types of establishments that require special procedures. Another enhancement is the ability to do ad hoc package modifications, such as the inclusion of special endorsement letters, based on O*NET-SOC occupations. Because the packages for specific O*NET-SOC occupations are customized, the POCs receive targeted materials that help communicate the data collection mission. In addition, tools have been developed to help operations management load-balance the number of cases assigned across the team of BLs. Another feature, for use with multisite organizations, permits shipping of questionnaires to more than one POC in an establishment.

An innovative RTI-developed approach to survey management known as adaptive total design (ATD) has been integrated into the CMS to provide real-time, graphical updates of process data captured during the execution of day-to-day operations.¹⁸ ATD ensures that managers are alerted to and can swiftly take action on unusual situations that can arise. It also allows managers to pair BLs to particular industries with which they have demonstrated exceptional recruiting effectiveness.

In an effort to help manage supplies (envelopes, brochures, questionnaires, etc.) associated with data collection, an inventory tracking system integrated with the CMS has been developed and deployed. As the BLs place orders for informational materials and questionnaires to be shipped to survey participants, the inventory system updates reports to show the expected remaining inventory. As stocks run low, operations staff are alerted to replenish supplies. The system provides a means to reconcile physical and expected inventory. As a result, the system has improved the efficiency of ordering, storing, and shipping data collection materials.

Questionnaires are prepared for each respondent by an automated order-fulfillment system. The system detects the questionnaire domain type and occupation assigned to the respondent, and it prints a complete package, including the individually customized and labeled questionnaire, ready to pack and ship.

The CMS automatically assigns questionnaire domain types sequentially as new respondents are added to the sample. Depending on how many of each type are actually returned by respondents, it is possible to receive enough responses of a particular questionnaire type for an occupation before receiving the total desired number of questionnaires of all types. The system continually monitors the number of returned questionnaires by type and overrides the sequential assignment of questionnaires as appropriate so that only the questionnaire types that are still required are sent. This feature enhances efficiency by eliminating the shipping of questionnaires that are no longer necessary to complete the data for an occupation. Moreover, by continually focusing resources only on the remaining questionnaire types required, it reduces the total time required to complete any occupation.

Section 508

Section 508 of the Rehabilitation Act of 1973, as amended, specifies that persons with disabilities shall have access to and use of the same information that persons without disabilities have. To comply with this section, the O*NET Data Collection Program designed its online application, O*NET OnLine, to ensure that the data and information are accessible to the widest possible audience, including persons with disabilities. O*NET OnLine also provides links to several accommodation and disability resources on the Internet. Site accessibility remains an important design component in the ongoing maintenance and development of O*NET Online with close adherence to the guidelines of the Web Accessibility Initiative (WAI) from the World Wide Web Consortium (W3C).

Additional Uses of the Internet for Data Collection

The O*NET Program uses the Internet to gather additional occupational information, such as lay titles and tools and technology. This use expands the O*NET database, providing easily maintained current information about tools and technology and enhancing users’ ability to find occupations relevant to their training and expertise by giving them a wider range of search terms. This enhancement is accomplished without additional burden to the public and at less cost than other means of data collection.

“Web scraping” is employed to gather tools and technology linked to in-demand occupations in high-growth industries, with particular emphasis on identifying rapidly changing information technologies, such as database user interface and query software and analytical or scientific software. Tools and technologies linked to specific occupations are found on job posting, career education, professional association, and other Web sites. Occupational analysts link the tools and technologies to the United Nations Standard Products and Services Code^®, an online classification system for tools and services (http://www.unspsc.org/). Before the information is published in O*NET OnLine, rigorous reviews are performed to ensure the quality and usefulness of the data. Information collected from the Web in this way represents a significant enhancement to the data available to O*NET users.

O*NET lay titles are collected from unmatched search terms submitted to America’s Career InfoNet (ACINet), a DOL-sponsored online resource for job seekers that helps individuals explore career opportunities and make informed choices about employment and education. O*NET lay titles, as well as tools and technology, are also collected from the ACINet Job Description Writer, an online tool offered by ACINet to help employers and human resource specialists write job descriptions. Search terms, such as Sports Medicine Specialist and Nanotechnologist, that do not result in occupational matches in ACINet are captured electronically. These terms, together with job titles collected through the Job Description Writer, are reviewed by O*NET occupational analysts for potential inclusion in the O*NET Lay Title database. Lay titles greatly enhance the keyword search functions for O*NET OnLine, ACINet, and O*NET Code Connector, a Web application to assist workforce professionals in matching job titles to O*NET-SOC codes.

Use of Web sites further enhances the O*NET database, provides greater search capabilities to users, and reduces burden. It also allows for rapid update of the data through user input and at minimal cost.

Efforts to Identify Duplication

To avoid duplication and reduce cost, several portions of the O*NET Content Model are provided from existing data sources. Specifically, as discussed in Section ‎A.1, the domain of Workforce Characteristics—including information on industries, job opportunities, and pay—is obtained through links to existing LMI databases. Information about occupational licensing, certifications, national industrial skill standards, and related instructional programs is provided from existing sources, such as the Manufacturing Skills Standards Council; the National Retail Federation; and several Web sites, including The CareerOneStop (COS) Certification Finder at http://www.careeronestop.org/EducationTraining/Find/certification-finder.aspx and the COS Licensed Occupations Database at http://careerinfonet.org/licensedoccupations/?ES=Y&EST=Licensed+Occupations.

The exhaustive reviews of existing labor market and occupational information conducted by the Advisory Panel for the Dictionary of Occupational Titles and DOL’s DOT review staff, as well as subsequent research, identified no other comprehensive, valid, and reliable sources that could be used for the data items included in the O*NET database. The development of the O*NET Program has involved staff and advisors with many years of experience in labor market and occupational information who are familiar with existing data sources. In fact, as discussed in Section ‎A.2, many existing systems that provide detailed occupational information are actually using information based on O*NET data or the predecessor DOT.

The few existing sources with similar measures are too limited to be used in the O*NET database. Some existing sources are valid and reliable—for example, information from the OPM and the U.S. Department of Defense—but are not comprehensive, because they represent only those jobs in federal civilian employment or the military. Some private sources of job analysis information exist; however, they are based on job analyses conducted for particular purposes or settings rather than on a representative sample of employers and workers. They are therefore limited in their coverage and not representative of the entire workforce. Furthermore, these analyses are not comparable because they do not use the prescribed O*NET common language to describe occupational requirements; it is not practical to combine them, because they include dissimilar components. Finally, these private data sources are not available to the general public.

Efforts to Minimize Burden on Small Establishments

All sizes of establishments are represented in the O*NET estimates for most occupations. For some occupations, the targeting strategy used in selecting an efficient sample may lead us to omit some small establishments from the sampling frame, but this omission occurs for few occupations. The omission is allowed when it is clear that sampling small establishments will greatly reduce the efficiency of the data collection or when it is clear that incumbents from small establishments are not working in the mainstream of the occupation.

Given that establishments of all sizes should be represented in the samples for most occupations, specific design provisions have been undertaken to avoid overly burdening small establishments. For example, Exhibit 11 shows the distribution of establishments by number of employees on the D&B list used as the sampling frame for O*NET data. Exhibit 11 illustrates how O*NET sampling selects small establishments at a much lower rate than that at which they occur in the population. For example, although 89.5% of establishments employ fewer than 25 employees (represented in the first three rows of the exhibit), the O*NET sample will consist of only 39.3% of such small establishments. On the other hand, large establishments (with 250 or more employees, as represented in the 7th to 9th rows of the exhibit) will make up 23.3% of the O*NET sample but only 0.4% of all establishments. Thus, the O*NET sample relies more heavily on large establishments to reduce the burden on small establishments with few employees. The disproportionate sampling of large and small establishments is properly accounted for in the analysis weighting, resulting in statistically consistent estimates.

In addition, data collection procedures place lower burden on small establishments than on large establishments. When a small establishment is selected, it likely employs fewer of the targeted occupations and has fewer employees working in the occupations. Thus, a POC at a small establishment generally spends less time preparing sampling lists and distributing questionnaires than a POC at a large establishment, which is more likely to employ several of the targeted occupations, with a large number of employees working in the occupations.

Exhibit 11. Distribution of Frame and Sample Establishments by Employment Size

Number of Employees	Total Number of Frame Establishmentsa	Frame Distribution (Percent)	*Actual Distribution of ONET Sampled Establishmentsb (Percent)**
1–4	13,387,846	71.9	15.6
5–9	1,930,786	10.4	5.8
10–24	1,338,367	7.2	17.9
25–49	539,454	2.9	8.5
50–99	327,960	1.8	14.9
100–249	179,345	1.0	9.6
250–499	43,733	0.2	13.2
500–999	15,982	0.1	5.8
1,000+	9,591	0.1	4.3
Unknown	844,714	4.4	4.4
Total	18,617,778^c	100%	100%

^aData based on January 2014 Dun & Bradstreet (D&B) frame of establishments.

^bData based on distribution of prior O*NET samples that used the D&B frame. Future O*NET samples will be similarly designed.

^cThe final frame used to select the O*NET sample has nearly 17 million establishments, rather than 18,617,778 establishments, after the removal of duplicate records. Distribution data is unavailable on the reduced frame.

Consequences of Collecting the Information Less Frequently

The O*NET database is the most comprehensive source of occupational information in the United States. No other similarly comprehensive, reliable, and valid source exists. If O*NET data are not collected, U.S. citizens, industry, business establishments, military, government and educational institutions, and the workforce investment system will have few options that meet their needs for occupational information. O*NET data are used to develop industry competency models and occupational competency profiles (i.e., industry- or occupation-specific job or work analyses). O*NET data also include information on transferable skills and are used for skills gap analysis, promoting the development of a mobile workforce responsive to changing economic needs.

The use of O*NET data by industry, employers, software developers, job seekers, students, educators, and workforce development specialists supports a well-functioning U.S. labor market and workforce investment system—both essential to U.S. competitiveness in the global 21st-century economy. O*NET-SOC occupations conform to the SOC, permitting O*NET data to be linked to and analyzed with sources of information on current occupational employment and trends, wages, and demographic data. Its electronic format is freely accessible, either through O*NET OnLine, as a download from the O*NET Resource Center (both at http://www.onetcenter.org), or through Web services.

The initial 3.1 version of the O*NET database was populated with information derived from the DOT. So far, data have been published for 1,572 O*NET-SOC occupations (including multiple updates). Additional data releases are planned through 2018 to update occupations and release new data on new and emerging occupations. The consequences of discontinuing data collection would be that the millions of users who rely on O*NET data for business and career decisions, for educational programming, and for work in human resources or workforce development would instead be using portions of information that are out of date and incomplete. If data were collected less frequently, the currency of some data would become questionable, especially for occupations that are changing as a result of new technologies. The focus of data collection on high-growth and new and emerging occupations reflects the need to provide current information in a demand-driven economy that is rapidly changing.

A 3-year extension of the O*NET Data Collection Program is being requested for the period spanning the period July 2015 through June 2018. This extension will provide for the updating of selected high-growth occupations and for data collection activities for new and emerging occupations. A dynamic and progressive U.S. economy requires continuous improvement to the data on which so many decisions are based. Millions of people are currently using O*NET information, and the numbers continue to expand as public agencies and private developers integrate O*NET data into their systems and products. The O*NET database provides valid, reliable, and current occupational information crucial to a strong U.S. workforce. O*NET database updates are scheduled to occur once a year to incorporate newly collected information on recently surveyed occupations. A schedule for data analysis is provided in Section ‎A.16.1; schedules for data collection and analysis are subject to annual appropriations.

Special Circumstances

The study will be conducted in a manner consistent with the guidelines in 5 C.F.R. § 1320.9. There are no special circumstances that might require deviation from the guidelines.

Consultation Outside the Agency

The Federal Register notice initiating the 60-day public comment period was published on November 26, 2014. No comments were received from the public.

The data collection contractor, RTI International, has several mechanisms in place to obtain ongoing feedback from study participants. The BLs have multiple telephone contacts with POCs and occupation experts; they are careful to document in their call notes significant comments or suggestions they receive. Subject matter experts at professional associations we contact to help identify occupation experts frequently provide feedback that the National Center for O*NET Development uses to refine occupation descriptions and tasks. Letters and brochures sent to POCs and occupation experts offer a toll-free number they can call with questions or comments. In addition, the project Web site, https://onet.rti.org/, which both study participants and the general public can access, has a “contact us” tab that offers both a toll-free number and an e-mail link. Furthermore, both the paper questionnaires and the online Web questionnaires provide respondents an opportunity to submit comments with their questionnaire responses. All feedback received from these sources is promptly reviewed by project management staff. Of course, the agency will comply with all Paperwork Reduction Act requirements should comments warrant changing the information collection.

Respondent Incentives

Since the origins of the survey, incentives have been offered to POCs, establishments, and employees to encourage their participation in the O*NET Data Collection Program. Although the procedures are designed to minimize respondent burden, the effort for the company and the POC participation is not insignificant.

Incentives for the Point of Contact and the Employer

The POC’s responsibilities include:

reading the introductory package to become familiar with the purpose of the O*NET Data Collection Program and the role of a POC;
seeking permission within the company, as necessary, to participate in the O*NET Data Collection Program;
making a roster of all employees at the location who work in as many as five different occupations;
participating in a sampling process that selects as many as 20 total employees from these occupations and maintaining this sample roster for future reference;
distributing questionnaires to the sampled persons within the company and addressing their questions and concerns about the survey; and
distributing follow-up materials to employees, including thank you/reminder cards and replacement questionnaires, and following up with nonrespondents to encourage participation.

Because POCs are the only link with the O*NET respondents, they must be fully committed to the data collection process. The POC is the O*NET Program’s representative within the establishment who communicates the importance of the O*NET Program.

The employer is also essential because he or she is being asked to:

support the O*NET Data Collection Program by agreeing to the company’s participation in data collection,
provide support and encouragement to the POC to carry out his or her responsibilities,
allow the POC to provide information regarding the number of persons employed in the establishment in the occupations of interest, and
support the participation of the sampled employees.

Incentives for both POCs and employers are essential to encourage the highest response rates possible. If they agree to participate, POCs receive a framed Certificate of Appreciation from DOL. The Certificate of Appreciation is printed in color on card stock, bears the DOL Seal and O*NET logo, displays the POC’s name, and is signed by a high-ranking DOL official. It has a solid oak frame and Plexiglas cover and is suitable for displaying on an office wall.

The employer receives the O*NET Toolkit for Business if the establishment agrees to participate. The toolkit is an O*NET Program information packet, including a guide for writing job descriptions, that managers can use for human resource planning. These materials are attractively organized in colored, glossy folders.

The continuation of these incentives is planned for both the POC and the employer because they seem to be working quite well, as evidenced by ONET’s competitive employer response rate (see Section A.1.5). An experiment was conducted from 2002 to 2004 to evaluate an additional incentive of $20 to the POC. Essentially, the incentive had no effect on POC cooperation rates or employee response rates, but it significantly increased data collection costs. The experimental findings suggested that the current POC and employer incentives are adequate for maximizing response rates at a reduced cost (Biemer, Ellis, Pitt, & Robbins, 2006).¹⁹

Incentives for the Employee

In keeping with what has been done since 2001, each employee is offered a prepaid incentive of $10 to ensure that a high percentage of the job incumbents respond by completing the questionnaire.

Monetary incentives have the greatest potential impact when the respondent has to exert some special effort, such as taking a test or filling out a multi-item questionnaire. The incentive encourages respondents in a task requiring higher levels of involvement and commitment than the typical one-time, face-to-face interview. Although the O*NET questionnaires are not tests, the cognitive demands they place on respondents resemble test-taking demands in that the respondents must assess the requirements of their jobs. The monetary incentive is instrumental in impressing upon the respondent the importance of this rating task. Respondents who perceive the rating task as important will likely make thoughtful, carefully considered assessments rather than hasty ones, thus improving the reliability of the data.

In addition, the monetary incentive is important because respondents are encouraged to complete the questionnaire, which takes about 30 minutes, on their own time rather than on the job. This encouragement not only minimizes the burden on employers, but it also improves the quality of the data because, otherwise, busy workers might be underrepresented in the data, which would bias the estimates for job performance.

The monetary incentive may at least partially offset its inherent cost through efficiencies created in the data collection process as a result of higher response rates (Statistics Research Division, 2000, October). For the job incumbent respondents especially—although they are not viewed as a difficult-to-reach population in the usual sense—considerable effort and resources are expended to identify and reach them through the sampling process. Each one represents a worker in a specific occupation in a specific establishment in a specific industry. The expense of reaching that particular respondent justifies the cost of a monetary payment to ensure a high rate of response.

With regard to the size of the employee incentive, payment amounts were evaluated in the pretest to determine the optimal means to maximize the response rate. On the basis of these data, a $10 cash incentive has been used since the initial wave of data collection in 2001. An ongoing analysis of employee response rate trends over time was implemented in 2014. The analysis compares current employee response rates with prior response rates achieved by the same occupations in earlier data collection waves. According to this analysis, employee response rates have remained fairly stable over the years; thus, we plan to continue using the $10 incentive for the foreseeable future. If subsequent response rate analyses indicate a significant downward trend, we will reassess the amount of the incentive.²⁰

Incentives for Occupation Experts

Occupation experts provide data for approximately 25% of the O*NET-SOC occupations. Each occupation expert receives a prepaid incentive of $40 and a framed Certificate of Appreciation from DOL if they agree to participate. Unlike job incumbents, who complete only one domain questionnaire, occupation experts are asked to complete all three domain questionnaires; thus, the $40 incentive is about $13.33 per questionnaire. This incentive is slightly higher than the $10 offered to Establishment Method respondents for completing one domain questionnaire. The increased incentive and the Certificate of Appreciation are necessary to gain cooperation from what is often a rare group of experts for an occupation. Moreover, the additional incentives seem commensurate with the effort involved in responding to multiple questionnaires, given that occupation experts are supervisors and/or trainers in the occupation and, as such, earn a higher salary than the average employee.

Provision and Legal Basis for Confidentiality Assurances

The contractor for the data collection task, RTI International, has extensive experience in protecting and maintaining the privacy of respondent data collected from surveys. To ensure privacy, RTI has drawn from its experience in designing the data collection procedures incorporated in this program. In addition, RTI’s Institutional Review Board is bound by the O*NET contract, federal regulations, and company policy to review and approve the research protocol to ensure compliance with federal regulations (45 C.F.R. § 46) concerning data privacy and the protection of human subjects from research risks.

Respondents are informed that their individual responses will be kept private to the extent permitted by law. Survey data are collected from job incumbents (Establishment Method) and from occupation experts (OE Method). Informational materials mailed to potential respondents contain essential program information and assurances of privacy that enable the person to make an informed decision about his or her voluntary participation in the data collection effort. Examples of informational materials provided to survey participants appear in Appendix B.

Employees sampled at establishments are asked to complete their questionnaires on their personal time, not company time. This stipulation enables the employee to select a comfortable and private setting, if desired, in which to complete the questionnaire. All respondents have a choice of completing paper questionnaires or completing the questionnaires online at the project Web site. Paper questionnaires are mailed directly to RTI in a business-reply envelope provided by RTI. The individual responses are processed according to a study ID number. All O*NET Data Collection Program staff are required to sign a privacy pledge that assures each respondent that the privacy of responses to the questionnaire will be maintained. Only authorized staff have access to the completed instruments and data files. The completed and processed questionnaires are stored in a secure document-control area until federal authorization has been granted to destroy them. All computer files, including those associated with the control system, are password protected.

The Internet-based system that allows respondents to provide their survey responses electronically has restricted access, including a user ID and password authentication scheme for respondents. The Web server includes an SSL certificate to allow encrypted transmission of all information over the Internet, the same technology used by e-commerce sites. The database containing the survey data is not accessible via the Internet; it resides on a server inside the RTI firewall. Only O*NET Data Collection Program staff have access to the master survey database.

The O*NET questionnaires (see Appendix A) collect very little personal information about the respondent, and what is collected contains no identifiers, such as personal name or place of employment. No individual-level data are published, nor are they accessible or provided to anyone except the O*NET Data Collection Program staff. Published results are made available only in aggregate, as one set of estimates for an entire occupation. Furthermore, no demographic data (e.g., sex, race) are released, even in aggregate form. Finally, estimates are not produced for any subpopulations within an occupation, such as geographic region or sociodemographic group, which otherwise may have allowed the identification of an individual.

Before publishing the O*NET tables on the public Web site, the O*NET Program team thoroughly examines the tables for any risk of disclosure of private information. In particular, each table is analyzed to identify any “sensitive” cells (i.e., cells that may reveal too much information about an individual employee). Willenborg and De Waal (1996) have recommended using an (n,k)-dominance rule that a cell be regarded as sensitive if the sum of the largest n contributions account for more than k% of the total cell value. Willenborg and De Waal further recommend that n = 5 and k = 80. Because every sampled employee contributes only one response, these recommendations translate into a minimum cell size of 5/0.80, or about 7. In fact, the minimum cell size for the O*NET tables is 10 respondents; any sensitive cells with fewer than 10 are suppressed. In addition, the (n,k)-dominance rule assumes a complete census. As Willenborg and Waal note, when applied to tables based on samples and where the cell entries are weighted averages, the (5,0.80) rule affords even greater disclosure control. This extra control ensures that these O*NET tabular data pose no disclosure risks to any individual respondent.

Questions of a Sensitive Nature

Only four questions in the O*NET questionnaires may be considered to be of a sensitive nature. In the Background Questionnaire, the survey uses the four disability questions developed for the Census Bureau’s American Community Survey. The first two questions ask respondents about serious hearing and sight difficulties. The next question has sub-parts that ask about difficulties with concentration, memory, or decision making; mobility; and self-care. The last question asks whether respondents’ physical or mental health makes it difficult to do errands alone. Completion rates for these items indicate that the great majority of participants (96%) elect to complete the questions.

The O*NET sampling strategy is to randomly select participants at the individual level. The disability questions, together with the demographic questions, provide descriptive information about the sample of respondents.

Respondents to the O*NET survey are informed that responding to all questions, which includes disability status and the other demographic characteristics, is voluntary. They complete the survey on their own time, in a private setting if they choose. No identifying information, such as the respondent’s name or place of employment, is recorded on the questionnaire.

Estimates of Annualized Hour Burden

As described in Section ‎A.1.3, there are two protocols for O*NET data collection—the Establishment Method and the OE Method. An estimated 75% of occupations are completed by the Establishment Method. The OE Method is used for occupations as necessary to improve sampling efficiency and avoid excessive burden, as when it is difficult to locate industries or establishments with occupation incumbents; employment is low; or employment data are not available, as is the case for many new and emerging occupations.

The Establishment Method uses a two-stage sample, with establishments selected at the first stage and employees selected at the second stage. Thus, there are burden hours associated with both establishments and employees. For each participating establishment, a POC is identified to coordinate data collection activities in the establishment. In Exhibit 12, the first and second columns of the Establishment Activity section present the number of responses and assumed average burden per response for the POC’s activities; these averages were obtained from previous years of O*NET data collection experience. When the total establishment burden was estimated, the estimated number of establishments that will complete each activity was multiplied by the average burden and summed across the activities.

The Employee Activity section of Exhibit 12 is based on the fact that selected employees under the Establishment Method will complete one domain questionnaire requiring an average of 30 minutes of effort, whereas occupation experts will complete all three domain questionnaires, requiring an average of 90 minutes of effort.

Exhibit 12 also displays the estimated number of sampling units and the estimated burden hours. As shown in Exhibit 14 in Section ‎A.16, we assume that 75 occupations will be completed under the Establishment Method and 25 will be completed under the OE Method each year. From July 2015 through June 2016, establishments are estimated to expend 6,842 burden hours, and employees are estimated to expend 8,533 burden hours, for a total of 15,375 burden hours. From July 2016 through June 2017, establishments are estimated to expend 6,081 burden hours, and employees are estimated to expend 7,837 burden hours, for a total of 13,918 burden hours. From July 2017 through June 2018, 5,964 burden hours are estimated for establishments, and 8,353 burden hours are estimated for employees, for a total of 14,317 burden hours. The slight decline in total burden hours during the 2-year period July 2016 through June 2018 (13,918; 14,317) relative to the July 2015–June 2016 period (15,375) results from initiating data collection for most occupations during the period July 2015–June 2016, which causes many of these occupations to complete data collection during the final 2 years.

The Total Respondents section of Exhibit 12 shows the estimated annual number of respondents by category and overall. The respondent totals include the number of POCs (the row name is “Screening call to POC”), the number of Establishment Method employee respondents, and the number of OE Method respondents. The category totals are based on prior sampling experience. The total number of respondents across all 3 years is 86,598.

The Total Burden Hours section of Exhibit 12 shows the annual number of burden hours by category and overall. The burden hour totals include both establishment burden and employee burden. The category totals are based on the estimated number of respondents for each category, the number of responses, and the average burden per response. The total number of burden hours across all 3 years is 43,610. This 3-year total reflects a slight increase in burden compared with the previous 3-year period of 2012 to 2015, during which 41,011 burden hours were requested (U.S. Department of Labor, Employment & Training Administration, 2012).

O*NET Establishment Method data collection has been designed to minimize the burden on the selected establishments:

Establishments are asked about no more than 10 occupations each, with questioning terminated once 5, or sometimes fewer, occupations are identified at an establishment.
Establishments are asked to complete rosters of employees only for the five or fewer occupations identified.
Establishments are selected no more than once within a 12-month period.
No more than 20 employees are selected from an establishment across all of its selected occupations.
If an occupation proves difficult to complete under the Establishment Method, the dual-frame approach may be used to supplement it. For occupations that are difficult to sample, the alternative OE Method may be used.

Annual Reporting Burden Cost

There are no respondent costs for capital or start-up or for operations, maintenance, and purchase of services. There are no costs to the employers, POCs, or sampled employees other than the time it takes them to participate in the survey.

Estimates of Annualized Cost to Government

The estimated annual cost to the government for the O*NET Data Collection Program for the period July 2015–June 2018 is approximately $6.3 million. This estimate includes all direct and indirect costs of conducting the sampling, data collection, and analysis activities of the O*NET Data Collection Program. In the chart below, Personnel and Fringe Benefit costs are for the grantee (NC State) personnel who manage the O*NET Data Collection Program; Contractual costs go to contractors that the grantee subcontracts to that support the program activities (e.g. survey operations and data analysis).

Federal Staff Costs(GS 14 Step 4 [$118,057] + GS 13 Step 4 [$99,905] x .5 [part-time])	$ 108,981
Personnel	$ 497, 860
Fringe Benefits	$ 142,379
Travel	$ 5,000
Equipment	$ 3,000
Supplies	$ 46,500
Contractual	$ 5,261,701
Costs for Incentives	$ 164,000
Total Direct Costs	$6,229,421
Indirect Charges	$ 79,560
Total	$ 6,308,981

Reasons for Program Changes or Adjustments Reported in Sections A.13 and A.14

Exhibit 13 compares the projected burden hours for the period July 2015–June 2018 with the average annual burden as estimated for the period June 2012–May 2015 in the OMB Supporting Statement dated April 10, 2012. The projected total annual burden hours for July 2015–June 2018 range from 13,918 to 15,375. The average annual burden is 14,536 hours, compared with an average annual burden of 13,671 hours requested for the previous 3-year period (2012–2015). In addition, as indicated in Exhibit 12, the total burden hours for the July 2015–June 2018 period, 43,610, reflect a slight increase in burden compared with the 2012–2015 period, for which a total 41,011 hours were requested (U.S. Department of Labor, Employment & Training Administration, 2012). The small increase in burden corresponds to an increase in the number of establishment sampling units that are contacted each year, an estimated 49,667 for the July 2015–June 2018 period compared with 43,500 for the June 2012–May 2015 period in the April 10, 2012, OMB Supporting Statement. The slight increase in the number of sampled establishments is attributable to changes in the schedule for fielding cases (occupations), which is impacted by eligibility rates and other sampling characteristics of the specific occupations being studied.

Exhibit 12. Estimate of Hour and Cost Burden by Year

	Number of Responses per Sample Unit	Average Burden per Response (Minutes)	July 2015 – June 2016		July 2016 – June 2017		July 2017 – June 2018
	Number of Responses per Sample Unit	Average Burden per Response (Minutes)	Sampling Units	Burden Hours	Sampling Units	Burden Hours	Sampling Units	Burden Hours
Establishment Activity
Verification calls to initial contact at establishment	1	2	18,000	600	16,000	533	15,667	522
Screening call to POC	1	3	14,933	747	13,274	664	12,997	650
Initial recruitment call to POC	1	12	8,213	1,643	7,300	1,460	7,148	1,430
POC creation of occupation lists for sampling	1	20	4,517	1,506	4,015	1,338	3,932	1,311
Call to POC to sample workers	1	10	4,517	753	4,015	669	3,932	665
POC’s distribution of questionnaire packets	1	15	4,156	1,039	3,694	924	3,617	904
Follow-up calls to POC	4	2	4,156	554	3,694	493	3,617	482
Total, establishment^a	NA	NA	NA	6,842	NA	6,081	NA	5,964
Employee Activity
Establishment Method employee respondents	1	30	15,040	7,520	13,648	6,824	14,681	7,340
Occupation Expert Method respondents	3	30	675	1,013	675	1,013	675	1,013
Total, employee^b	NA	NA	NA	8,533	N/A	7,837	NA	8,353
Total Respondents
Private sector	NA	NA	13,644	NA	12,128	NA	11,875	NA
Federal government	NA	NA	963	NA	856	NA	838	NA
State/local/tribal governments	NA	NA	326	NA	289	NA	283	NA
Subtotal, establishment respondents	NA	NA	14,933	NA	13,274	NA	12,997	NA
Individuals/households	NA	NA	15,715	NA	14,323	NA	15,356	NA
Total, all respondents^c,d	NA	NA	30,648	NA	27,597	NA	28,353	NA
Total Burden Hours
Private sector	NA	NA	NA	6,252	NA	5,556	NA	5,449
Federal government	NA	NA	NA	441	NA	392	NA	385
State/local/tribal governments	NA	NA	NA	149	NA	133	NA	130
Subtotal, establishment burden hours	NA	NA	NA	6,842	NA	6,081	NA	5,964
Individuals/households	NA	NA	NA	8,533	NA	7,837	NA	8,353
Total, all burden hours^e	NA	NA	NA	15,375	NA	13,918	NA	14,317

continued

Exhibit 12. Estimate of Hour and Cost Burden by Year (continued)

Number of Responses per Sample Unit

Average Burden per Response (Minutes)

July 2015 – June 2016

July 2016 – June 2017

July 2017 – June 2018

Sampling Units

Burden Hours

Sampling Units

Burden Hours

Sampling Units

Burden Hours

Time Burden Monetized

Respondent Type

July 2014–June 2016

July 2014–June 2017

July 2014–
June 2018

Establishments^f

$366,047

$331,597

$331,419

Employees^g

$263,158

$246,317

$267,547

Total

$629,205

$577,914

$598,966

Note: NA = not applicable.

^aIncludes total burden time for all establishments (private sector, federal government, and state/local/tribal governments).

^bIncludes total burden time for individuals/households.

^cThe total number of respondents across all 3 years = 86,598.

^dThe total number of respondents was derived by summing the number of POCs (row name is “Screening call to POCs”), Establishment employees (row name is “Establishment Method employee respondents”), and occupation experts (row name is “Occupation Expert Method respondents”).

^eThe total number of burden hours across all 3 years = 43,610.

^fAssumed hourly total compensation rates: $53.50 for July 2015–June 2016; $54.53 for July 2016–June 2017; $55.57 for July 2017–June 2018.

^gAssumed hourly total compensation rates: $30.84 for July 2015–June 2016; $31.43 for July 2016–June 2017; $32.03 for July 2017–June 2018.

Source: Bureau of Labor Statistics, U.S. Department of Labor. (2014, March). Employer costs for employee compensation summary. Available from Databases, tables and calculators by subject, http://data.bls.gov/timeseries/CIS1010000000000Q

Exhibit 12 displays the estimated annualized cost to respondents for burden hours by year. The cost burden was estimated with use of average total compensation rates obtained from the December 2013 Employer Costs for Employee Compensation Summary issued by BLS on March 12, 2014 (Bureau of Labor Statistics, 2014, March). The average total compensation per hour for private industry was $29.63, which was inflated based on the Employment Cost Index to a median hourly total compensation of $30.84 for July 2015–June 2016, $31.43 for July 2016–June 2017, and $32.03 for July 2017–June 2018. These are the total compensation rates used for estimating the employee cost burden. Given that the POC will often be a human resources manager, the December 2013 total compensation rate of $51.41 for the Management, Professional and Related category was inflated to a median hourly total compensation of $53.50 for July 2015–June 2016, $54.53 for July 2016–June 2017, and $55.57 for July 2017–June 2018 for estimating the establishment cost burden.

Exhibit 12 shows that the combined establishment and employee total cost burden is $629,205 for July 2015–June 2016, $577,914 for July 2016–June 2017, and $598,966 for July 2017–June 2018. As with the burden hours, the slight decrease in total cost burden across the 2‑year period July 2016–June 2018 relative to the July 2015–June 2016 period results from initiation of data collection for most occupations during July 2015–June 2016, which causes many of these occupations to complete data collection during the final 2 years.

Exhibit 13. Comparison of Hour and Cost Burden Between 2012–2015 and July 2015–June 2018

	Annual Average 2012–2015^a		July 2015–June 2016			July 2016–June 2017			July 2017–June 2018
	Sampling Units	Burden Hours	Sampling Units		Burden Hours	Sampling Units		Burden Hours	Sampling Units		Burden Hours
Establishment Activity
Verification calls to initial contact at establishment	14,500	484	18,000		600	16,000		533	15,667		522
Screening call to POC	12,035	602	14,933		747	13,274		664	12,997		650
Initial recruitment call to POC	6,980	1,396	8,213		1,643	7,300		1,460	7,148		1,430
POC creates occupation lists for sampling	3,909	1,303	4,517		1,506	4,015		1,338	3,932		1,311
Call to POC to sample workers	3,909	651	4,517		753	4,015		669	3,932		655
POC distributes questionnaire packets	3,674	919	4,156		1,039	3,694		924	3,617		904
Follow-up calls to POC	3,674	490	4,156		554	3,694		493	3,617		482
Total establishment		5,845	NA		6,842	NA		6,081	NA		5,964
Employee Activity
Establishment Method employee respondents	13,552	6,776	15,040		7,520	13,648		6,824	14,681		7,340
Occupation Expert Method respondents	700	1,050	675		1,013	675		1,013	675		1,013
Total, employee	NA	7,826	NA		8,533	NA		7,837	NA		8,353
Total, Establishments and Employees
Total		13,671			15,375			13,918			14,317
	Annual Average 2012–2015^a			Cost Burden
				July 2015–June 2016			July 2016– June 2017			July 2017–June 2018
Respondent Type
Establishments^b	$306,557			$366,047			$331,597			$331,419
Employees^c	$230,264			$263,158			$246,317			$267,547
Total	$536,821			$629,205			$577,914			$598,966

^a From the OMB Supporting Statement dated April 10, 2012.

^b Assumed hourly total compensation rates: $53.50 for July 2015–June 2016; $54.53 for July 2016–June 2017; $55.57 for July 2017–June 2018.

^c Assumed hourly total compensation rates: $30.84 for July 2015–June 2016; $31.43 for July 2016–June 2017; $32.03 for July 2017–June 2018.

Exhibit 13 compares the estimated costs to respondents for July 2015–June 2018 with the average annual cost estimated for 2012–2015. The annual costs have increased since 2012–2015, primarily because of inflation in the benefits portion of employee compensation and also because of the increase in the number of sampling units being contacted per year relative to the 2012–2015 period.

A few minor questionnaire revisions have been implemented since 2012. In addition, several minor questionnaire revisions for the Knowledge Questionnaire and the Background Questionnaire are pending in this submission. All of these revisions are described in detail in Appendix A. These minor revisions do not represent an increase in respondent burden.

Time Schedule, Publication, and Analysis Plans

The major components of the O*NET Data Collection Program include sampling, data collection operations, and analysis. Exhibit 14 shows the expected schedule for annual data analysis cycles and data publications for the next several years.

Exhibit 14. Data Analysis and Publication Schedule

Analysis Cycle	Analysis Cycle Start Date	Analysis Cycle End Date	*Estimated Establishment Method ONET-SOC Occupations**	*Estimated Occupation Expert Method ONET-SOC Occupations**	*Estimated ONET-SOC Occupations Published**	Publication Date
17	July 2015	July 2016	75	25	100	July 2016
18	July 2016	July 2017	75	25	100	July 2017
19	July 2017	July 2018	75	25	100	July 2018

Data Analysis Tasks Conducted for Each Cycle

Described here is the approach used for data cleaning and editing, as well as the analyses that are performed annually.

Data Cleaning

Paper questionnaires are manually and machine edited so that completely blank questionnaires are dropped; responses to items that should have been skipped are blanked out; multiple responses are blanked out; and codes indicating missing data, multiple responses, and legitimate skips are inserted. Codes for legitimate skips and missing responses are also inserted in the records for questionnaires obtained through the Web. In addition, an electronic check is conducted to detect duplicate questionnaires from the same respondent.

Identification and Evaluation of Anomalous Cases

Analyses of the questionnaire ratings are based on the assumption that raters were qualified, willing, and able to engage in the rating tasks. This task is accomplished by requiring each case to first pass through a series of machine edits using prescribed eligibility criteria, including having at least one task rated important and having at least 50% of the domain questionnaire items completed. Cases not meeting these criteria are excluded from the analysis file. Cases with certain questionable characteristics are flagged in this editing process, and analysts review these cases to determine their completion status. Flagged for review are all cases for which the respondent (1) indicated in the “global match” item that the target O*NET-SOC occupation description did not at all describe his or her own job, and (2) rated fewer than one third of the tasks as important. An analyst reviews the self-reported job titles of these cases to determine if they are at all likely to belong in the O*NET-SOC occupation. If a case does not belong, it is removed from further analysis. If, in the judgment of the analyst, there is a reasonable chance that the case belongs in the O*NET-SOC occupation, it is sent to the next stage of review.

Finally, cases that pass the machine edits and the analyst review are subjected to a deviance analysis designed to identify cases that are outliers relative to other cases in their occupation. The deviance analysis involves two procedures: (1) a statistical procedure to quantitatively identify potential outliers and (2) an analyst review of these potential outliers to make the final decision for each case. Cases that do not pass the analyst review are deemed deviant within their occupation and are removed from the data set. On average, these activities eliminate about 9% of all returned questionnaires. The cases passing all data cleaning criteria are used to create the estimates for publication.

Computation of Sampling Statistics

Basic sampling weights are applied to the data to make inferences to the population of incumbents for each occupation.²¹ These weights are computed as the inverses of the overall selection frequencies and the selection probabilities for each selected establishment and each individual participant. The analysis weights for the eligible sample units are adjusted to compensate for unit nonresponse for both establishments and employees, multiple subwaves of sampling, and sample adjustments. In addition, when variation in the weights is large, the weights are trimmed to reduce the variation. To maximize comparability of O*NET estimates to estimates from other federal sources, the final sample weights are also ratio-adjusted to occupation estimates obtained from the OES survey.

Sampling errors are computed. The analysis weights used in the sampling error computations, as noted, have been adjusted for nonresponse and are consistent with the complex sampling design.

Calculation of Descriptor Values and Reliability

For each occupation, the sample size, mean, standard deviation, and standard error of the ratings for each descriptor are calculated, together with the 95% confidence interval around the mean. Estimates with questionable precision are flagged and recommended for suppression if any of the following conditions is true:²²

The sample size (i.e., number of respondents who answered the question) is less than 10.
The variance is zero and the sample size is less than 15.
The relative standard error (RSE) is greater than 50%.²³

On average, approximately 2.5% of the estimates are flagged or suppressed.

Interrater Reliability and Agreement

Interrater reliability and agreement are assessed with three different analyses. The first two measure reliability in terms of the covariation among ratings, and the third analysis reflects rater agreement. The intraclass correlation coefficients (ICCs) for each questionnaire item across all occupations are computed. These results allow one to compare respondent rating variance within an occupation with respondent rating variance across occupations. In addition to the ICCs, the mean interrater correlations (Pearson’s r) are calculated for all pairs of raters within each occupation. Finally, to assess the absolute difference among ratings of each item within each occupation, an average deviation index is calculated for each item within each occupation. For any given item and occupation, the average deviation index measures the average extent to which each individual rating deviates from the item mean. Some differences in ratings within occupations are expected because O*NET-SOC occupations comprise a variety of different jobs in most cases.

Each of the reliability analyses conducted (rater, standard errors) is influenced by the number of respondents. The O*NET data collection methods include a sufficient number of respondents in each occupation to ensure reliability (Peterson, Mumford, Levin, Green, & Waksberg, 1997).

Creation of the Occupation Database

The O*NET database is scheduled to be updated annually. Each update will include data for those occupations collected and analyzed during the previous 12-month period. Consequently, a database update includes occupations from multiple data collection waves, depending on the number of prior waves for which analysis was completed that year. For each occupation collected, the newly calculated means data replace existing analyst-based or incumbent-based means data in the database. Users are provided with metadata regarding when the data were collected and any other pertinent information that will assist the users in interpreting the data.

The O*NET database is designated with a version number denoting each update (e.g., from O*NET 17.0 to O*NET 18.0). The database is developed and administered with the MySQL database management system. Once the MySQL database is updated, it is used to generate the database for public release as a series of flat text files. It is accessible to the public on the O*NET Program Web site at http://www.onetcenter.org/.

Display of Expiration Date

The expiration date will be displayed on the cover of the survey questionnaires.

Exceptions to Certification Statement

There are no exceptions.

References

Abel, J. R., Gabe, T. M., & Stolarick, K. (2012, February). Workforce skills across the urban-rural hierarchy. (Federal Reserve Bank of New York Staff Report No. 552). Retrieved from http://www.newyorkfed.org/research/staff_reports/sr552.pdf

ACT. (2006). Ready for college and ready for work: Same or different?). Retrieved from http://www.act.org/research/policymakers/pdf/ReadinessBrief.pdf

Alssid, J. L., & Goldberg, M. (2013, November). Rise of the medical assistant and five other frontline and nonclinical healthcare jobs that are growing in number,complexity, and importance. Manchester, NH: College for America. Retrieved from http://collegeforamerica.org/reports/WSC_Healthcare_report.pdf

Alterman, T., Grosch, J., Chen, X., Chrislip, D., Petersen, M., Krieg, E., Jr., Chung, H., & Muntaner, C. (2008). Examining associations between job characteristics and health: linking data from the Occupational Information Network (O*NET) to two U.S. national health surveys. [Research Support, U.S. Gov't, P.H.S.]. Journal of Occupational and Environmental Medicine, 50(12), 1401-1413. doi:10.1097/JOM.0b013e318188e882

Anseel, F., Lievens, F., Schollaert, E., & Choragwicka, B. (2010). Response rates in organizational science, 1995-2008: A meta-analytic review and guidelines for survey researchers. Journal of Business Psychology, 25, 335-349.

Baruch, Y., & Holtom, B. C. (2008). Survey response rate levels and trends in organizational research. Human Relations, 61, 1139-1160.

Biemer, P., Ellis, C., Pitt, A., & Robbins, K. (2006). Effects on response rates and costs of a monetary incentive for the point of contact in an establishment survey. Paper presented at the Proceedings of the American Association of Public Opinion Research, Montreal, Canada.

Bureau of Labor Statistics, U.S. Department of Labor (2014, March). Employer costs for employee compensation summary. Available from Databases, tables and calculators by subject). Retrieved from http://data.bls.gov/timeseries/CIS1010000000000Q

Burrus, J., Jackson, T., Xi, N., & Steinberg, J. (2013, November). Identifying the most important 21st century workforce competencies: An analysis of the Occupational Information Network (O*NET). (Research Report ETS RR–13-21). Princeton, NJ. Educational Testing Service. Retrieved from http://www.ets.org/Media/Research/pdf/RR-13-21.pdf

Campion, M. A., Fink, A. A., Ruggeberg, B. J., Carr, L., Phillips, G. M., & Odman, R. B. (2011). Doing competencies well: Best practices in competency modeling. Personnel Psychology, 64, 225-262.

Cole, M. S., Bedeian, A. G., & Feild, H. S. (2006). The measurement equivalence of Web-based and paper-and-pencil measures of transformational leadership. Organizational Research Methods, 9, 339-368.

Couper, M. (2008). Designing effective Web surveys. New York, NY: Cambridge University Press.

Dierdorff, E. C., Norton, J. J., Drewes, D. W., Kroustalis , C. M., Rivkin, D., & Lewis, P. (2009). Greening of the world of work: Implications for O*NET-SOC and new and emerging occupations. Raleigh, NC: National Center for O*NET Development. Retrieved from http://www.onetcenter.org/dl_files/Green.pdf

Elliott, S. W. (2007, May). Projecting the impact of computers on work in 2030. Paper presented at the Workshop on Research Evidence Related to Future Skill Demands Center for Education, National Research Council, Washington, DC.

Evanoff, B., Zeringue, A., Franzblau, A., & Dale, A. M. (2014). Using job-title-based physical exposures from O*NET in an epidemiological study of carpal tunnel syndrome. Human Factors, 56, 166-177.

Fleishman, E. A., & Mumford, M. D. (1988). The ability requirements scales. In S. Gael (Ed.), The job analysis handbook for business, industry, and government. New York: Wiley.

Hanna, J. (2008, December 8). How many U.S. jobs are "offshorable"? (Harvard Business School Working Knowledge newsletter). Retrieved from http://hbswk.hbs.edu/item/6012.html

Hanser, L. M., Campbell, J., Pearlman, K., Petho, F., Plewes, T., & Spenner, K. (2008). Final report of the panel on the Department of Defense human capital strategy. Report prepared for the Office of the Secretary of Defense by the RAND Corporation.

Johnson, R. W., Mermin, B. T., & Resseger, M. (2007, November). Employment at older ages and the changing nature of work (Urban Institute Report No. 2007-20). Washington, DC: AARP.

Klein, R. J., Proctor, S. E., Boudreault, M. A., & Tuczyn, K. M. (2002). Healthy People 2010 criteria for data suppression. Hyattsville, MD: National Center for Health Statistics.

Koys, D. J. (2013). Using the Department of Labor's O*NET database in teaching HRM. Journal of Human Resources Education, 7(3/4), 1-21.

Landrum, R. E. (2009). Finding jobs with a psychology bachelor’s degree: Expert advice for launching your career. Washington, DC: American Psychological Association.

Lanz, R., Miroudot, S., & Kyvik Nordås, H. (2012). Trade in tasks. In D. Lippoldt (Ed.), Policy priorities for international trade and jobs (pp. 237-248). Paris, France: Organisation for Economic Cooperation and Development, International Collaborative Initiative on Trade and Employment. Available from http://www.oecd.org/site/tadicite/

Leeuw, A. (2006). The butcher, the baker and the candlestick-maker revisited: Indiana's new skills-based career clusters. INcontext, 7(12).

Leung, Y. T., & Glissman, S. M. (2010). A clustering approach to the organization design of knowledge-intensive service providers (RJ10478). Retrieved from http://domino.research.ibm.com/library/cyberdig.nsf/papers/58C6B1D509E8DCA68525780000603960

Lowenthal, A. B., & Bassett, D. S. (2012). Transition Assessment: Using Gap Analysis to Enhance Effective Transition Planning. Intervention in School and Clinic, 48(1), 30-37. doi:10.1177/1053451212443131

McCloy, R. A., Byrum, C. N., Munoz, C. S., & Tsacoumis, S. (2006). Generation of occupation-specific personality, skill, and aptitude profiles for the Career Decision-Making System (FR-06-07)). Retrieved from http://www.careerdecisionmaker.com/

Navarro, A., King, K. E., & Starsinic, M. (2011, September). Comparison of the American Community Survey voluntary versus mandatory estimates. Final report. Washington, DC. U.S. Census Bureau, American Community Survey Research and Evaluation Program. Retrieved from http://www.census.gov/acs/www/Downloads/library/2011/2011_Navarro_01.pdf

Paullin, D., Ingerick, M., Tripp, D. M., & Wasco, L. (2011, December). Identifying best bet entry-level selection measures for US Air Force remotely piloted aircraft (RPA) pilot and sensor operator (SO) occupations. Randolph Air Force Base, TX: Air Force Personnel Center. Retrieved from http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA554209

Paxon, M. C., Dillman, D. A., & Tarnai, J. (1995). Improving response to business mail survey. In B. G. Cox (Ed.), Business survey methods. New York: Wiley.

Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., & Fleishman, E. A. (1995). Development of prototype Occupational Information Network (O*NET) content model. Salt Lake City: Utah Department of Workforce Service through a contract with the American Institutes of Research.

Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., Fleishman, E. A., & Levin, K. Y. (Eds.). (1997). O*NET final technical report. Salt Lake City: Utah Department of Workforce Services through a contract with American Institutes for Research.

Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., Fleishman, E. A., Levin, K. Y., Campion, M. A., Mayfield, M. S., Morgeson, F. P., Pearlman, K., Gowing, M. K., Lancaster, A. R., Silver, M. B., & Dye, D. M. (2001). Understanding work using the Occupational Information Network (O*NET): Implications for practice and research. Personnel Psychology, 54(2), 451–492. doi:10.1111/j.1744-6570.2001.tb00100.x

Peterson, N. G., Mumford, M. D., Levin, K. Y., Green, J., & Waksberg, J. (1997). Research method: Development and field testing of the content model. In N. G. Peterson, M. D. Mumford, W. C. Borman, P. R. Jeanneret & K. Y. Levin (Eds.), O*NET final technical report. Salt Lake City: Utah Department of Workforce Services, through a contract with the American Institutes for Research.

Peterson, N. G., Mumford, M. D., Levin, K. Y., Green, J., & Waksberg, J. (1999). Research method: Development and field testing of the content model. In N. G. Peterson, M. D. Mumford, W. C. Borman, P. R. Jeanneret & E. A. Fleishman (Eds.), An occupational information system for the 21st century: The development of O*NET (pp. 31–47). Washington, DC: American Psychological Association.

Peterson, N. G., Owens-Kurtz, C., Hoffman, R. G., Arabian, J. M., & Whetzel, D. C. (1990). Army synthetic validation project. Alexandria, VA: U.S. Army Research Institute for the Behavioral Sciences.

Petroni, R., Sigman, R., Willimack, D., Cohen, S., & Tucker, C. (2004). Response rates and nonresponse in BLS and Census Bureau establishment surveys. In Proceedings of the Survey Research Methods Section (pp. 4159–4166). Alexandria, VA: American Statistical Association.

Renski, H., & Wallace, R. (2012). Workforce skills and the changing knowledge economy in Massachusetts. MassBenchmarks, 14(1).

Rothwell, J. (2013, June). The hidden STEM economy. Washington, DC: The Brookings Institution. Retrieved from http://www.brookings.edu/research/reports/2013/06/10-stem-economy-rothwell

RTI International (Producer). (2004). SUDAAN language manual (Release 9.0).

Statistics Research Division, Research Triangle Institute. (2000, October). O*NET report: Results of statistical analysis of pretest. Research Triangle Park, NC. Retrieved from

Strietska-Ilina, O., Hofmann, C., Durán Haro, M., & Jeon, S. (2011). Skills for green jobs: A global view. Synthesis report based on 21 country studies. Geneva, Switzerland: International Labour Office. Retrieved from http://www.ilo.org/global/publications/books/WCMS_159585/lang--en/index.htm

Swaney, K. B., Allen, J., Casillas, A., Hanson, M. A., & Robbins, S. B. (2012). Interests, work values, and occupations: Predicting work outcomes with the WorkKeys fit assessment. Journal of Career Assessment, 20, 359-374.

Tarnai, J., & Paxon, M. C. (2004). Survey mode preferences of business respondents. Paper presented at the Proceedings of the American Association of Public Opinion Research, Phoenix, AZ.

Thompson, K. R., & Koys, D. J. (2010). The management curriculum and assessment journey: Use of Baldridge criteria and the Occupational Network Database. Journal of Leadership and Organizational Studies, 17(2), 156–166.

Tracey, J. B., Sturman, M. C., & Tews, M. J. (2007). Ability versus personality: Factors that predict employee job performance. Cornell Hotel and Restaurant Administration Quarterly, 48(3), 313-322.

Treasury Board of Canada Secretariat. (2009, May). Public servants on the public service of Canada: Summary of the results of the 2008 Public Service Employee Survey. Retrieved from http://www.tbs-sct.gc.ca/pses-saff/2008/report-rapport-eng.asp#appa RESOURCEID = 16339 Retrieved from http://www.tbs-sct.gc.ca/pses-saff/2008/report-rapport-eng.asp#appa RESOURCEID = 16339

Tsacoumis, S., & Van Iddekinge, C. H. (2006). A comparison of incumbent and analyst ratings of O*NET skills. Alexandria, VA: Human Resources Research Organization.

Tulp, D. R., Jr., Hoy, E., Kusch, G., & Cole, S. (1991). Nonresponse under mandatory vs. voluntary reporting in the 1989 survey of pollution abatement costs and experiments (PACE). In Proceedings of the American Statistical Association, Section on Survey Research Methods (pp. 272–277). Washington, DC: American Statistical Association.

U.S. Department of Labor, Bureau of Labor Statistics, and U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. (2003). Respirator usage in private sector firms). Retrieved from http://www.cdc.gov/niosh/docs/respsurv/

U.S. Department of Labor, Employment & Training Administration. (2012). O*NET data collection program: Office of Management and Budget clearance package supporting statement. Washington, DC: Author.

U.S. Office of Personnel Management. (2010). Federal Employee Viewpoint Survey. Washington, DC: Author. Retrieved from http://www.fedview.opm.gov/

Willenborg, L., & De Waal, T. (1996). Statistical disclosure control in practice series: Lecture notes in statistics (Vol. 111). New York, NY: Springer.

Worden, G., & Hoy, E. (1992). Summary of nonresponse studies conducted by industry division, 1989–1991. U.S. Bureau of the Census. Washington, DC.

1 For details on the uses of the O*NET Program, see Section A.2. More information about the O*NET Data Collection Program can be found at the National O*NET Program’s public Web site, http://www.onetcenter.org/ (the O*NET portal page that links to several O*NET-related Web sites) and at http://www.doleta.gov/programs/
onet (Web site of the Employment and Training Administration at the U.S. Department of Labor).

2 For detailed information on these methods, see Sections B.1 and B.2 in Part B.

3 For a discussion of the preferred data source, see Section A.1.2.

4 See Revision of O*NET data collection instruments, available at http://www.onetcenter.org/resData.html.

5 The goal of the study was to compare the psychometric quality of Incumbent Skills ratings with that of Analyst Skills ratings across a large sample of O*NET-SOCs. Although some mean differences between Incumbent and Analyst ratings were observed, the results yielded only minimal differences between the two systems of obtaining Skills information.

6 For a description of the pretest, see Section B.4.1 in Part B.

7 For a discussion of current and future efforts to improve response rates, see Section B.3 in Part B.

8For additional information about sampling with the Establishment Method, see Section B.1.2 in Part B.

9 For additional information about sampling with the OE Method, see Section B.1.3 in Part B.

10 See Exhibit 2 for the list of questionnaires, number of items and scales, and data sources.

11 For a description of both processes, see Section A.16.1.

12 For more information on the calculation of weights and variance estimates, see Section B.1.1, Part B. For information on the suppression of estimates, see Section A.16.1.

13 For a description of analysis processes, see Sections A.16.1 and A.16.2.

14 See Section B.3 in Part B for a description of methods used to maximize response rates and evaluate the potential for nonresponse bias.

15 Green occupations will likely change as a result of the green economy. Green economy activities and technologies are increasing the demand for occupations, shaping the work and worker requirements needed for occupational performance or generating new and emerging occupations. (See Dierdorff et al. (2009)].)

16 Bright outlook occupations are new and emerging occupations, those expected to grow rapidly in the next several years, or those expected to have large numbers of job openings.

17 See Appendix C: Advisory Panel for the Dictionary of Occupational Titles: Final Report.

18 See Section B.4.10, Part B, for a description of ATD.

19 For a more detailed discussion of the Biemer et al. (2006) results, see Section B.4 in Part B.

20 See Section B.4.12, Part B, for more information about the response rate analysis initiative.

21 For details about the weighting and estimation procedures, see Section B.1.1, Part B.

22 O*NET suppression criteria are based on substantive expert recommendations (Peterson, Mumford, Borman, et al., 1997) and on other large government surveys (see Klein, Proctor, Boudreault, & Tuczyn, 2002).

23 RSE = the standard error of the mean divided by the mean.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-25