US Data Release Notes

This section outlines major changes coming with the next datarun that users should be aware of.

Upcoming Changes: 2020.4

Changes to the QCEW Class of Worker

In an effort to continually improve our core LMI industry data, this quarter we revisited and improved the processes that create Emsi industry data based on the Quarterly Census of Employment and Wages. In summary, the improvements will be visible as more reasonable earnings estimates, fewer unreasonably small employment numbers (less than one job), and fewer jobs and earnings arbitrarily added where QCEW published zeros for certain establishments. Due to these changes, users should not expect historical data to match previous Emsi dataruns except where that historical data was completely disclosed by QCEW. There may also be slight differences between published annual data from QCEW and Emsi’s employment estimates.

Growth/Openings Calculation Bug

We fixed a bug in how we calculated growth for occupational employment. Growth is part of the calculation for openings, so openings were also affected. Previously, if employment was zero for a year and greater than zero for the following year, growth was incorrectly calculated to be zero. We’ve corrected the calculation so growth now equals the employment in the following year. Nationwide for all occupations, the amount of growth we were previously dropping was between 6,000 and 16,000 jobs for historical years and between 500 and 3,000 for projected years. The maximum percentage error was just under 0.6%.

QCEW Supplemental Earnings

We improved our process for estimating supplemental earnings for QCEW industry data to reduce the volatility in areas where our BEA source data was suppressed. The new methodology relies more heavily on the parent geography’s supplemental earnings data and thus will more closely match the source data.

Veterans Population Bug (API only)

We fixed a bug in 2009 veterans population data that was overstating the veteran population by roughly 5x. We pull veterans population counts from the Census Bureau’s ACS API, and failed to notice that the variable definitions changed between 2009 and 2010. Only latest year (currently 2018) veterans population data is currently shown in our tools, so only API users may notice a change.

Geography Constraint Changes in Core LMI API (API only)

We’ve simplified the interface for accessing different levels of geography. For example, instead of querying a separate endpoint – Emsi.US.Industry.ZIP – for ZIP code industry data, users can simply prepend “ZIP” to geography constraints sent to Emsi.US.Industry. MSAs are also available by prepending “MSA” to geography constraint. For more details and a full list of datasets that now support this simplified interface, see https://api.emsidata.com/updates/core-lmi-changelog.

Distance Completions for 2012

We neglected to include 2012 IPEDS distance completions data when we originally began publishing this data in 2013. Thanks to an observant client pointing this out, we now include 2012 data in our distance completions dataset.

Extended Proprietors Updates

We made some minor modifications to the methodology used to estimate our Extended Proprietors (Class of Worker 4) data to improve its quality and reliability. The immediate effect of the changes is minor: industry earnings per job increased by 0.2%

IRS Migration Data Improvements

The underlying data behind our population migration data is based on IRS tax exemptions, which do not always represent a person. Following a recommendation from the IRS, we now multiply the number of exemptions by 0.9 to better estimate the actual number of people migrating between counties.

Occupation Data Changes (API only)

We’re working hard to unify related datasets to simplify the API user experience. Part of that effort is visible in this release. API users can now access ZIP code and MSA data directly from the Emsi.US.Occupation endpoint by prefixing ZIP codes with “ZIP” and MSA codes with “MSA”.  Nation, state, and county FIPS codes do not need to be prefixed and will work exactly as they did before (and don’t worry, we didn’t get rid of Emsi.US.Occupation.ZIP yet).

State Minimum Wage Bug Fix

The minimum wage data that informs our occupational earnings models formerly contained the state minimum wage even if it was lower than the mandated federal minimum wage. We’ve modified the minimum wage for such states so that they now match the federal minimum wage. The increased minimum wage will be the lower bound for earnings percentiles in the lowest earning occupations.

Staffing Pattern Improvements

We made some improvements to our staffing pattern process to improve the loading speed of staffing pattern reports. This change will result in extremely small employment numbers clustering more tightly around disclosed data and being less widely distributed (e.g. there will be more occupations with three jobs and fewer occupations with one job).

Improved ACS Methodology

For certain records, ACS cannot determine a specific industry to classify a job as, so they place these indeterminate records in less-detailed “catchall” categories. Emsi now distributes these values among the industries represented by the catchall, instead of dropping these records. This will result in 50k-80k jobs, depending on the year, being added to self-employed.

Improved Earnings Estimations

Emsi has reduced the volatility of earnings in occupation data for areas with small employment. Year over year data should be more continuous and less erratic as a result.

Occupation Hires and Separations Improvements

Previous to this release, Emsi estimated occupation hires and separations using a combination of industry data from QWI and Emsi employment staffing patterns, implicitly assuming that hires and separations rates within an industry were the same across all occupations. Emsi’s new methodology estimates occupation hires and separations by combining QWI industry hires and separations with national occupation separation rates from the BLS and regional staffing pattern growth and decline. This change will not take effect immediately in Emsi’s tools, but will be released and messaged sometime later in the quarter.

Staffing Pattern Area Changes

In order to improve the quality and stability of their estimates, the BLS reduced the level of geography detail available in the 2018 release of Occupational Employment Statistics (OES):

Consolidation of some nonmetropolitan areas. Some nonmetropolitan areas published in the May 2017 estimates have been combined to form larger nonmetropolitan areas. The May 2018 estimates contain data for 134 nonmetropolitan areas, compared with 167 nonmetropolitan areas in the May 2017 estimates.

Elimination of metropolitan division data. OES no longer publishes data for the metropolitan divisions within the 11 large metropolitan areas that are further broken down into divisions. Data for these 11 areas are available at the Metropolitan Statistical Area (MSA) only.

This change does not affect the level of geography detail available to Emsi clients, though detailed regions that fall within the same consolidated OES region may look more similar to one another.

Historical Occupational Wage Estimates

Emsi is now publishing historical occupational wage estimates back to 2005. Current year occupational wage estimates are more stable and accurate as a result. This data is currently accessible only via Emsi’s LMI data API, but will appear in Emsi’s tools in the near future.

Self Employed Earnings

Emsi sources self-employed earnings from the American Community Survey public use microdata. Over the past year, Emsi conducted a complete methodology review of ACS usage and decided to make these changes:

Five-year sample window: Because ACS is a survey with limited sample size, it is standard practice to combine five years of data to improve coverage and stability. When Emsi began using ACS data, ACS three-year tabulations were the standard, and five-year tabulations were relatively new; consequently, Emsi created an ad hoc combination of years (1,3,5,9) to increase sample size. Not seeing any advantage to this ad hoc methodology over the standard, Emsi now follows the example of official ACS tabulations and uses a five-year window. Total self-employed employment is roughly identical to what it was previously, but how those jobs are distributed across industries and occupations will be minorly affected. On average, staffing patterns have 5% more occupations than before.

Improved earnings outlier removal: Another drawback to the limited sample size of surveys is that outliers can disproportionately affect the quality of estimates, especially in granular tabulations. In this release, Emsi improved the methodology for ACS earnings outlier removal to prevent earnings from being skewed low (because of older years of data) and to allow removal of low-earning outliers (only high outliers could be detected in Emsi’s previous methodology). The end result is that average self-employed industry and occupation earnings are on average 30% higher.

Improved occupation earnings percentiles estimation: Prior to this release, Emsi estimated percentiles after tabulating ACS microdata, assuming that it was valid to aggregate weighted percentiles. In 2017, Emsi stopped aggregating weighted earnings percentiles elsewhere in US data, but neglected to fix the initial creation of self-employed earnings percentiles. Consequently, these percentiles were artificially flat (skewed toward the average). The new methodology correctly builds percentiles from the original microdata, resulting in more reasonable percentile curves.

Earnings for Military-Only Occupation

See the changes to Self Employed Earnings; Emsi’s earnings for the military-only occupation come from ACS. On average, earnings are 5% higher and percentile curves are no longer skewed toward the average.

Local Absorption in the Input-Output Model

Emsi adjusted the methodology in the input-output model that regulates local absorption of goods produced in region.

Gross Regional Product Bug

We discovered a bug in our process that estimates gross regional product for years where the detailed regional estimates in BEA’s GDP by state lag behind national estimates from the BEA’s National Income and Product Accounts (NIPA). When regional data lags behind national data, we fill in the missing regional detail using previous-year regional data and current NIPA data. Previously, Emsi worked under the assumption that Gross Domestic Income (GDI) was an alternative way to calculate GDP and should be equal to it, so we used GDI data from NIPA to bring forward the regional estimates. However, GDI is not equivalent to GDP in BEA estimates, despite the definition of the term, so we were incorrectly using GDI when we should have been using GDP. With this release, Emsi is now correctly using GDP data to bring forward the regional GDP estimates. This bug caused the most recent two years of GDP to be approximately 2% higher than they should have been at all levels of detail. This also affected the US SAM Input-Output model’s induced multipliers.

Military Employment Bug

We discovered a minor bug in the process that estimates non-QCEW employment for federal government military. Military employment data was missing from some counties affected by the county redefinitions in data prior to 2009.

Government Hires and Separations Bug

We discovered a minor bug in the process that estimates hires and separations (sourced from QWI) for industry and occupation workforce demographics. Hires and separations were erroneously zero for government industries and related occupations in a few state-year combinations where QWI lacked sufficient data. We now (correctly) use ratios from less detailed data to fill in those holes. This data is only visible to API clients and users of the hires portion of the job-postings report in Analyst.

Incorporated Self-Employed and Unemployed Employees from ACS

We discovered an error in how we use the Incorporated Self-Employed (ISE) class of worker from American Community Survey (ACS) data. ISE represents individuals who are employees of their own corporations, and thus are included in QCEW (Emsi CoW 1). In some cases, Emsi was previously treating these incorporated self-employed incorporated individuals as CoW 3 instead of CoW 1. Emsi’s self-employed class of worker (CoW 3) now does not include these individuals. See below for a summary of the effects on final data.

We also began making use of data from ACS respondents that were unemployed at the time the survey was collected, but had worked as an employee within the last year. Similar to how annualized QCEW jobs are reported, we down-weight unemployed ACS records based on the number of weeks the person worked (e.g. – a person who works three months of the year is counted as a quarter of a job). These new data feed into:

Workforce demographics (detailed industry and occupation data): Emsi uses ACS data to supplement and help unsuppress Quarterly Workforce Indicators (QWI) data, which is in turn combined with final Emsi industry and occupation data to estimate workforce demographic breakouts (e.g. – “We know how many plumbers work in X region, how many of them are white males?”). Previously, we were estimating a small portion of the demographic breakouts, where estimates were not available from QWI, using the wrong subsection of the US economy; the breakouts for CoW 1 and 2 mistakenly did not make use of ISE and the breakouts for CoW 3 and 4 mistakenly did make use of ISE. However, because the primary data source for demographic breakouts is QWI not ACS, the effects on final data are minor.

Staffing Patterns for QCEW and Non-QCEW (Emsi CoW 1 and 2): ACS data is used to create staffing patterns for industries that are not included in OES staffing patterns. These ACS staffing patterns were mistakenly created from all employee records from ACS except ISE and the currently unemployed. Again, since ACS is not the primary dataset from which Emsi creates staffing patterns, the effects of the bug are limited in scope to only industries that are missing from OES: Agriculture (NAICS 111000, 112000, 113110, 113210, 115310, 114111, 114112, 114119, 114210) and Private Households (NAICS 814110). The differences in the ACS-based staffing patterns will be minor, since most employees were correctly included. Note that this also minorly affects Emsi’s final occupation data which is calculated from industry and staffing data.

MSA Delineation Changes

In September 2018, the Census Bureau released new delineation files for Micro- and Metropolitan Statistical Areas. The 138 changes are fairly extensive and affect most of the major metropolitan areas in the United States. While county definitions did not change, 56 counties are now included in MSAs that were not previously, 42 counties have been removed from MSAs, and 40 counties have moved to new or different MSAs. If you have previously viewed data at the MSA level and are noticing large changes between current and previous data for your MSA, this is likely the cause. For more detail, contact your customer service representative.

Occupational Employment Statistics

Emsi continues to do research and preparation for creating a time series out of OES data. The objectives are to provide more accurate historical occupation employment estimates and begin producing historical occupation earnings. The research is still ongoing on both fronts, but various changes have been implemented to lay the groundwork for these future improvements. First, we tightened controls on occupational employment estimates, so they are now clustered more tightly around disclosed OES data. Second, we improved the algorithms that estimate top-coded earnings (earnings that OES suppresses because they are above $100/hour). Third, we began applying our occupational earnings aggregation methodology to the earnings unsuppression process. On the whole, the accuracy of unsuppressed earnings estimates did not change, but percentile wage curves are more reasonable and much more statistically defensible.

Changes in QCEW in South Carolina

QCEW is the foundation of Emsi’s employment and earnings data. The BLS has reported that QCEW data in South Carolina “are showing unusual movements which may be a result of a change in reporting.” For more information, see: https://www.bls.gov/cew/2018-notice-regarding-south-carolina-employment-and-wages-data.htm

New Benchmark Data from the BEA

For the past five or more years, the Social Accounting Matrix (SAM) and Input-Output (I-O) data have had at their core the 2007 benchmark make and use tables from the BEA. This year, the BEA published 2012 benchmark files, and Emsi incorporated them alongside the previously existing 2007 benchmark files. Because this update accounts for five years of data updates, we expect to see substantial changes throughout Emsi’s SAM and I-O data and models.

None. There were no classification updates or major methodology changes.

New Emsi Occupation Classification (Emsi SOC 2017)

Emsi’s occupation classification (SOC) is being updated to match the reduced level of detail available in the 2017 release of OES. See https://www.bls.gov/oes/changes_2017.htm for a summary of the changes. Where the BLS aggregated detailed occupations to the broader “parent” occupation, Emsi copied the parent code’s data down to a new Emsi-specific detailed code ending with an “8”, so clients can still browse occupation data by detail level. Data will no longer be available for the old detailed SOC codes. For more details or if you have questions, please contact your customer service representative.

Historical Staffing Patterns

Emsi is now using historical staffing patterns based on historical Occupational Employment Statistics (BLS) data. All QCEW and Non-QCEW historical staffing and occupation data is affected. OES advises against using historical OES data as a time series for reasons here: https://www.bls.gov/oes/oes_ques.htm#qf1. Emsi took those factors into account and made a significant effort to smooth the volatility between historical OES releases. In the near future, we hope to extend the use of historical OES data beyond national staffing patterns to regional employment and earnings.

Military Staffing Pattern

Due to lack of good, detailed military occupation data, Emsi previously lumped all military employment (NonQCEW, industry 901200) into a single occupation (55-9999). In response to various requests from clients, we now use a national staffing pattern from the American Community Survey (Census) to break out military employment between military-specific occupations (55-9999) and occupations that are shared with civilians (eg. pilots, mechanics). In 2016, approximately 44% of military employment was in military-specific occupations and 56% was in occupations shared with civilians. This change more closely reflects the design of the official SOC classification and also clears up an equivocation on the definition of “military” that was causing some confusion about earnings (Emsi was using earnings for military-specific occupations for all of military employment).

Non-QCEW (Emsi Class of Worker 2) Employment

We discovered a long-standing bug in one of our Non-QCEW Employees processes that was underestimating employment in a handful of industries:

• NAICS 524 – Insurance Carriers and Related Activities
• NAICS 531 – Real Estate
• NAICS 54 – Professional, Scientific, and Technical Services

Users should notice a 2x or 3x (depending on the industry) increase in Non-QCEW employment and total earnings for these industries and their corresponding occupations (via the staffing pattern). Occupations in SOC 41 will be the most affected by the increase in employment.

Local Absorption in the I-O model

Emsi uses data from the Census Bureau’s Commodity Flow Survey (CFS) supplemented with proprietary data to create industry-specific transportation impedance estimates. We recently discovered that CFS is too general to create accurate impedance estimates for some industries, so we opted for using only the proprietary data in those cases. Consequently, users should expect to see changes in local absorption (demand satisfied in region) for the following industry sectors: 211, 213, 425, 44-45 (except 454), 4542, 45439, 48-49 (except 493), 51 (except 511), 551111, 551112. Retail industries, particularly, will have much higher local absorption.

Occupation by Residence for Government Jobs

Emsi uses industry commuting data from Census’s Longitudinal Origin-Destination Employment Statistics (LODES) dataset to create occupation data by place of residence. Previously we made use of the job type JT02 (private jobs) and assumed no commuting for government jobs, so that the place of work and place of residence were the same. Emsi now uses JT00 (all jobs) as well as JT02 to create commuting patterns for government jobs. This also improves the accuracy of the regional Social Accounting Matrix.

Metropolitan Statistical Areas (MSA) Definitions

This release uses the latest MSA delineations from August 2017 as provided by the Census Bureau (previous releases used the July 2015 delineations). The only change was that Twin Falls, ID was changed from a micropolitan statistical area to a metropolitan statistical area.

Input-Output Transaction (Z) Matrix

Emsi now defines national inter-industry transactions as significant only if they sum to more than $500K. This matches the BEA’s functional definition, since they round their transaction matrix to the nearest $1M. Emsi previously had no minimum value on national inter-industry transactions, allowing scores of insignificant transactions to clutter and add noise to the matrix. This change will have minimal visible affects, since the primary goal is to remove insignificant transactions and reduce noise in industry supply chains.

North American Industry Classification System (NAICS) 2017

Emsi switched from using NAICS 2012 to NAICS 2017. Summary of the changes:

• 4-digit: 3 removed, 3 new, 0 combined, 2 recoded, 1 split
• 5-digit: 10 removed, 7 new, 7 combined, 1 recoded, 2 split
• 6-digit: 28 removed, 20 new, 13 combined, 11 renamed, 4 split

Occupational Information Network (O*NET) Version 22.0

With this release we updated from O*NET version 21.3 to version 22.0, to include updated Knowledge, Skill, and Abilities for 100 occupations. This will slightly impact the compatibility indices of occupation pairs.

Non-QCEW (Class of Worker 2) Earnings

We fixed a bug in the code that was causing earnings to be skewed low. In addition to normal data changes, earnings in this class of worker are approximately 1.5% higher than previous releases.

ZIP Code Population Demographics

We improved the usage of the probability matrices used to disaggregate county-level population demographics to ZIP codes, resulting in more accurate estimates for approximately 25% of the nation. Rural areas are the most improved.

ZIP Code Industry Employment Estimates

We improved our estimation of ZIP code industry employment with a more nuanced use of ZBP (ZIP Code Business Patterns, Census). Previously, if a county’s industry was not found in ZBP, we would default to using an USPS business delivery statistics to distribute that industry’s employment to all ZIP codes in the county. With the new methodology, if a county’s industry isn’t found in ZBP, we search for similar industries in ZBP before defaulting to USPS data. The use of USPS dropped from 20% to 0.5%, significantly increasing the accuracy of ZIP code employment estimates.

This change affects all ZIP code employment data (industry and occupation) and does not target any specific counties or industries. However, the most significant changes will be in rural areas and in regions where an industry’s employment is small. Changes will appear as a more focused distribution of data within the ZIP codes in each county.

Local Absorption in Emsi’s Social Accounting Matrix (I-O)

We slightly changed the constraints on the SAM’s bi-proportional algorithm to further maximize local absorption. This fixes some anomalies uncovered in the 2017.3 release where local absorption was too low.

Unemployment Data

Unemployment data has been brought up to date and the methodology adjusted to reflect current conditions more closely. The new methodology averages the past 12 months of unemployment to form a figure free from seasonal volatility. This change has been introduced with the 2017.4 data release; because of this, clients should expect to see differences in unemployment between the 2017.4 data release and prior releases.

Industry and Occupation (2-digit) Unemployment

We changed and simplified our methodology for estimating unemployment by 2-digit industry and occupation to more accurately reflect the data from the US Department of Labor’s Characteristics of the Insured Unemployed and rely less heavily on survey-based unemployment data from the Current Population Survey. This change allows us to publish monthly unemployment estimates, which may be available soon; in the meantime, we will publish only the latest month of data. Note that the new monthly data may be less stable when compared to our previous methodology, which was annualized.

Openings

We changed how we calculate both replacements and growth. For details, see https://kb.economicmodeling.com/openings-changes-set-for-2017

Balance-of-State (BOS) ZIP codes

QCEW uses a BOS county code to track employment where no county was reported. To make Emsi’s ZIP code and county data more comparable, Emsi is introducing BOS ZIP codes that match the BOS county codes. Where “SS” is the state FIPS code, BOS county codes are formatted as “SS999” and BOS ZIP codes are formatted as “001SS.” As with BOS county codes, BOS ZIP codes will represent all data for a state that was not able to be tagged with a valid ZIP code. They will appear in ZIP industry, occupation, and demographics data.

Estimates Derived from Quarterly Workforce Indicators (QWI)

We fixed a bug in our methodology for estimating industry and occupation employment by demographics that caused the wrong year of national-average QWI data to be used as an input when an entire state was missing from QWI. Wyoming employment breakouts were affected for Emsi’s 2016.3-2017.1 releases.

Occupation by Place of Residence Data

Emsi’s 2017.1 release was the first to include occupation by place of residence data. One of the known limitations with that initial release was the absence of data in “999” or balance-of-state counties. This only affected state and national sums, and given that balance-of-state counties account for only a fraction of each state’s total employment and that most users will not be using place of residence data at those geography levels, the missing data should not have caused any major problems. Regardless, in Emsi’s 2017.2 release, the data for 999 counties are included and the state and national totals by place of residence should better reflect reality.

Occupational Earnings Percentiles

State and national occupational earnings percentiles for employee classes of worker (1 and 2) are no longer based on job-weighted aggregations of county estimates. Instead, earnings percentiles at the state and national levels are preserved from OES. The result is more accurate earnings percentiles at the state and national levels. Average hourly earnings are unaffected.

Social Accounting Matrix (SAM) and Input-Output (I-O) Years

The data is now for 2015-2016.

Occupation by Place of Residence Data

We now have occupational employment data by place of residence derived from Emsi industry and staffing data and Census LODES data.

ZipCode Industry and Occupation Data

We improved our method of breaking out Emsi county-level data to the ZipCode level, making better use of USPS Delivery Statistics and Census’ ZipCode Business Patterns (ZBP) and adding and making use of the following datasets:

• Four new tabulations of ACS five-year ZCTA employment estimates
• NCES Common Core of Data and IPEDS employment
• Railroad Retirement Board zip employment
• IPEDS Employees by Assigned Position

Earnings for Self-Employed and Extended Proprietors

We improved our methodology for estimating Self-Employed and Extended Proprietors earnings, improving accuracy in both industry and occupation data. While some differences may be noticeable, most will not exceed the normal variability between releases.

Employment for Extended Proprietors

We improved our methodology for producing employment estimates for Extended Proprietors, slightly lowering employment counts across the nation. In certain circumstances, our previous methodology double-counted a small number of Self-Employed workers as Extended Proprietors.

State Projections

We reduced the level of detail we accept from state-published industry projections due to their varying quality and currency, effectively weighting Emsi and BLS industry projections more heavily.

Current Employment Statistics

Due to recent improvements in QCEW and Emsi release schedules, we no longer use CES to inform our current year industry employment estimates.