Back to Blog

I just released onet2r 0.4.1, an R package for working with O*NET data. This is a short tour of what it does and why I built it.

O*NET is the US Department of Labor database of what occupations involve. Tasks, skills, abilities, knowledge, and work context, scored for around 900 occupations. It sits underneath a large share of the AI-exposure, automation, and skills research I read, including a fair amount of my own. It is also easy to use badly, and the most common way to use it badly is to treat it as a time series.

O*NET is not a time series

The O*NET Web Services API serves the current release. If you want to know how an occupation looks today, the API is excellent. If you want to know how it changed between two releases, the API will not help you, because O*NET was never built as a longitudinal panel, a dataset that follows the same occupations across time. Historical work has to be reconstructed from the archived database files, and that reconstruction is where most mistakes happen.

I kept running into the same three problems, so I built a package that handles each one in the open rather than hiding it.

Three things that go wrong

The first problem is that the API only knows about the present. Longitudinal work needs the archived releases, which are published as downloadable database files going back years. onet2r reads those archives into one normalized panel.

The second problem is that occupation codes move. O*NET labels jobs with O*NET-SOC codes, a more detailed version of the federal Standard Occupational Classification, and that system is revised every several years. The editions, called vintages, do not line up one to one. Occupations split, merge, appear, and disappear. A comparison that holds the code fixed across a taxonomy change is comparing two different things. onet2r bridges the vintages and marks the rows where the bridge is uncertain.

The third problem is the quiet one. A value that changed between two releases is not always a real content change. It can be a recode, a transition row, or a suppressed estimate. And a value that stayed the same is not always stable. It can be a carryforward from an occupation that was not resurveyed, or a fresh survey that happened to land on the same rating. onet2r reads the source dates and domain sources and labels each row, so you know what kind of change you are actually looking at.

What the package does

The core workflow reads archived releases, reconciles them, and tells you which differences you can trust.

library(onet2r)

# The package ships with a small example archive for two releases
archive_base <- system.file("extdata", "onet-mini", package = "onet2r")
archives <- c(
  `30.2` = file.path(archive_base, "db_30_2_text"),
  `30.3` = file.path(archive_base, "db_30_3_text")
)

# Read both releases into one panel
panel <- onet_panel(
  "Abilities",
  versions = c("30.2", "30.3"),
  scale = "IM",                 # IM is the Importance scale
  archives = archives
)

# 30.2 and 30.3 share the 2019 O*NET-SOC taxonomy, so this is the
# same-vintage case. Reconcile the releases and label every row.
changes <- onet_panel_reconcile(panel, onet_crosswalk_bridge("2019", "2019"))

After onet_panel_reconcile(), every row carries a change_type and a safely_comparable flag. A difference can come back as a real update, a stale carryforward, a resampled-but-stable value, or a recode flag. Rows that cross a taxonomy seam (a point where the occupation codes changed) or sit on transition data are marked so they do not get treated as clean within-occupation change. You read the verdict before you trust the number.

The example above compares two releases that share a taxonomy. When releases cross an O*NET-SOC boundary, you pass a real crosswalk bridge instead of the same-vintage one, and the docs walk through that harder case.

From there the package handles the parts that turn a descriptor table into a labor-market answer.

  • Bring your own task or occupation measure, validate the keys against a specific archived release, and roll task scores up to occupations.
  • Weight results by BLS OEWS or Census PUMS employment. OEWS is the federal wage and employment series; PUMS is Census survey microdata. The weight panel records the source, year, and reference taxonomy.
  • Read coverage and provenance through onet_coverage() and onet_provenance() so a result carries its own audit trail.
  • Stress-test a measure across alternative taxonomy bridges, employment-weight panels (OEWS or PUMS), and task-handling rules with onet_measure_sensitivity(), then decompose an aggregate change into within, between, interaction, and unclassifiable pieces.

None of this decides which exposure or skill measure is correct. That is the researcher's call. The package handles the plumbing around the measure and keeps a record of what was done.

Why this matters for workforce research

A lot of the work on this site depends on occupation-level data holding still long enough to measure change. When an AI-exposure score or a skills index is built on O*NET, the comparison across years is only as honest as the bookkeeping underneath it. A trend that is really a taxonomy recode is worse than no trend, because it looks like a finding. onet2r is my attempt to make that bookkeeping the default rather than an afterthought.

Clean bookkeeping is not the same as a clean experiment. Survey methods and who responds shift between releases too, so even a row marked as a real update is a starting point for judgment, not a settled result. If you commission or read O*NET-based analysis rather than run it, the useful questions are which release it used, how it handled taxonomy changes across years, and whether the result survives alternative bridges and weights.

Try it

onet2r is free and open source. It lives on GitHub and is not on CRAN yet, so install the development version.

# install.packages("pak")
pak::pak("farach/onet2r")

The documentation site has runnable articles for each part of the workflow, including a longer piece on why longitudinal O*NET analysis is hard and a guide to the ways a cross-release comparison can fool you. The docs live at farach.github.io/onet2r and the code is at github.com/farach/onet2r.

If you work with O*NET, OEWS, task measures, or workforce data, in R or otherwise, I would like to hear which part of the workflow still gets in your way.