Parsing prices from text

Turn pay mentions into amounts with units

entity enrichment

price parsing

workforce research

Learn how simple R rules pull wages and stipends from text while keeping ranges, missing units, and no-number cases visible.

After labelling sentences from the job board and training flyer, the team faces a new question from the coordinator: which notices say anything concrete about pay?

A phrase such as $18.50 per hour looks easy. A phrase such as competitive pay looks useful to a reader but gives a computer no amount to compare. The team needs a rule that can extract a value and also say when the text did not contain a usable price.

Note

The Riverton job board and flyer in this lesson are fictional teaching materials.

What you will learn

By the end of this lesson, you will be able to:

explain why a number alone is not a price;
write stringr patterns for money-like text;
separate amount, currency, and pay period;
keep ranges and missing values visible; and
compare a small rule set with this page’s expected answers.

Build examples with expected answers

The setup reads the Riverton CSV with readr, shapes tables with dplyr and tibble, and asks stringr to match the money-like patterns. The examples are author-created pay strings in the Riverton style. A price here means an amount, a currency, and a unit such as hour, year, or stipend.

library(readr)
library(dplyr)
library(tibble)
library(stringr)

sentences <- read_csv(
  "data/workforce/workforce_sentences.csv",
  na = character(),
  col_types = cols(
    sentence_id = col_character(),
    document_id = col_character(),
    source_line = col_character(),
    text = col_character(),
    reference_label = col_character(),
    uncertainty = col_character(),
    annotator_id = col_character(),
    rationale = col_character(),
    codebook_version = col_character(),
    codebook_hash = col_character(),
    derived = col_character(),
    transformation = col_character()
  )
)

pay_examples <- tibble(
  example_id = sprintf("pay-%02d", 1:13),
  text = c(
    "$18.50 per hour",
    "18.50/hr",
    "$38,000 a year",
    "$17-$19 an hour",
    "competitive pay",
    "$1,250 stipend",
    "$20 hourly",
    "Pay starts at $16 per hour",
    "$42,500 annually",
    "Stipend of 750 dollars",
    "Up to $21/hour",
    "18 dollars an hour",
    "$38000 a year"
  ),
  expected_currency = c(
    "USD", NA, "USD", "USD", NA, "USD",
    "USD", "USD", "USD", "USD", "USD", "USD",
    "USD"
  ),
  expected_unit = c(
    "hour", "hour", "year", "hour", NA, "stipend",
    "hour", "hour", "year", "stipend", "hour", "hour",
    "year"
  ),
  expected_lower = c(
    18.50, 18.50, 38000, 17, NA, 1250,
    20, 16, 42500, 750, 21, 18,
    38000
  ),
  expected_upper = c(
    NA, NA, NA, 19, NA, NA,
    NA, NA, NA, NA, NA, NA,
    NA
  )
)

knitr::kable(
  pay_examples,
  col.names = c(
    "Example ID", "Text", "Expected currency", "Expected unit",
    "Expected lower amount", "Expected upper amount"
  ),
  caption = "Author-created price examples with expected answers",
  row.names = FALSE
)

Author-created price examples with expected answers
Example ID	Text	Expected currency	Expected unit	Expected lower amount	Expected upper amount
pay-01	$18.50 per hour	USD	hour	18.5	NA
pay-02	18.50/hr	NA	hour	18.5	NA
pay-03	$38,000 a year	USD	year	38000.0	NA
pay-04	$17-$19 an hour	USD	hour	17.0	19
pay-05	competitive pay	NA	NA	NA	NA
pay-06	$1,250 stipend	USD	stipend	1250.0	NA
pay-07	$20 hourly	USD	hour	20.0	NA
pay-08	Pay starts at $16 per hour	USD	hour	16.0	NA
pay-09	$42,500 annually	USD	year	42500.0	NA
pay-10	Stipend of 750 dollars	USD	stipend	750.0	NA
pay-11	Up to $21/hour	USD	hour	21.0	NA
pay-12	18 dollars an hour	USD	hour	18.0	NA
pay-13	$38000 a year	USD	year	38000.0	NA

The table contains 13 examples. One writes a five-digit amount without a comma. That case matters because a partial match would look like a real answer.

Pull out the visible pieces

The pattern below uses the ICU regular expression engine through stringr. A regular expression is a compact search pattern. This one looks for an optional dollar sign, a number that may contain commas or cents, and an optional second number after a hyphen or the word to.

parse_prices <- function(data) {
  number_pattern <- "(?:\\d{1,3}(?:,\\d{3})+|\\d+)(?:\\.\\d+)?"
  amount_pattern <- paste0(
    "(?i)(?:\\$\\s*)?", number_pattern,
    "(?:\\s*(?:-|to)\\s*(?:\\$\\s*)?", number_pattern, ")?"
  )

  data |>
    mutate(
      amount_text = str_extract(text, amount_pattern),
      currency = case_when(
        str_detect(text, fixed("$")) ~ "USD",
        str_detect(str_to_lower(text), "\\bdollars?\\b") ~ "USD",
        TRUE ~ NA_character_
      ),
      unit = case_when(
        str_detect(str_to_lower(text), "per hour|/hr|/hour|an hour|hourly") ~ "hour",
        str_detect(str_to_lower(text), "a year|annually|annual|year") ~ "year",
        str_detect(str_to_lower(text), "stipend") ~ "stipend",
        TRUE ~ NA_character_
      ),
      amount_clean = str_remove_all(amount_text, "[$,]"),
      amount_lower = as.numeric(str_extract(amount_clean, "\\d+(?:\\.\\d+)?")),
      amount_upper = as.numeric(
        str_match(amount_clean, "(?:-|to)\\s*(\\d+(?:\\.\\d+)?)")[, 2]
      ),
      parser_result = case_when(
        is.na(amount_lower) ~ "not a price",
        is.na(unit) ~ "number without unit",
        !is.na(amount_upper) ~ "range",
        TRUE ~ "single price"
      )
    )
}

price_results <- parse_prices(pay_examples)

knitr::kable(
  price_results |>
    select(example_id, text, amount_text, currency, unit, amount_lower, amount_upper, parser_result),
  col.names = c(
    "Example ID", "Text", "Matched amount", "Currency", "Unit",
    "Lower amount", "Upper amount", "Parser result"
  ),
  caption = "Parsed amount, currency, and unit",
  row.names = FALSE
)

Parsed amount, currency, and unit
Example ID	Text	Matched amount	Currency	Unit	Lower amount	Upper amount	Parser result
pay-01	$18.50 per hour	$18.50	USD	hour	18.5	NA	single price
pay-02	18.50/hr	18.50	NA	hour	18.5	NA	single price
pay-03	$38,000 a year	$38,000	USD	year	38000.0	NA	single price
pay-04	$17-$19 an hour	$17-$19	USD	hour	17.0	19	range
pay-05	competitive pay	NA	NA	NA	NA	NA	not a price
pay-06	$1,250 stipend	$1,250	USD	stipend	1250.0	NA	single price
pay-07	$20 hourly	$20	USD	hour	20.0	NA	single price
pay-08	Pay starts at $16 per hour	$16	USD	hour	16.0	NA	single price
pay-09	$42,500 annually	$42,500	USD	year	42500.0	NA	single price
pay-10	Stipend of 750 dollars	750	USD	stipend	750.0	NA	single price
pay-11	Up to $21/hour	$21	USD	hour	21.0	NA	single price
pay-12	18 dollars an hour	18	USD	hour	18.0	NA	single price
pay-13	$38000 a year	$38000	USD	year	38000.0	NA	single price

The number and the unit come from different parts of the phrase. 18.50/hr contains an amount and a unit, but the rule marks the currency as missing. The rule treats every dollar sign as US dollars, which would be wrong on a board that mixed US, Canadian, and Australian postings. Units outside this small set, such as monthly pay, would need another branch.

Count the special cases

A parser is easier to trust when it reports what it could not settle. The code below counts the rows that ask for special handling.

case_counts <- tibble(
  case = c(
    "thousands separator",
    "range with two numbers",
    "missing currency",
    "no number",
    "plain four-plus digit number"
  ),
  examples = c(
    sum(str_detect(price_results$amount_text, fixed(",")), na.rm = TRUE),
    sum(!is.na(price_results$amount_upper)),
    sum(is.na(price_results$currency)),
    sum(is.na(price_results$amount_lower)),
    sum(
      str_detect(str_remove_all(price_results$amount_text, "[$,]"), "\\d{4,}") &
        !str_detect(price_results$amount_text, fixed(",")),
      na.rm = TRUE
    )
  )
)

knitr::kable(
  case_counts,
  col.names = c("Case", "Examples"),
  caption = "Cases the parser keeps visible",
  row.names = FALSE
)

Cases the parser keeps visible
Case	Examples
thousands separator	3
range with two numbers	1
missing currency	2
no number	1
plain four-plus digit number	1

Three examples contain thousands separators. One is a range with two amounts. Two lack a dollar sign or dollar word. One has no number. The categories overlap, so this table is a checklist rather than a count of distinct rows.

Score the parser against the marking

Because the parser and expected answers come from the same author, a high score here does not estimate accuracy. Read it only as a check that the code behaves as this page says it should.

values_match <- function(observed, expected) {
  (is.na(observed) & is.na(expected)) |
    (!is.na(observed) & !is.na(expected) & observed == expected)
}

price_score <- price_results |>
  summarise(
    examples = n(),
    lower_amount_matches = sum(values_match(amount_lower, expected_lower)),
    upper_amount_matches = sum(values_match(amount_upper, expected_upper)),
    unit_matches = sum(values_match(unit, expected_unit)),
    currency_matches = sum(values_match(currency, expected_currency))
  )

knitr::kable(
  price_score,
  col.names = c(
    "Examples", "Lower amount matches", "Upper amount matches",
    "Unit matches", "Currency matches"
  ),
  caption = "Agreement with this page's expected answers",
  row.names = FALSE
)

Agreement with this page’s expected answers
Examples	Lower amount matches	Upper amount matches	Unit matches	Currency matches
13	13	13	13	13

The parser matches the expected lower amount, upper amount, unit, and currency on all 13 examples. The comma-free case would have been dangerous without the wider number pattern: $38000 must not be cut to $380.

Check the Riverton stipend line

The real Riverton teaching file contains a stipend phrase, but not a stipend amount. The rule should return a miss rather than inventing one.

riverton_pay_line <- sentences |>
  filter(sentence_id == "s025") |>
  transmute(
    example_id = sentence_id,
    text,
    expected_currency = NA_character_,
    expected_unit = "stipend",
    expected_lower = NA_real_,
    expected_upper = NA_real_
  )

riverton_price_result <- parse_prices(riverton_pay_line)

knitr::kable(
  riverton_price_result |>
    select(example_id, text, amount_lower, currency, unit, parser_result),
  col.names = c("Sentence ID", "Text", "Amount", "Currency", "Unit", "Parser result"),
  caption = "The Riverton stipend line has a unit but no amount",
  row.names = FALSE
)

The Riverton stipend line has a unit but no amount
Sentence ID	Text	Amount	Currency	Unit	Parser result
s025	Paid training stipend	NA	NA	stipend	not a price

The line tells a reader that some stipend exists. It does not say how much. A number without a unit is not a price, and a unit without a number is not an amount the coordinator can compare.

The limits of custom rules

A custom regular expression is a quick way to find highly constrained formats, but it breaks quickly on varied real-world data.

failure_examples <- tibble(
  text = c("thirty-eight thousand a year", "1.250,50 / hr", "$40/hr (CAD)")
)

failure_results <- parse_prices(failure_examples)

failure_results |>
  select(text, amount_lower, currency) |>
  knitr::kable(caption = "Representative failures of the custom regex parser")

Representative failures of the custom regex parser
text	amount_lower	currency
thirty-eight thousand a year	NA	NA
1.250,50 / hr	1.25	NA
$40/hr (CAD)	40.00	USD

Value normalization: The pattern misses "thirty-eight thousand" entirely because it only looks for digits.
Locale formats: It reads the European format 1.250,50 incorrectly: it matches 1.250 without warning and yields an amount of 1.25 because it expects commas as thousands separators.
Ambiguous currencies: It assumes $ means USD, which is wrong for CAD.

In standard NLP workflows, relying entirely on custom regex for entity parsing is an anti-pattern. Systems use Named Entity Recognition (like MONEY entities in spaCy) to detect relevant spans, and separate value parsers (like Duckling) to normalize amounts, currencies, and locales into structured data.

What to remember

A usable price needs an amount, a currency, and a unit.
Ranges should keep both numbers.
A parser must not return a partial number when the text contains a longer one.
These counts describe 13 teaching examples and one Riverton line.

The stipend line can be flagged as pay-related, but it cannot support a wage comparison because it gives no amount.

Build examples with expected answers

Pull out the visible pieces

Count the special cases

Score the parser against the marking

Check the Riverton stipend line

The limits of custom rules

What to remember

Sources