Resolving coreference

Link pronouns to the noun phrases they may refer to

entity enrichment

coreference resolution

workforce research

Learn how a nearest-noun baseline handles pronouns in a constructed Riverton passage and where it fails.

Pronouns make the short applicant note harder than it first appears: she is clear enough to a reader, while he could point to more than one person.

Coreference resolution links different expressions that refer to the same thing. A pronoun such as they may refer back to the workers. This lesson uses a small baseline rule so its mistakes remain visible.

Note

The Riverton passage and reference setting below are fictional.

What you will learn

By the end of this lesson, you will be able to:

define coreference in plain language;
inspect noun phrases and pronouns from a spaCy parse;
apply a nearest-agreeing-noun baseline;
compare the baseline with this page’s expected answers; and
explain why document-level context matters.

Use a constructed passage

The lesson first checks the Riverton CSV, then uses a constructed passage because the stored sentences are too thin for coreference teaching.

library(readr)
library(dplyr)
library(tibble)
library(purrr)
library(stringr)
library(spacyr)
source("R/use-spacy.R")

pipeline <- use_project_spacy()
pipeline_version <- spacy_pipeline_version()

sentences <- read_csv(
  "data/workforce/workforce_sentences.csv",
  na = character(),
  col_types = cols(
    sentence_id = col_character(),
    document_id = col_character(),
    source_line = col_character(),
    text = col_character(),
    reference_label = col_character(),
    uncertainty = col_character(),
    annotator_id = col_character(),
    rationale = col_character(),
    codebook_version = col_character(),
    codebook_hash = col_character(),
    derived = col_character(),
    transformation = col_character()
  )
)

riverton_parsed <- spacy_parse(
  setNames(sentences$text, sentences$sentence_id),
  pos = TRUE,
  lemma = TRUE
) |>
  as_tibble()

riverton_pronouns <- riverton_parsed |>
  filter(pos == "PRON")

teaching_passage <- paste(
  "The coordinator posted the training notice on Monday.",
  "The applicant read it before the bus arrived.",
  "The supervisor called the applicant after she finished the form.",
  "The supervisor told the coordinator that he could open another class.",
  "The workers thanked the supervisor because they needed evening seats.",
  "The centre moved the flyer near the entrance, and it stayed there all week.",
  sep = " "
)

parsed_raw <- spacy_parse(
  c(teaching_passage = teaching_passage),
  pos = TRUE,
  tag = TRUE,
  lemma = TRUE,
  entity = TRUE,
  dependency = TRUE,
  nounphrase = TRUE,
  additional_attributes = c("morph")
)

parsed <- parsed_raw |>
  as_tibble() |>
  mutate(
    global_token = row_number(),
    morph = map_chr(morph, as.character)
  )

phrases <- nounphrase_extract(parsed_raw)

knitr::kable(
  riverton_pronouns |>
    select(doc_id, token, pos),
  col.names = c("Sentence ID", "Token", "spaCy part of speech"),
  caption = "Pronouns found in the 28 Riverton sentences",
  row.names = FALSE
)

Pronouns found in the 28 Riverton sentences
Sentence ID	Token	spaCy part of speech
s016	This	PRON

knitr::kable(
  tibble(
    name = pipeline_version$name,
    version = pipeline_version$version,
    language = pipeline_version$lang,
    license = pipeline_version$license,
    spacy = pipeline_version$spacy
  ),
  col.names = c("Pipeline", "Version", "Language", "License", "spaCy"),
  caption = "spaCy pipeline used for the parse",
  row.names = FALSE
)

spaCy pipeline used for the parse
Pipeline	Version	Language	License	spaCy
core_web_sm	3.8.0	en	MIT	3.8.7

knitr::kable(
  phrases,
  col.names = c("Document", "Sentence", "Noun phrase"),
  caption = "Noun phrases extracted from the constructed passage",
  row.names = FALSE
)

Noun phrases extracted from the constructed passage
Document	Sentence	Noun phrase
teaching_passage	1	The_coordinator
teaching_passage	1	the_training_notice
teaching_passage	1	Monday
teaching_passage	2	The_applicant
teaching_passage	2	it
teaching_passage	2	the_bus
teaching_passage	3	The_supervisor
teaching_passage	3	the_applicant
teaching_passage	3	she
teaching_passage	3	the_form
teaching_passage	4	The_supervisor
teaching_passage	4	the_coordinator
teaching_passage	4	he
teaching_passage	4	another_class
teaching_passage	5	The_workers
teaching_passage	5	the_supervisor
teaching_passage	5	they
teaching_passage	5	evening_seats
teaching_passage	6	The_centre
teaching_passage	6	the_flyer
teaching_passage	6	the_entrance
teaching_passage	6	it

spaCy marks one pronoun in the 28 Riverton sentences. One example is not enough to teach pronoun linking, so the constructed passage gives the rule more chances to fail.

The installed en_core_web_sm pipeline has named-entity, tagger, parser, and other components, but no coreference component. The rule below is a baseline, not a full coreference system.

Apply the baseline rule

The rule is explicit: link each personal pronoun to the nearest preceding noun phrase that has the same number feature. Here, number means singular or plural as reported by spaCy morphology on the noun-phrase head. The thing a pronoun points back to is called its antecedent.

number_from_morph <- function(morph_value) {
  case_when(
    str_detect(morph_value, "Number=Plur") ~ "Plur",
    str_detect(morph_value, "Number=Sing") ~ "Sing",
    TRUE ~ NA_character_
  )
}

phrase_tokens <- parsed |>
  mutate(
    np_inside = nounphrase != "",
    np_id = if_else(np_inside, cumsum(str_detect(nounphrase, "beg")), NA_integer_)
  ) |>
  filter(np_inside)

noun_phrases <- phrase_tokens |>
  summarise(
    phrase = paste(token, collapse = " "),
    root_token = token[str_detect(nounphrase, "root")][1],
    root_pos = pos[str_detect(nounphrase, "root")][1],
    root_morph = morph[str_detect(nounphrase, "root")][1],
    start_token = min(global_token),
    end_token = max(global_token),
    .by = np_id
  ) |>
  mutate(number = number_from_morph(root_morph))

antecedents <- noun_phrases |>
  filter(root_pos %in% c("NOUN", "PROPN"), !is.na(number)) |>
  rename(antecedent = phrase, antecedent_number = number) |>
  mutate(join_key = 1L)

pronouns <- parsed |>
  filter(pos == "PRON", str_to_lower(token) %in% c("it", "she", "he", "they")) |>
  transmute(
    pronoun_id = paste0("p", row_number()),
    pronoun = token,
    sentence_id,
    pronoun_token = global_token,
    pronoun_number = number_from_morph(morph),
    join_key = 1L
  )

rule_links <- pronouns |>
  left_join(antecedents, by = join_by(join_key), relationship = "many-to-many") |>
  filter(end_token < pronoun_token, antecedent_number == pronoun_number) |>
  arrange(pronoun_id, desc(end_token)) |>
  summarise(
    rule_antecedent = first(antecedent),
    .by = c(pronoun_id, pronoun, sentence_id, pronoun_number)
  )

knitr::kable(
  rule_links,
  col.names = c("Pronoun ID", "Pronoun", "Sentence", "Number", "Rule antecedent"),
  caption = "Nearest preceding noun phrase with matching number",
  row.names = FALSE
)

Nearest preceding noun phrase with matching number
Pronoun ID	Pronoun	Sentence	Number	Rule antecedent
p1	it	2	Sing	The applicant
p2	she	3	Sing	the applicant
p3	he	4	Sing	the coordinator
p4	they	5	Plur	The workers
p5	it	6	Sing	the entrance

The rule chooses The applicant for the first it because that is the nearest singular noun phrase. A reader can see that it means the training notice.

Score the rule

The expected answers treat he as genuinely ambiguous because the supervisor or the coordinator could open another class. That row is counted separately.

marked_coreference <- tibble(
  pronoun_id = c("p1", "p2", "p3", "p4", "p5"),
  expected_antecedent = c(
    "the training notice",
    "the applicant",
    "ambiguous",
    "The workers",
    "the flyer"
  ),
  scoreable = c(TRUE, TRUE, FALSE, TRUE, TRUE)
)

coreference_score <- rule_links |>
  left_join(marked_coreference, by = join_by(pronoun_id)) |>
  mutate(
    result = case_when(
      !scoreable ~ "ambiguous reference",
      str_to_lower(rule_antecedent) == str_to_lower(expected_antecedent) ~ "correct",
      TRUE ~ "wrong"
    )
  )

score_counts <- tibble(result = c("correct", "wrong", "ambiguous reference")) |>
  left_join(
    coreference_score |>
      count(result, name = "pronouns"),
    by = join_by(result)
  ) |>
  mutate(pronouns = coalesce(pronouns, 0L))

spacy_finalize()

knitr::kable(
  coreference_score |>
    select(pronoun_id, pronoun, rule_antecedent, expected_antecedent, result),
  col.names = c("Pronoun ID", "Pronoun", "Rule antecedent", "Expected answer", "Result"),
  caption = "Coreference baseline compared with expected answers",
  row.names = FALSE
)

Coreference baseline compared with expected answers
Pronoun ID	Pronoun	Rule antecedent	Expected answer	Result
p1	it	The applicant	the training notice	wrong
p2	she	the applicant	the applicant	correct
p3	he	the coordinator	ambiguous	ambiguous reference
p4	they	The workers	The workers	correct
p5	it	the entrance	the flyer	wrong

knitr::kable(
  score_counts,
  col.names = c("Result", "Pronouns"),
  caption = "Correct, wrong, and ambiguous pronoun links",
  row.names = FALSE
)

Correct, wrong, and ambiguous pronoun links
Result	Pronouns
correct	2
wrong	2
ambiguous reference	1

Because the baseline and expected answers share an author, the score is only a check on this page. The two correct answers are easy cases: The workers is the only plural phrase before they, and the rule links she without using gender. Excluding the ambiguous he row changes the scored denominator from five pronouns to four. Real coreference systems need document context, syntax, and meaning.

Modern coreference resolution

The deterministic nearest-agreeing-noun syntax rule shown here is an early baseline intended only to expose the mechanism of coreference. Most complex coreference tasks are now approached using neural models (such as AllenNLP or large language models) rather than fixed syntax rules.

When evaluating modern systems, keep these limits in mind: - Cluster metrics: Coreference models are evaluated using specialized clustering metrics (like MUC, B³, and CEAF) rather than simple pairwise accuracy, because errors compound when linking long chains of mentions across a document. - Ambiguity: Language is frequently ambiguous (like the he example). Deterministic models often force a link where a human would recognize true ambiguity. - Stereotype and coverage asymmetry: Coreference models often inherit training biases, successfully resolving pronouns that match occupational gender stereotypes (e.g., linking he to doctor and she to nurse) while failing on anti-stereotypical or gender-neutral (they) examples.

What to remember

Coreference links expressions that refer to the same thing.
en_core_web_sm does not include a coreference resolver.
A nearest-agreeing-noun rule is a baseline for inspection.
Ambiguous pronouns should be marked as ambiguous, not forced into certainty.

After checking the five pronouns, the coordinator has two easy matches, two mistakes, and one ambiguity. That is enough to explain the baseline, not enough to use it as a resolver.

Use a constructed passage

Apply the baseline rule

Score the rule

Modern coreference resolution

What to remember

Sources