Skip to main content

Moving a notice from English to French

Run one fixed translation direction and inspect what survived

Learn how a local machine-translation checkpoint changes language while preserving the source message.

An archive sends short notices to English- and French-speaking visitors. A translation must change the language while preserving the people, quantities, dates, permissions, and limits in the source.

The model in this lesson has one direction: English to French. That direction belongs to the checkpoint. It is not a language choice discovered from each input, and it is not changed by adding a target-language label to the output.

TipWhat you will learn

This lesson shows how to:

• load a revision-pinned English-to-French model and tokenizer locally;
• keep constructed source records and generated translations tied to IDs;
• count source tokens against the checkpoint’s 512-token input limit;
• compare translation with a source-copy baseline;
• inspect names, numbers, and negation without calling them adequacy scores; and
• reserve meaning and register judgments for a bilingual reviewer.

Load one local translation direction

huggingfaceR::hf_load_tokenizer() and huggingfaceR::hf_load_pipeline() load the prepared local snapshot through the course helper. Offline mode prevents a render-time download. The model and tokenizer files came from the same immutable revision.

suppressPackageStartupMessages({
  library(dplyr)
  library(huggingfaceR)
  library(jsonlite)
  library(knitr)
  library(purrr)
  library(reticulate)
  library(stringr)
  library(tibble)
  library(tidyr)
})

source("R/use-nlg.R")

translation_model <- load_nlg_pipeline(
  "opus_en_fr",
  "translation"
)

model_record <- translation_model$metadata |>
  select(model_id, revision, license, task_scope)

kable(
  model_record,
  col.names = c("Model", "Revision", "License", "Use in this course"),
  caption = "Pinned local checkpoint for English-to-French translation",
  row.names = FALSE
)

Pinned local checkpoint for English-to-French translation

Model	Revision	License	Use in this course
Helsinki-NLP/opus-mt-en-fr	dd7f6540a7a48a7f4db59e5c0b9c42c8eea67f18	Apache-2.0	machine translation

The package also exports hf_translate(), which calls a remote inference service. This lesson does not use it. Generation runs through the local model object returned by the pinned pipeline.

Read direction from the checkpoint

The local tokenizer configuration names English as the source and French as the target. A different direction requires a different checkpoint.

tokenizer_config_path <- file.path(
  "data-raw",
  ".cache",
  "nlg-models",
  translation_model$metadata$local_directory,
  "tokenizer_config.json"
)
generation_config_path <- file.path(
  "data-raw",
  ".cache",
  "nlg-models",
  translation_model$metadata$local_directory,
  "generation_config.json"
)

tokenizer_config <- fromJSON(tokenizer_config_path)
generation_config <- fromJSON(generation_config_path)

direction <- tibble(
  checkpoint = translation_model$metadata$model_id,
  source_language = tokenizer_config$source_lang,
  target_language = tokenizer_config$target_lang,
  runtime_language_switch = FALSE,
  checkpoint_beams = generation_config$num_beams
)

kable(
  direction,
  col.names = c(
    "Checkpoint",
    "Source",
    "Target",
    "Runtime direction switch",
    "Checkpoint beams"
  ),
  caption = "Translation direction recorded in the tokenizer configuration",
  row.names = FALSE
)

Translation direction recorded in the tokenizer configuration

Checkpoint	Source	Target	Runtime direction switch	Checkpoint beams
Helsinki-NLP/opus-mt-en-fr	en	fr	FALSE	4

Prepare constructed notices

These four English notices were written for the lesson. They are demonstrations, not human reference translations, a held-out test set, or a population quality benchmark.

translation_inputs <- tribble(
  ~source_id, ~source_text, ~protected_name, ~requires_negation,
  "MT-01",
  "Maya Chen will open Room 5 on 14 September.",
  "Maya Chen",
  FALSE,
  "MT-02",
  "The library did not cancel the workshop.",
  "",
  TRUE,
  "MT-03",
  "Visitors must bring 2 forms.",
  "",
  FALSE,
  "MT-04",
  "The volunteers may use 12 computers on Tuesday.",
  "",
  FALSE
)

kable(
  translation_inputs,
  col.names = c(
    "Source ID",
    "Constructed English source",
    "Protected name",
    "Negation required"
  ),
  caption = "Constructed records for the translation demonstration",
  row.names = FALSE
)

Constructed records for the translation demonstration

Source ID	Constructed English source	Protected name	Negation required
MT-01	Maya Chen will open Room 5 on 14 September.	Maya Chen	FALSE
MT-02	The library did not cancel the workshop.		TRUE
MT-03	Visitors must bring 2 forms.		FALSE
MT-04	The volunteers may use 12 computers on Tuesday.		FALSE

Count tokens before calling the model

Marian tokenizes words into subword units. Its configured input ceiling is 512 tokens. Truncation is not enabled here because dropping the end of a notice can drop its meaning.

translation_budget <- translation_inputs |>
  mutate(
    input_tokens = map_int(
      source_text,
      \(text) nlg_token_count(translation_model$tokenizer, text)
    ),
    model_input_limit = 512L,
    fits_without_truncation = input_tokens <= model_input_limit
  )

kable(
  translation_budget |>
    select(source_id, input_tokens, model_input_limit, fits_without_truncation),
  col.names = c(
    "Source ID",
    "Tokenizer units",
    "Input limit",
    "Fits without truncation"
  ),
  caption = "Input budgets are checked before translation",
  row.names = FALSE
)

Input budgets are checked before translation

Source ID	Tokenizer units	Input limit	Fits without truncation
MT-01	12	512	TRUE
MT-02	9	512	TRUE
MT-03	7	512	TRUE
MT-04	10	512	TRUE

For a longer source, setting truncation = TRUE would discard source tokens and become a lossy data decision. A production workflow would section the document or stop for review.

Generate French candidates

The source text goes directly to the encoder. The checkpoint ships with a four-beam generation default. The course helper overrides it with num_beams = 1 and do_sample = FALSE, a deterministic greedy choice used to keep this teaching workflow consistent. It is not a claim that the checkpoint’s four-beam setting is nondeterministic or inferior.

The output limit is 80 newly generated tokens. The returned record separately identifies whether the decoder emitted its end-of-sequence token and whether it used the full output budget. Both can be true when the checkpoint forces EOS at the cap. Neither field proves that the translation is complete for a reader.

translation_results <- translation_budget |>
  mutate(
    generation = map(
      source_text,
      \(text) {
        nlg_generate(
          translation_model,
          text,
          max_new_tokens = 80L
        )
      }
    ),
    french_candidate = map_chr(generation, "text"),
    output_tokens = map_int(generation, "output_tokens"),
    ended_by_eos = map_lgl(generation, "ended_by_eos"),
    hit_token_cap = map_lgl(generation, "hit_token_cap"),
    source_copy_baseline = source_text
  )

kable(
  translation_results |>
    select(source_id, source_text, french_candidate, source_copy_baseline),
  col.names = c(
    "Source ID",
    "English source",
    "Generated French candidate",
    "Untranslated copy baseline"
  ),
  caption = "Real local translations beside a no-translation baseline",
  row.names = FALSE
)

Real local translations beside a no-translation baseline

Source ID	English source	Generated French candidate	Untranslated copy baseline
MT-01	Maya Chen will open Room 5 on 14 September.	Maya Chen ouvrira la salle 5 le 14 septembre.	Maya Chen will open Room 5 on 14 September.
MT-02	The library did not cancel the workshop.	La bibliothèque n’a pas annulé l’atelier.	The library did not cancel the workshop.
MT-03	Visitors must bring 2 forms.	Les visiteurs doivent apporter 2 formulaires.	Visitors must bring 2 forms.
MT-04	The volunteers may use 12 computers on Tuesday.	Les volontaires peuvent utiliser 12 ordinateurs mardi.	The volunteers may use 12 computers on Tuesday.

The copy baseline preserves the English source exactly and performs no translation. It prevents a changed-looking string from being treated as automatic evidence that the task succeeded.

Inspect visible details

The next checks compare exact Arabic numerals, look for the protected person name, and look for a common French negation marker when the English source contains did not. These checks can find some clear failures. They cannot decide whether the sentence is grammatical, whether must and may have the right force, or whether the register fits the audience.

extract_numbers <- function(text) {
  str_extract_all(text, "\\b\\d+\\b")[[1]] |>
    sort()
}

translation_checks <- translation_results |>
  mutate(
    numerals_preserved = map2_lgl(
      source_text,
      french_candidate,
      \(source, output) {
        identical(
          extract_numbers(source),
          extract_numbers(output)
        )
      }
    ),
    name_present = map2_lgl(
      protected_name,
      french_candidate,
      \(name, output) {
        if (!nzchar(name)) {
          NA
        } else {
          str_detect(output, fixed(name))
        }
      }
    ),
    negation_visible = map2_lgl(
      requires_negation,
      french_candidate,
      \(required, output) {
        if (!required) {
          NA
        } else {
          str_detect(
            output,
            regex("\\b(pas|jamais|aucun)\\b", ignore_case = TRUE)
          )
        }
      }
    )
  ) |>
  select(
    source_id,
    numerals_preserved,
    name_present,
    negation_visible
  )

translation_check_display <- translation_checks |>
  mutate(
    across(
      c(numerals_preserved, name_present, negation_visible),
      nlg_screen_label
    )
  )

kable(
  translation_check_display,
  col.names = c(
    "Source ID",
    "Arabic-numeral screen",
    "Protected-name screen",
    "French-negation screen"
  ),
  caption = "Conservative surface screens for the constructed notices",
  row.names = FALSE
)

Conservative surface screens for the constructed notices

Source ID	Arabic-numeral screen	Protected-name screen	French-negation screen
MT-01	not flagged by screen	not flagged by screen	not applicable
MT-02	not flagged by screen	not applicable	not flagged by screen
MT-03	not flagged by screen	not applicable	not applicable
MT-04	not flagged by screen	not applicable	not applicable

No BLEU or chrF number is reported. Those metrics require a documented reference corpus, preprocessing signature, and corpus-level interpretation. Even then, one reference cannot represent every acceptable French rendering, and automatic overlap does not replace bilingual judgment.

Not flagged by screen means only that this finite string check found no problem. It does not verify translation adequacy. Not applicable means the source contained no protected name or negation condition to test.

Leave approval to a bilingual reader

translation_screen_counts <- translation_checks |>
  rowwise() |>
  mutate(
    applicable_screens = sum(!is.na(c_across(c(
      numerals_preserved,
      name_present,
      negation_visible
    )))),
    flagged_screens = sum(c_across(c(
      numerals_preserved,
      name_present,
      negation_visible
    )) %in% FALSE)
  ) |>
  ungroup()

translation_review <- tibble(
  review = c(
    "local model execution",
    "source within the 512-token input limit",
    "decoder stopped at EOS or the output cap",
    "surface preservation checks",
    "bilingual meaning and register review"
  ),
  status = c(
    sprintf(
      "completed: %d candidates",
      sum(nzchar(translation_results$french_candidate))
    ),
    sprintf(
      "within budget: %d/%d candidates",
      sum(translation_budget$fits_without_truncation),
      nrow(translation_budget)
    ),
    sprintf(
      "recorded: %d/%d candidates",
      sum(
        translation_results$ended_by_eos |
          translation_results$hit_token_cap
      ),
      nrow(translation_results)
    ),
    sprintf(
      "scan completed: %d flags across %d applicable checks",
      sum(translation_screen_counts$flagged_screens),
      sum(translation_screen_counts$applicable_screens)
    ),
    "pending"
  )
)

kable(
  translation_review,
  col.names = c("Review gate", "Status"),
  caption = "Automatic checks do not grant bilingual approval",
  row.names = FALSE
)

Automatic checks do not grant bilingual approval

Review gate	Status
local model execution	completed: 4 candidates
source within the 512-token input limit	within budget: 4/4 candidates
decoder stopped at EOS or the output cap	recorded: 4/4 candidates
surface preservation checks	scan completed: 0 flags across 6 applicable checks
bilingual meaning and register review	pending

The lesson contains no human ratings. A bilingual reviewer still has to compare each pair for meaning, tone, register, and natural French.

What to remember

• Translation changes language while trying to preserve the source message.
• This checkpoint’s English-to-French direction is fixed in its model files.
• Count model tokens before generation and treat truncation as data loss.
• Keep an untranslated copy baseline beside generated candidates.
• Names, numbers, and negation checks can reveal errors but cannot prove adequacy.
• Human bilingual review remains necessary before publication.

Sources

• Helsinki-NLP English-to-French model card
• OPUS-MT: open translation models for many languages
• Papineni et al. on corpus-level BLEU
• Post on reproducible machine-translation evaluation
• Hossain et al. on negation in machine translation