Learn why sentiment scores from word lists and trained labels can disagree, especially with short feedback.
At the end of the week, Nia receives five customer comments and one blank rating cell. The dashboard wants a quick mood score, but the comments are too short to hide the weak spots.
One row thanks the service. Another says a parcel arrived late. A third says the sender could not reset a password. The same column holds praise, complaint, and language the English word list will miss.
TipWhat you will learn
By the end of this lesson, you will be able to:
score comments with the Bing sentiment lexicon;
count how many comments receive no lexicon match;
identify context failures such as negation and domain wording;
contrast a lexicon score with labels from the same domain; and
explain why a word score is not a claim about a person’s inner state.
Load the feedback file
The feedback file is fictional. It was written for teaching and contains no real customers, parcels, accounts, or service records.
readr opens the CSV, dplyr and tibble shape the rows, tidyr handles missing cells, purrr repeats scoring, stringr checks words, tidytext supplies the lexicon, ggplot2 gives a small plot, and quanteda supplies the bundled inaugural corpus through the shared helper. The file keeps the blank rating as a string first because the project does not let read_csv() guess missing values.
library(readr)library(dplyr)library(tibble)library(tidyr)library(purrr)library(stringr)library(tidytext)library(ggplot2)library(quanteda)feedback <-read_csv("data/customer_feedback.csv",na =character(),col_types =cols(feedback_id =col_integer(),submitted_at =col_date(),channel =col_character(),rating =col_character(),comment =col_character() )) |>mutate(rating =if_else(rating =="", NA_integer_, as.integer(rating)) )knitr::kable( feedback,col.names =c("ID", "Submitted", "Channel", "Rating", "Comment"),caption ="Customer feedback used in this lesson",row.names =FALSE)
Customer feedback used in this lesson
ID
Submitted
Channel
Rating
Comment
1001
2026-08-21
email
5
Delivery was quick, thank you!
1002
2026-08-22
web
4
The café listing was easy to find.
1003
2026-08-23
chat
NA
I could not reset my password.
1004
2026-08-24
email
2
The parcel arrived late.
1005
2026-08-25
web
5
Muy útil y fácil de usar.
The blank rating stays visible. Turning it into a zero would invent a rating the customer did not give.
Count sentiment words
A sentiment lexicon is a word list with labels attached. The Bing Liu opinion lexicon in tidytext comes from product-review research by Minqing Hu and Bing Liu. It has 6,786 English words, each marked positive or negative. The source page asks users to cite the work and states research-use terms rather than an open-data licence. A lexicon score counts matches and gives positive words +1 and negative words -1.
In this five-row walkthrough, rows 1003, 1004, and 1005 contain no Bing word at all. Treat those as inspectable examples, not as a rate. The English lexicon misses the Spanish praise in row 1005. It also gives the late parcel complaint no negative word.
Measure lexicon silence on a larger corpus
To ask how often the lexicon has nothing to say, use a larger text source. The shared inaugural helper returns 1,377 public-domain paragraphs after a 25-word minimum-length filter. That filter matters because longer documents have more chances to contain a word from any lexicon. The same Bing word list is applied to each paragraph, and the code counts paragraphs with no positive or negative match. This is also a domain-mismatch example: a product-review lexicon is being applied to political oratory, so any score must be read as a warning sign, not as validated sentiment.
knitr::kable( coverage_by_length |>mutate(zero_match_share =round(zero_match_share, 4)),col.names =c("Paragraph length bin","Paragraphs","Paragraphs with no Bing match","Share with no Bing match","Median words" ),caption ="Bing lexicon silence falls as paragraph length rises",row.names =FALSE)
Bing lexicon silence falls as paragraph length rises
Paragraph length bin
Paragraphs
Paragraphs with no Bing match
Share with no Bing match
Median words
25-49 words
441
33
0.0748
35
50-99 words
497
8
0.0161
69
100+ words
439
0
0.0000
154
On the larger corpus, 41 of 1,377 paragraphs have no Bing match, a share of 0.0298 after the 25-word filter. The median paragraph has 6 matched words. The length table gives the more honest finding: 7.48 percent of 25-49 word paragraphs have no match, 1.61 percent of 50-99 word paragraphs have no match, and none of the 439 paragraphs with at least 100 words lack a match. That shows the direction expected from document length. It does not estimate the feedback result: those comments are shorter than every paragraph measured here and come from another domain. The 1-24 words bin is empty because the shared corpus helper keeps paragraphs of at least 25 words.
Read the failures before trusting the score
The lexicon sees words, not the situation around them. Negation means a word such as helpful can be flipped by nearby not. Domain wording means a plain customer-service problem, such as a password reset, may carry no sentiment word. Sarcasm is harder still because the same surface words can be sincere or barbed.
failure_cases <- lexicon_scores |>mutate(issue =case_when( feedback_id ==1003L ~"account wording with no matched sentiment word", feedback_id ==1004L ~"domain complaint with no matched word", feedback_id ==1005L ~"non-English praise",TRUE~NA_character_ ) ) |>filter(!is.na(issue)) |>select(feedback_id, rating, lexicon_score, issue, comment)thank_tokens <- token_scores |>filter(feedback_id ==1001L, word =="thank") |>select(word, sentiment, token_score)negation_examples <-tibble(phrase =c("helpful", "not helpful")) |> tidytext::unnest_tokens(word, phrase, drop =FALSE) |>left_join(bing, by =join_by(word), multiple ="all") |>mutate(token_score =case_when( sentiment =="positive"~1L, sentiment =="negative"~-1L,TRUE~0L ) ) |>summarise(matched_words =sum(token_score !=0L),lexicon_score =sum(token_score),.by = phrase )knitr::kable( failure_cases,col.names =c("ID", "Rating", "Lexicon score", "Failure type", "Comment"),caption ="Rows where the word list misses the practical reading",row.names =FALSE)
Rows where the word list misses the practical reading
ID
Rating
Lexicon score
Failure type
Comment
1003
NA
0
account wording with no matched sentiment word
I could not reset my password.
1004
2
0
domain complaint with no matched word
The parcel arrived late.
1005
5
0
non-English praise
Muy útil y fácil de usar.
knitr::kable( negation_examples,col.names =c("Phrase", "Matched sentiment words", "Lexicon score"),caption ="A bag-of-words lexicon does not flip the score for negation",row.names =FALSE)
A bag-of-words lexicon does not flip the score for negation
Phrase
Matched sentiment words
Lexicon score
helpful
1
1
not helpful
1
1
Row 1003 is an account-access sentence with no matched sentiment word. The separate not helpful example shows a real negation failure: it scores the same as helpful because helpful is positive and not has no flipping rule here. Row 1001 shows the opposite risk: thank is positive, but a word list cannot tell from the word alone whether thanks are warm, routine, or sarcastic.
Train on labels from the setting
A trained approach starts with labels from the setting that matters. Here the rating field supplies three tiny labels for demonstration: ratings of 5 become satisfied, rating 2 becomes frustrated, and rating 4 is left out because it is mild. The seed before this deterministic fit is 4901.
The score is an equal-prior, Laplace-smoothed multinomial naive Bayes calculation. Adding one to every word count prevents an unseen word from making a class probability zero. “Equal-prior” means the calculation gives the two labels equal starting weight instead of using the two-to-one training-row imbalance; a deployed naive Bayes classifier would make that prior choice explicit.
A tiny rating-trained scorer compared with the lexicon
ID
Rating
Training role
Comment
Lexicon score
Rating-trained score
Rating-trained label
1001
5
fitting row
Delivery was quick, thank you!
1
1.897
satisfied
1002
4
excluded mild rating (no reference label)
The café listing was easy to find.
1
-0.627
frustrated
1003
NA
unrated row
I could not reset my password.
0
0.000
no seen training words
1004
2
fitting row
The parcel arrived late.
0
-4.027
frustrated
1005
5
fitting row
Muy útil y fácil de usar.
0
2.277
satisfied
This trained scorer learns from only three labelled rows, so it is a mechanics example rather than an evaluation. Rows 1001, 1004, and 1005 are fitting rows, and reporting success on them would be resubstitution. Row 1002 was excluded because its mild rating has no reference label in this two-class scheme. The scorer still returns frustrated, but that is inspection output with no right-or-wrong answer, not a held-out evaluation. A larger trained model can use context if the training labels record the domain you care about. Its cost is that labels must be collected, checked, and kept in step with the way people write.
Emotion needs stronger evidence
Sentiment is usually a coarse positive-negative score. Emotion analysis asks for labels such as anger, fear, sadness, joy, or surprise. That claim is much stronger.
score_range <- lexicon_scores |>summarise(lowest =min(lexicon_score),highest =max(lexicon_score),distinct_scores =n_distinct(lexicon_score),.groups ="drop" )score_plot <- lexicon_scores |>count(lexicon_score, name ="comments")knitr::kable( score_plot,col.names =c("Lexicon score", "Comments"),caption ="Only two lexicon scores appear in the feedback file",row.names =FALSE)
Only two lexicon scores appear in the feedback file
Lexicon score
Comments
0
3
1
2
Figure 1: Only two Bing lexicon scores appear in the five fictional feedback comments.
Figure 2: Only two Bing lexicon scores appear in the five fictional feedback comments.
A positive-word count is not an emotion. Self-reported emotion and observed emotion can differ, and neither appears in this file. A score such as 0.62 would be a number about words or a model output, not a claim about a person’s inner state.
What to remember
A sentiment lexicon counts words it recognizes.
On the 25-word-filtered inaugural corpus, 41 of 1,377 paragraphs have no Bing match, and silence falls as paragraph length rises.
Negation, language, domain wording, and sarcasm can break a word-count score.
A trained approach needs labels from the setting it will serve.
Emotion claims require evidence beyond positive and negative words.
Bing Liu opinion lexicon, created for product-review sentiment research by Minqing Hu and Bing Liu; the source page requests citation and states research-use terms.
