Learn how LDA groups co-occurring words, how diagnostics inform the topic count, and why no single metric chooses k for you.
Priya receives a table with five numbered topics and a request to name them before lunch. The table looks tidy enough: each row has a few common words and a number.
The risk is that the neat table invites a stronger story than the model earned. A topic model will always produce topics when asked, so the analyst has to check whether the same answer appears under different random starts.
TipWhat you will learn
This lesson shows how to:
define Latent Dirichlet Allocation in everyday language;
build a paragraph-level document-feature matrix;
compare candidate topic counts with three diagnostics;
distinguish an LDA model from the algorithm used to fit it;
fit a five-topic LDA model with explicit seeds;
match topics across runs by their top terms; and
compare observed agreement with a random-label null.
Build the matrix
Latent Dirichlet Allocation, usually shortened to LDA, groups words that tend to appear in the same documents. Here each document is one paragraph. A document-feature matrix stores document rows and word columns. The value k is the number of topics the analyst asks the model to find.
Trimming keeps words that appear at least five times in the paragraph corpus. The matrix has 1,377 paragraph documents and 2,587 word features.
Using paragraphs rather than whole speeches gives the model 1,377 documents instead of 60. The tradeoff is that each document contains fewer words, so its topic mixture is based on less evidence. Document size is part of the model design, not a housekeeping choice. This is also a modest corpus for LDA, so the results describe these speeches and should not be treated as a general map of political language.
Compare candidate topic counts
There is no single correct value of k. Fewer topics make broad groups; more topics divide the same vocabulary into finer pieces. A useful value depends on what the topics will be used for.
The committed diagnostic study fits k = 3 through k = 8 under three random starts. It builds the vocabulary from training speeches and evaluates perplexity on held-out speeches. Three measures ask different questions:
Held-out perplexity asks how surprised the model is by word counts in speeches it did not fit. Lower is better.
Semantic coherence asks whether a topic’s most probable words appear together in the training paragraphs. Higher values, which are less negative here, are better.
Adjusted top-word exclusivity asks whether those words concentrate in one topic rather than being spread across all topics. Higher is better. The adjustment removes the (1/k) share expected if a word were distributed equally.
Three diagnostics disagree about the number of topics
k
Coherence
Coherence rank
Adjusted exclusivity
Exclusivity rank
Held-out perplexity
Perplexity rank
Used below
3
-1.623
4
0.274
1
1084.9
1
no
4
-1.618
3
0.196
2
1086.0
2
no
5
-1.603
1
0.157
3
1087.0
3
yes
6
-1.617
2
0.131
4
1087.6
4
no
7
-1.631
5
0.117
5
1088.3
5
no
8
-1.642
6
0.101
6
1088.9
6
no
The diagnostics do not agree. Three topics have the best held-out perplexity and adjusted exclusivity. Five topics have the best mean coherence. This lesson uses five because it gives coherence priority for this teaching example. Three would also be a defensible choice if prediction or top-word concentration mattered more. The later seed check examines only k = 5, so it cannot choose between five and three.
The coherence lead over three topics is only 0.02, while the three five-topic starts span 0.095. Rank 1 is not a decisive win here. These ranges come from three starts on one speech-grouped split, not confidence intervals or a validated optimum for another corpus.
Do not fit several values of k, read every word table, and choose the one that is easiest to name. That lets the interpretation choose the model after seeing its output. Set the diagnostics and their priority first, then read the topics. Human review still matters because predictive fit and automated coherence do not guarantee that a topic is useful or even understandable.
NoteThe model and fitting method are separate choices
The topicmodels package can fit LDA in two ways. Variational expectation maximization, or VEM, approximates the model’s probability distributions. It is the method used in this lesson. Gibbs sampling draws repeated samples from those distributions and needs choices about iterations, warm-up, and how many samples to keep. Both methods fit LDA, but they can return different answers.
Other topic models change the assumptions. A correlated topic model allows topics to appear together rather than treating their proportions as independent. A structural topic model can use document information such as year when estimating topic prevalence or content. Seeded and supervised topic models add analyst guidance or an outcome. Those are different questions, not upgrades that every analysis needs.
Fit five topics three times
The analyst chooses k = 5. The seed in control = list(seed = ...) fixes the random starting values used by the LDA fitting algorithm. It does not choose k, and it does not prove that the fitted topics are the only possible solution.
The committed diagnostic study used training and held-out speeches to inform that choice. The three objects below are new full-corpus fits used to inspect the selected resolution; they are not the diagnostic models.
top_terms <-function(model, n =10L) { beta <-posterior(model)$termsbind_rows(lapply(seq_len(nrow(beta)), function(topic) { ordered <-sort(beta[topic, ], decreasing =TRUE)tibble(topic = topic,term =names(ordered)[seq_len(n)],beta =as.numeric(ordered[seq_len(n)]) ) }))}set.seed(42)lda_a <-LDA( lda_input,k =5,method ="VEM",control =list(seed =42))set.seed(7)lda_b <-LDA( lda_input,k =5,method ="VEM",control =list(seed =7))set.seed(2024)lda_c <-LDA( lda_input,k =5,method ="VEM",control =list(seed =2024))terms_a <-top_terms(lda_a, n =10L)terms_b <-top_terms(lda_b, n =10L)terms_c <-top_terms(lda_c, n =10L)topics_a <- terms_a |>slice_head(n =8, by = topic) |>summarise(top_terms =paste(term, collapse =", "), .by = topic)kable( topics_a,col.names =c("Topic number", "Top terms"),caption ="Top terms from a five-topic LDA model with seed 42",row.names =FALSE)
Top terms from a five-topic LDA model with seed 42
Topic number
Top terms
1
country, power, can, us, must, government, people, new
The table is the most tempting output in topic modeling. It is also easy to overread. The row names are just topic numbers, and the words are high-probability terms under this fitted model.
Match topics across random starts
Topic numbers are arbitrary. Topic 1 in one run does not have to be topic 1 in another run. The comparison below matches topics by the largest total overlap in their top-10 term sets.
A permutation null reruns the comparison after breaking the term labels. The p-value reports how often those shuffled runs match or beat the observed overlap.
Frequency-preserving allocation null for seed 42 vs seed 7 overlap
Observed
Null mean
Null low
Null high
p-value
Replicates
Alternative
5.6
4.901
4.4
5.4
0.03
500
greater
The three full-corpus five-topic fits show modest repeatability. Matched topics share 5.6, 5.4, and 5.0 of their top 10 terms on average across the three pairwise comparisons. Distinct topics within the seed-42 fit already share 3.9 terms on average.
The null keeps the same 50 top-term rows, including repeated high-frequency terms, but randomly reallocates them across five equal-sized topics. Its mean is 4.901, compared with observed overlap 5.6 and p = 0.03. That is limited evidence of repeatability for these three k = 5 fits, not evidence that five is more stable than another topic count.
Changing k changes the question
A four-topic fit is useful as a sensitivity check, but it is not the same object as a five-topic fit. Once k changes, topics are not identified across models.
set.seed(42)lda_4 <-LDA( lda_input,k =4,method ="VEM",control =list(seed =42))topics_4 <-top_terms(lda_4, n =8L) |>summarise(top_terms =paste(term, collapse =", "),top_term =first(term),.by = topic )shared_first_terms <-length(intersect(topics_a$top_terms |>str_extract("^[^,]+"), topics_4$top_term))kable( topics_4 |>select(topic, top_terms),col.names =c("Topic number", "Top terms"),caption ="Top terms from a four-topic LDA model with seed 42",row.names =FALSE)
Top terms from a four-topic LDA model with seed 42
Topic number
Top terms
1
must, government, people, can, us, power, country, now
government, can, may, people, country, us, nation, world
4
public, free, upon, must, nation, states, good, world
kable(tibble(shared_first_terms = shared_first_terms),col.names ="Shared first terms with the seed-42 five-topic fit",caption ="A small comparison after changing k",row.names =FALSE)
A small comparison after changing k
Shared first terms with the seed-42 five-topic fit
2
The first terms for the four-topic model are must, people, government, and public. Two of those are also first terms in the five-topic seed-42 fit. That number is a warning about interpretation, not a stability estimate, because changing k changes the model’s target.
What to remember
LDA groups words that co-occur across paragraph documents.
This lesson used 1,377 paragraph documents and 2,587 features.
No diagnostic supplies a single correct k; the choice depends on the intended resolution and use.
Perplexity, coherence, and exclusivity can favor different topic counts.
Three full-corpus k = 5 fits showed modest repeatability; that check did not compare topic counts.
VEM and Gibbs sampling are different ways to fit LDA; other topic-model families answer different questions.
Changing k changes the question, so topic labels start over.
A topic name is a reading aid, not a finding produced by the model.
Keep the topic numbers attached to their model settings. If the setting changes, the interpretation starts over.