1

RAG-TEC: Extracting and Classifying Topics in

Digital News Collections Using LLMs

WSDL Research Expo — 2026

Department of Computer Science

Old Dominion University, Norfolk, VA

June 23, 2026

Brian Llinas

Advisor: Dr. Michael L. Nelson

2

I want to learn about 180 countries around the world

WSDL Research Expo – 2026

3

I want to learn about 180 countries around the world

WSDL Research Expo – 2026

4

I do not know anything about Mozambique

Mozambique is

• a country located in Southeast Africa.
• The country was colonized by Portugal in the 15th century and remained under Portuguese rule until 1975.
• Mozambique gained independence in 1975 after a long and bloody civil war.

Summary from https://en.wikipedia.org/wiki/Mozambique

WSDL Research Expo – 2026

5

https://allafrica.com

https://www.reuters.com

https://www.africanews.com

But … I could read the news to check what is happening

​

WSDL Research Expo – 2026

I do not know anything about Mozambique

6

I found a news article, what can I do?

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

WSDL Research Expo – 2026

7

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

Read the title

I found a news article, what can I do?

WSDL Research Expo – 2026

8

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

Read the title

Write a summary

I found a news article, what can I do?

Analytics Research Lab Pro Presentation

WSDL Research Expo – 2026

Mozambique is in turmoil following the assassination of two prominent opposition figures, Elvino Dias and Paulo Guambe, escalating tensions ahead of planned protests against the election results. The killings come as the country awaits the contested election results, with opposition parties alleging fraud and calling for a nationwide strike.

9

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

Read the title

Write a summary

Mozambique is in turmoil following the assassination of two prominent opposition figures, Elvino Dias and Paulo Guambe, escalating tensions ahead of planned protests against the election results. The killings come as the country awaits the contested election results, with opposition parties alleging fraud and calling for a nationwide strike.

I found a news article, what can I do?

Topic Identification: Mozambique opposition assassinations

WSDL Research Expo – 2026

10

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

https://beta.referencer.org

Identify the events that mention in the article

I found a news article, what can I do?

WSDL Research Expo – 2026

11

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

https://beta.referencer.org

Mind Map to get a contextualization

I found a news article, what can I do?

WSDL Research Expo – 2026

12

Reading many news articles and understanding a collection is time-consuming

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

WSDL Research Expo – 2026

13

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

https://www.reuters.com/world/africa/least-10-shot-dead-mozambique-post-election-protests-medical-associations-say-2024-10-30/

WSDL Research Expo – 2026

Reading many news articles and understanding a collection is time-consuming

14

https://www.africanews.com/2024/10/20/mozambique-rocked-by-killing-of-two-prominent-opposition-figures/

https://www.reuters.com/world/africa/least-10-shot-dead-mozambique-post-election-protests-medical-associations-say-2024-10-30/

https://allafrica.com/stories/202411210032.html

…

n

WSDL Research Expo – 2026

Reading many news articles and understanding a collection is time-consuming

15

Understanding news collections is time-consuming—we need a rapid and reliable contextualization through thematic understanding.

The Challenge of Understanding News Collections

WSDL Research Expo – 2026

16

One way to get contextualization is to identify the topics

[1] Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993-1022.

[2] Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794.

[3] Kapoor, S., Gil, A., Bhaduri, S., Mittal, A., & Mulkar, R. (2024). Qualitative Insights Tool (QualIT): LLM Enhanced Topic Modeling. arXiv preprint arXiv:2409.15626.

[4] Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[5] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.

​

​

Latent Dirichlet Allocation (LDA) [1]

BERTopic [2]

• Bag-of-words limitation and the need to specify the number of clusters. [2, 3]
• Fails to capture contextual nuances and relies on predefined rules.[4, 5]

​

​

• Leverages deep neural networks to learn rich, contextual representations from large amounts of text data.[2]

​

​

• Assumes that each document contains only a single topic, which may not align with real-world cases. [2, 3]
• Topic representation is still based on bag-of-words, leading to redundancy and less informative topics. [2]

• Probabilistic Topic Modeling and allows documents to be represented as mixtures of topics, ensuring interpretability, scalability, and flexibility for large-scale text analysis. [1]

​

Advantage

Limitations

WSDL Research Expo – 2026

17

How can we use LLMs to extract and classify topics from news collections for rapid contextualization?

[1] Chew, R., Bollenbacher, J., Wenger, M., Speer, J., & Kim, A. (2023). LLM-assisted content analysis: Using large language models to support deductive coding. arXiv preprint arXiv:2306.14924.

[2] Dolphin, R., Dursun, J., Chow, J., Blankenship, J., Adams, K., & Pike, Q. (2024). Extracting structured insights from financial news: An augmented llm driven approach. arXiv preprint arXiv:2407.15788.

[3] Tong, Z., Ding, Z., & Wei, W. (2025, January). EvoPrompt: Evolving Prompts for Enhanced Zero-Shot Named Entity Recognition with Large Language Models. In Proceedings of the 31st International Conference on Computational Linguistics (pp. 5136-5153).

[4] Nakshatri, N., Liu, S., Chen, S., Roth, D., Goldwasser, D., & Hopkins, D. (2023, December). Using LLM for improving key event discovery: Temporal-guided news stream clustering with event summaries. In Findings of the Association for Computational Linguistics: EMNLP 2023 (pp. 4162-4173).

[5] López, A. B., Pastor-Galindo, J., & Ruipérez-Valiente, J. A. (2024). LLM-assisted topic modeling for hate speech characterization.

WSDL Research Expo – 2026

18

RAG-TEC: Retrieval-Augmented Generation for Topic Extraction and Classification

WSDL Research Expo – 2026

A five-stage framework

19

We manually curated four collections

WSDL Research Expo – 2026

20

WSDL Research Expo – 2026

We manually curated four collections

21

From top k relevant documents within the collection, we use gpt-4o-mini to extract topics

LLM serve as a Information Extraction Model

We do not need a Natural Language Processing Model to train

WSDL Research Expo – 2026

22

Six resulting topics from the Mozambique Collection

WSDL Research Expo – 2026

From top k relevant documents within the collection, we use gpt-4o-mini to extract topics

23

LLM serve as a Classification Model

We do not need a Machine Learning Model to train

WSDL Research Expo – 2026

We use gpt-4o-mini to classify one document into one main topic and many sub-topics

24

LLM serve as a Classification Model

We do not need a Machine Learning Model to train

WSDL Research Expo – 2026

We use gpt-4o-mini to classify one document into one main topic and many sub-topics

25

To assess the topic quality for the collection, we use Topic Diversity and Coherence

WSDL Research Expo – 2026

26

WSDL Research Expo – 2026

To assess the topic quality for the collection, we use Topic Diversity and Coherence

27

WSDL Research Expo – 2026

To assess the topic quality for the collection, we use Topic Diversity and Coherence

28

Finally, we can understand the collection from a topic perspective without relying on manual inspection

WSDL Research Expo – 2026

29

WSDL Research Expo – 2026

Finally, we can understand the collection from a topic perspective without relying on manual inspection

We illustrate the process with the Mozambique collection.

Election Violence (EV), Climate Challenges (CC), Insurgency Crisis (IC), Social Media Restrictions (SMR), Economic Disparities (ED), and Humanitarian Response (HR).

30

Limitations

WSDL Research Expo – 2026

31

Future Work

WSDL Research Expo – 2026

32

Conclusion

WSDL Research Expo – 2026