1 of 10

Part 4: Topic Modeling

Introduction to Natural Language Processing for Social Scientists

2 of 10

Topic Models

The general goal of a topic model is to produce interpretable document representations which can be used to discover the topics or structure in a collection of unlabeled documents.
In other words, we assume that latent topics are behind the text written and we are trying to estimate/explore such latent topics
We find latent topics by words that occur statistically more together in the document
An example of such an interpretable document representation is: document X is 20% topic a, 40% topic b and 40% topic c.
Or the social media conversation A talks 90% about war while conversation B talks 40% about hunger
You can literally model topics of any corpus

Topic Models (TM)

Underlying idea is that the words that describe one specific topic are co-ocuring together
You could, for example, provide a topic model with a set of news articles and the topic model will divide the documents in topics according to word usage
There are two main algorithms/models to estimate topics:

As said on previous slide, the topics are represented by clusters of words that are occurring together more than would be statistically likely.
This means that you have to guess the name of the topic from such clusters of words!
Let’s try it out!

Topics of news, NYT

Detecting latent topics and trends in Computer & Education Journal

Subject is “topics of research published in C&E journal over the last 40 years”
3,963 Computers & Education articles published between 1976 and 2018
Chen, X., Zou, D., Cheng, G., & Xie, H. (2020). Detecting latent topics and trends in educational technologies over four decades using structural topic modeling: A retrospective of all volumes of Computers & Education. Computers and Education, 151, 103855. https://doi.org/10.1016/j.compedu.2020.103855