A frustratingly easy way of extracting political networks from text

Naim Bro

Escuela de Gobierno

Universidad Adolfo Ibáñez

Demo chatGPT

Little data or very domain-specific

• Initially, SNA was constrained by the manual collection and analysis of data, which limited studies to smaller, more manageable networks
• While modern digital platforms provide vast amounts of data, they tend to be limited in their scope
• "Twitter Conference" Phenomenon: The abundance of Twitter data and its accessibility through an API led to its predominant use in computational social science research during the 2010s

Text-to-graph techniques change this landscape

• GPT-4 processes large-scale datasets to study complex networks efficiently
• Adaptable to various fields, allowing broad usage without specialized network analysis skills
• Practical Examples:
• network of relations in the bible
• network of relations among jet set using tabloids
• network of relation among politicians during the 1891 civil war using historical newspapers
• etc

Current approaches

• Social science
• Manual Coding: Traditionally relies on labor-intensive manual coding to map relationships based on detailed qualitative analysis.
• Often constrained by the availability and accessibility of data, limiting the size and scope of networks analyzed.
• Focuses on in-depth understanding of network dynamics within specific contexts, such as political influence or elite networks.
• Computer science
• Less accessible to social scientists

“The specific political distinction to which political actions and motives can be reduced is that between friend and enemy.”

​

Carl Schmitt

link

Objectives

• Demonstrate the application of GPT-4 in the field of sociology and political science, specifically for network extraction from text
• Show the importance of the emotional valence of links in political networks
• Validate the extracted networks using 'legislative agreement' as a metric, assessing how sentiments and voting patterns correlate

Data

• Analyzed 1,009 Chilean political news articles from November 2021 to September 2023.
• Using Event Registry, used news clips that mention members of the Chilean Chamber of Deputies

The prompt

• GPT-4 employed for automatic text analysis focusing on entity recognition and linking
• Incorporated sentiment analysis to differentiate between positive, negative, and neutral relationships

You are an advanced text analysis system, skilled in processing political news related to the Chilean Congress. Your expertise lies in analyzing written content in both Spanish and English to identify relationships between members of Chile’s Chamber of Deputies, based on a predefined list of their names. Here’s the list of Chilean deputies you’ll focus on: {names}.

Your task is to cross-reference mentions in the news clips with this list to accurately identify the deputies. Remember, only consider those deputies who are explicitly mentioned in the clip, based on this list.

Output

La diputada, Pamela Jiles, apuntó contra Camila Vallejo, luego de sus declaraciones por la aprobación de la Pensión Garantizada Universal. Pamela Jiles tiene el objetivo de sacar adelante la iniciativa del Quinto Retiro de los fondos previsionales, sin embargo, desde el Gobierno de Boric han manifestado que no es una opción viable para ellos.

Network of interactions in the media

35%

95%

Nathalie Castillo

Agustín Romero

Chiara Barchiesi

Luis Sánchez

Legislative agreement

Legislative agreement: distribution

Min: 35.4%

Max: 96.3%

Median: 63.2%

Mean: 63.3%

Standard dv.: 11.5%

Min: 35.4%

Max: 96.3%

Median: 63.2%

Mean: 63.3%

Standard dv.: 11.6%

Regression at the level of edges

Regression from the node embeddings

Conclusion

• Text-to-graph methods have the potential to solve data issues traditionally associated with social network analysis
• no only larger networks, but also more diverse fields of application
• At least for political networks, the emotional valence of links is crucial

What’s next

• LLMs are very accessible to many social scientists, and do a great job at extracting graphs from text
• But we need a comparative study of its performance versus more specialized, state-of-the-art methods in computer science
• Also, we are building a great network of Chilean elites over the next several months
• What questions can we ask the resulting network?

Thank you!

​

naim.bro.k@uai.cl