Sentiment-Aware Network Extraction from News Corpus using LLMs: An Empirical Validation with Legislative Agreement

Naim Bro

School of Government, Adolfo Ibáñez University, Santiago, Chile

Millennium Institute Foundational Research on Data, Santiago, Chile

​

Complex Networks 2023

Menton, France

Introduction

• Objective: Extend computational approaches to elite network identification by employing GPT-4 to extract graphs from textual data
• Motivation: Understanding power structures in society

Background

• Prior Research: Highlight Traag's (2015) study on national political elites using text-based methods
• Limitations: Co-occurrence as a relational metric; need for a more nuanced approach

Data and Methods

• News corpora: Event Registry
• filter news clips mentioning at least two current Chilean deputies
• news clips in the period 1-Nov-2021 to 1-Sept-2023
• Methodology: GPT-4 for network extraction.
• Network Structure: Nodes represent parliamentarians; edges represent relationships with sentiment scores

GPT-4 Prompt

• Role: You are an expert political journalist in Chile, who covers news related to the Chilean Congress. You personally know many of Chile's chamber of deputies members, and you have a good understanding of who gets along with whom.
• Prompt:

Given the current state of a graph and a prompt, find in the prompt any of the names from a list that I will provide to you. Identify the relationships between these persons, and then update the state.

Example:

current state: {{ "nodes": [], "edges": [] }}

prompt: Juan Sánchez criticó a Pedro González, pero felicitó a Manuel Muñoz. Manuel Muñoz, por su lado, aplaudió a Juan Sánchez.

new state: {{ "nodes": [ {{ "id": 1, "label": "Juan Sánchez", "type" : "person"}}, {{ "id": 2, "label": "Pedro González", "type" : "person"}},{{ "id": 3, "label": "Manuel Muñoz", "type" : "person"}}], "edges": [ {{ "from": 1, "to": 2, "label": "criticized", "sentiment": "negative"}},{{ "from": 1, "to": 3, "label": "congratulated", "sentiment": "positive"}},{{ "from": 3, "to": 1, "label": "aplaudió", "sentiment": "positive"}} ] }} state: {state} prompt: {news}

Graph Representation

• Key Features: Number of deputies 46 out of 155 deputies for whom relationships were found in the news corpus, and 483 links

​

Empirical Validation

• Legislative agreement
• The proportion of times that two deputies vote in the same way
• Regression analysis correlating network sentiment with legislative behavior

​

Legislative Agreement=β0+β1×Negative Sentiment+β2×Positive Sentiment+ε

Results Overview

Key Finding: Negative sentiment predict a 5% drop in legislative agreement, independent of political party, sector, and region

R^2: 0.56

Conclusion

• Sentiment-aware graph extraction from text mirrors real-world political alliances and conflicts
• This is just the beginning: the more text input, the more accurate the description of links between public figures

Thanks!

​

Email:

naim.bro.k@uai.cl

​

Website:

https://naimbro.github.io/

​