1 of 1

CausalSense: Leveraging Common Sense Knowledge and LLMs for Joint Event Extraction and Relation Classification

Youssra Rebboud, Pasquale Lisena, Raphael Troncy

Current approaches to event relation extraction focus on coarse causal links. We introduce a framework and dataset for modeling four fine-grained relations: cause, enable, prevent, and intend.

Extracts 4 fine-grained relations between events�(enable, cause, prevent, intend)

“The government has implemented a series of laws to prevent the abuse of animals.”

RESEARCH GOAL

CAUSALSENSE DATASET STATISTICS

DATASET CREATION

Repository�bit.ly/kflow-rel-extraction

RESULTS

+32.3% F1 improvement over state-of-the-art
End-to-end models outperform pipeline approaches with more consistent results
LLMs are less competitive
Commonsense significantly boosts performance

MAIN FINDINGS

EVENT RELATION EXTRACTION

series of laws

abuse of animals

prevent

Category	Total	Cause	Enable	Prevent	Intend	No-relation
News Data	5,207	1,978	450	500	501	1778
Common Sense	521,657	82,242	65,485	53,456	146,588	173,886
TOTAL Full Dataset	526,864	84,220	65,935	53,956	147,089	175,664
TEST SET Only real-world examples	632	351	89	52	40	100

News Data

Common Sense

3 979

5 207

320 980

526 864

Initial dataset� (Rebboud, 2023)

663

Causal News Corpus

LLM-Generated News

ATOMIC

LLM-Generated�Common Sense

Given a relation type ERx:

Prompt(ERx) = definition(Event) + definition(ERx) + request(ER) + 5 examples(ERx)

Data Augmentation with GPT3.5

Ingest data from ATOMIC

Subject	Relation	Object	Mapped relation
PersonX looks before you leap	xIntent	to be cautious	intends-to-cause
PersonX looks towards PersonY	xWant	to greet PersonY	intends-to-cause
PersonX loses 15 pounds	xEffect	has more energy	causes

Data Augmentation on Common Sense�(using Zephyr and Truthful-DPO models)

Given a relation type ERx:

Prompt(ERx) = definition(Event) + definition(ERx) + Task Instruction + Examples

Negative Sampling by randomly swapping subjects and objects

Binary classification: (causal) relation/no-relation

Sequence classification, 5 classes:�(Cause, Enable, Prevent, Intend, No-relation)

Span detection: identify the subject and object segments (event1 and event2)

3 sub-tasks

Relation Detection (RD)

Relation Classification (RC)

Event Extraction (EE)

3 strategies

Separate Tasks

End-to-End

LLM

One Model for each Task (trained and tested separately)�Tested models: BERT and RoBERTa

3 heads trained jointly�Tested models: REBEL and RoBERTa

Zero-shot and few-shot prompting �Closed and open models (GPT4 and Zephyr)

Strategy	Model	RD F1	RC F1	EE F1	Avg F1
End-to-End	REBEL	0.60	0.75	0.70	0.68
End-to-End	RoBERTa	0.98	0.78	0.20	0.65
Separate	BERT	0.92	0.70	0.60	0.74
Separate	RoBERTa	0.92	0.73	0.64	0.763
LLM	Zephyr (4-shot)	0.29	0.10	0.20	0.19
LLM	GPT-4 (2-shot)	0.49	0.46	0.35	0.43

Improve modeling of hard-to-distinguish relations� (e.g., enable vs cause, prevent)
Explore advanced prompting and reasoning strategies �(e.g., chain-of-thought) for LLM-based approaches
Apply the framework to large-scale corpora to build and refine causal knowledge graphs

FUTURE WORK