A Partition Filter Network for Joint Entity and Relation Extraction

Zhiheng Yan, Chong Zhang, Jinlan Fu, Qi Zhang and Zhongyu Wei

​

​

Task Definition

NER Identify boundary and type of entity

RE Identity relation between entity pair

Joint Extract Relational triples - (subject, relation, object)

Limitations of Current Encoding Schemes

Sequential Encoding: Encoding NER and RE with a pre-defined order -> imbalanced interaction

A Novel Cascade Binary Tagging Framework for Relational Triple Extraction ACL 2020

Parallel Encoding: Encoding NER and RE with independent encoders -> insufficient interaction

A Frustratingly Easy Approach for Entity and Relation Extraction NAACL 2021

Our Solution – Joint Encoding

Joint Encoding: Encoding task-specific feature with a joint encoder – Partition Filter Encoder

Partition: Segment a set of cell neurons into two intra-task partition and one inter-task partition with two scissor gates

​

Filter: Generate task-specific feature with shared partition and corresponding task partition

​

Enforcing bilateral interaction between NER and RE:

• Partitions are generated with the joint effort of entity and relation gates
• Information in shared partition in evenly accessible to both tasks

Encoder - Partition

1. Identify cut-off points (scissor gates)

​

2. Generate three partitions with the gates

Gate Calculation

Cummax = Softmax + Cumulation

identify cut-off point

e.g. (0.1, 0.1, 0.6, 0.1, 0.1)

segment neurons before and after the cut-off point

e.g. (0.1, 0.2, 0.8, 0.9, 1.0) ≈ (0, 0, 1, 1, 1)

Partition Generation

Gather information from cells

Encoder - Filter

Feature Extraction

Entity Feature = Entity Partition + Shared Partition

Relation Feature = Relation Partition + Shared Partition

Partition Result

Entity Partition - Information reserved for NER only

Relation Partition – Information reserved for RE only

Shared Partition – Information valuable to both tasks

Decoder – Table Filling

NER Table

Word Pair prediction : Entity Span

Table Type : Entity Type

Please refer to our paper for other details of the network

RE Table

Word Pair prediction : Subj-Obj Triple (head token)

Table Type : Relation Type

Triple Extraction: Subj-Obj Triple (head + tail token)

Experiments – Data and Evaluation Standard

Evaluation Metric - F1 score

NER – Boundary and type of entity must be correct

RE – Boundary and type of entity, as well as relation between entity pair must be correct

Hyper-parameter setting

Best average result of NER and RE on development set

Experiments – Main Result

Experiment – Ablation Study

Encoder Side

Decoder Side

Investigate the Effect of RE on NER�(Comparison of In-triple and Out-of-triple Entity Extraction)

Entity Extraction is Much easier for In-triple than Out-of-triple

Absence of RE has a considerable bearing on NER

Investigate the Effect of RE on NER�(Robustness Test on Named Entity Recognition)

Introducing relation signal into network training improves robustness on NER

CrossCategory: Substitute Entities for random ones with multiple entity types

Ori: I met George at the airport

Aug: I met Washington (PER or GPE) at the airport

Conclusion

• We present a novel joint encoder – partition filter encoder , designed for Multi-task learning
• We conduct extensive experiments on a diverse pool of datasets to verify the effectiveness of our model
• We investigate the effect of RE on NER. The result suggests that RE is beneficial to NER to a large extent.

In the future, we are interested in

• Adapting the encoder in a non-autoregressive way for computation efficiency.
• Applying the encoder to tasks other than NER and RE

Thank you for your attention