Knowledge Graph Embedding by Translating on Hyperplanes

TransH

TREY �research

Agenda

• Problems
• Contributions
• The propose framework
• Experiments
• Conclusions

The contents of my presentation

2

TREY �research

Knowledge Graphs

• A knowledge graph is a multi-relational graph composed of entities as nodes and relations as different types of edges.
• An instance of edge is a triplet of fact (head entity, relation, tail entity) (denoted as (h, r, t)).
• Two major difficulties in building large scale knowledge graphs:
• A knowledge graph is a symbolic and logical system while applications often involve numerical computing in continuous spaces;
• It is difficult to aggregate global knowledge over a graph.

​

TransH

3

TREY �research

Problems

The current state of the art methods have some limitations.

• TransE does not do well in dealing with reflexive, one-to-many, many-to-one, and many-to-many mapping properties.
• Some complex models are capable of preserving these mapping properties but sacrifice efficiency in the process. The model complexity and running time is significantly increased accordingly

=> How to make a good trade-off between model capacity and efficiency to deal with this problem?

TransH

4

TREY �research

Contributions

The idea is a mechanism of projecting to the relation-specific hyperplane, it enables different roles of an entity in different relations/triplets.

• Proposed a method named translation on hyperplanes (TransH) which interprets a relation as a translating operation on a hyperplane.
• It can deal with the problems of TransE on reflexive/one-to-many/many-to-one/many-to-many relations.
• keeping the model complexity almost the same as that of TransE.
• Proposed a simple trick to reduce the chance of false negative labeling by utilizing the mapping properties of relations in turn.

=> The mode shows the impressive improvements on different metrics of predictive accuracy.

TransH

5

TREY �research

The proposed framework

TransH

6

TREY �research

The proposed framework - Training

There are 3 components:

TransH

7

TREY �research

Reducing False Negative Labels

+ Previous methods simply get negative triplets by randomly corrupting the golden triplet

+ As a real knowledge graph is often far from completed, this way of randomly sampling may introduce many false negative labels into training.

+ Using different probabilities for replacing the head or tail entity when corrupting the triplet, which depends on the mapping property of the relation, i.e., one-to-many, many-to-one or many-to-many.

=> give more chance to replacing:

• the head entity if the relation is one-to-many
• the tail entity if the relation is many-to-one

+ Among all the triplets of a relation r, we first get the following two statistics:

+ tph: the average number of tail entities per head entity.

+ hpt: the average number of head entities per tail entity.

• Define a Bernoulli distribution to replace head or tail:

+ corrupt the triplet by replacing the head by probability

​

+ corrupt the triplet by replacing the tail by probability

TransH

8

TREY �research

Experiments - Tasks

+ They empirically study and evaluate related methods on three tasks:

• Link prediction.
• Triplets classification.
• Relational fact extraction.

All three tasks evaluate the accuracy of predicting unseen triplets, from different viewpoints and application context.

+ They used several datasets for specific tasks.

TransH

9

TREY �research

Experiments - Link Prediction

+ They used the same two data sets which are used in TransE including:

- WN18, a subset of Wordnet;

- FB15k, a relatively dense subgraph of Freebase where all entities are present in Wikilinks database.

+ Aggregated over all the testing triplets, two metrics are reported: the averaged rank (denoted as Mean), and the proportion of ranks not larger than 10 (denoted as Hits@10).

TransH

10

TREY �research

Experiments - Link Prediction

+ They dig into the detailed results of different mapping categories of relations to confirm that the reflexive/one-to-many/many-to-one/many-to-many relations can be better handled.

TransH

11

TREY �research

Experiments - Link Prediction

+ The results of Hits@10 on some typical one-to-many/many-to-one/many-to-many/reflexive relations shows the improvement of TransH over TransE on these relations are very promising.

TransH

12

TREY �research

Experiments - Triplets Classification

+ This task is to confirm whether a given triplet (h, r, t) is correct or not, i.e., binary classification on a triplet.

+ Three data sets are used in this task. Two of them are the same as in NTN:

- WN11, a subset of Wordnet;

- FB13, a subset of Freebase;

- FB15k.

​

TransH

13

TREY �research

Experiments - Relational Fact Extraction from Text

+ Extracting relational facts from text is an important channel for enriching a knowledge graph.

+ This experiment involves two major parts: text side extraction model and knowledge graph embedding.

+ The precision-recall curves for TransE and TransH may be obtained by applying the same rule of combining the score from knowledge graph embedding with the score from the text side model.

+ TransH consistently outperforms TransE as a “prior” model on improving the text side extraction method Sm2r.

TransH

14

TREY �research

Conclusion

+ A knowledge graph can be embedded in a continuous vector space using the novel model TransH.

+ This model can inherit the effectiveness of TransE while addressing the shortcomings of the reflexive/one-to-many/many-to-one/many-to-many interactions.

+ Extensive testing on the link prediction, triplet classification, and relational fact extraction problems demonstrates that TransH offers promising advancements over TransE.

+ The method for decreasing false negative labeling suggested in this paper is also supported by research.

TransH

15

TREY �research

Questions?

TransH

16

TREY �research

17

TREY �research