Hyperboloid embeddings and Hypersphere similarity metrics for Sentence-BERT
Garvit Goyal, Gunjan Gupta, Tusheet Goli
Sentence-BERT
(SBERT)
Sentence-BERT
(SBERT)
Architecture
Our Goal
Datasets
STSb
(Semantic Textual Similarity Benchmark)
SNLI
(Stanford Natural Language Inference)
Model
Proposed Architecture
Hyperspherical loss functions
Source: Wang and Isola (2020)
Hyperboloid Embeddings
Implementation Details
Implementation Details
Implementation Details
Results
Model | Distance Function | Loss Function | STS Benchmark Test Performance (Spearman coefficient) |
BERT-STSB-Baseline | Cosine | Contrastive | 0.839 |
BERT-STSB-CNN + Mean Pooling | Cosine | Contrastive | 0.840 |
BERT-STSB-CNN + Max Pooling | Cosine | Contrastive | 0.824 |
BERT-STSB-hyperboloid | Lorentz | Contrastive | 0.591 |
BERT-NLI-hypersphere | Euclidean | Uniformity - Alignment | 0.547 |
BERT-NLI-hypersphere + hyperboloid | Lorentz | Uniformity - Alignment | 0.536 |
Future Work
References
Thank You!