Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks(2019)
Sentence-BERT derives semantically meaningful sentence embedding that can be compared using cosine-similarity. BERT achieved new state-of-the art performance on various sentence-pair regression tasks using a cross-encoder. A cross-encoder accepts two sentences as input to the transformer network and the target value is predicted. Semantic textual similarity is one of the sentence-pair regression tasks. However, this setup is often not scalable for various pair regression tasks due to many possible combinations. The semantic search that maps each sentence to a vector space where semantically similar sentences are close alleviates the combinatorial explosion. Sentence-BERT uses a siamese network in which the two BERT networks have tied weights such that the produced sentence embeddings can be semantically compared using cosine-similarity.