Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks — mentioned by montebicyclelo

Wed, 25 Oct 2023 07:37:49 GMT

LLM vectors do have decent linear properties already. But for document embedding purposes they are often further trained for retrieval via cosine similarity, which enhances this, e.g. see table 1 in [1], avg retrieval performancs using BERT goes up from 54 to 76 after fine-tuning for embeddings. [1] https://arxiv.org/pdf/1908.10084.pdf

Paper and discussion history. A mention is not an endorsement.

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks — mentioned by PaulHoule

Tue, 07 Feb 2023 16:25:47 GMT

Most of the practitioners I see attempting this are running text through an embedding and then using cosine similarity or something similar as a metric. Nils has written a lot of papers https://www.nils-reimers.de/ and I think the Medium post you are talking about is https://medium.com/@nils_reimers/openai-gpt-3-text-embedding... and that SBERT is a Siamese network over BERT embeddings https://arxiv.org/abs/1908.10084 which one would expect to do better than cosine similarity if it was trained correctly. I'd imagine the same Siamese network approach he is using would work better than cosine…

Paper and discussion history. A mention is not an endorsement.

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks · HackerPapers discussions

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks — mentioned by montebicyclelo

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks — mentioned by PaulHoule