Text Embeddings Reveal (Almost) As Much As Text · HackerPapers discussions

Text Embeddings Reveal (Almost) As Much As Text — mentioned by hallh

Fri, 17 Jan 2025 23:24:18 GMT

No, the original text can largely be recovered from embeddings[0] if you know which embedding model was used. [0] https://arxiv.org/abs/2310.06816

Paper and discussion history. A mention is not an endorsement.

Text Embeddings Reveal (Almost) As Much As Text — mentioned by visarga

Fri, 12 Jul 2024 08:41:37 GMT

I remembered about a paper that sheds light on this issue. An embedding can store/recover exactly a short sentence: > a multi step method that iteratively corrects and re embeds text is able to recover 92% of 32-token text inputs exactly https://arxiv.org/abs/2310.06816 So it's probably 1 sentence == 1 embedding

Paper and discussion history. A mention is not an endorsement.

Text Embeddings Reveal (Almost) As Much As Text — mentioned by tripplyons

Sun, 16 Jun 2024 02:04:30 GMT

Text embeddings should not be used as a form of encryption. They are optimized to contain as much information as possible from the text they represent and can often be decoded into their input: https://arxiv.org/abs/2310.06816

Paper and discussion history. A mention is not an endorsement.

Text Embeddings Reveal (Almost) As Much As Text — mentioned by jncraton

Tue, 05 Mar 2024 19:43:39 GMT

You might be interested in "Text Embeddings Reveal (Almost) As Much As Text": > We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes. https://arxiv.org/pdf/2310.06816.pdf There's certainly information loss, but there is also a lot of information still present.

Paper and discussion history. A mention is not an endorsement.

Text Embeddings Reveal (Almost) As Much As Text — mentioned by wint3rmute

Fri, 15 Dec 2023 21:56:40 GMT

A recent paper [1] shows that what you're describing is possible to some degree - you can reproduce text from its embeddings. The paper provides the code implementing the reversal process, so you could quickly hack together a prototype :) I also recommend a video by Yannic Kilcher [2], explaining the paper. [1] https://arxiv.org/abs/2310.06816 [2] https://www.youtube.com/watch?v=FY5j3P9tCeA

Paper and discussion history. A mention is not an endorsement.

Text Embeddings Reveal (Almost) As Much As Text — mentioned by visarga

Wed, 01 Nov 2023 05:49:36 GMT

Phrase embeddings could bring a 32x reduction in sequence length because: > Text Embeddings Reveal (Almost) As Much As Text. ... We find that although a naïve model conditioned on the embedding performs poorly, a multi step method that iteratively corrects and re embeds text is able to recover 92% of 32-token text inputs exactly. We train our model to decode text embeddings from two state of the art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes. https://arxiv.org/abs/2310.06816

Paper and discussion history. A mention is not an endorsement.

Text Embeddings Reveal (Almost) As Much As Text — HN submission

Fri, 13 Oct 2023 07:03:47 GMT

Text embeddings reveal almost as much as text

165 points; 42 discussion replies on HN. Paper and source history. A submission is not an endorsement.