Simple Set Sketching · HackerPapers discussions

Simple Set Sketching — mentioned by hundredwatt

Sat, 19 Jul 2025 04:04:57 GMT

> To be pedantic, not guaranteed. The xor of multiple elements may erroneously have a passing checksum, resulting in an undetected false decode The false decodes can be detected. During peeling, deleting a false decode inserts a new element with the opposite sign of count. Later, you decode this second false element and end up with the same element in both the A / B and B / A result sets (as long as decode completes without encountering a cycle). So, after decode, check for any elements present in both A / B and B / A result sets and remove them. -- Beyond that, you can also use the cell…

Paper and discussion history. A mention is not an endorsement.

Simple Set Sketching — mentioned by thomasahle

Tue, 13 Dec 2022 16:21:10 GMT

One issue with this technique is when the set of characters grow above 64, so you can no longer give each one a unique bit. This could happen if we are instead interested in a window of unique words in a long document. Amazingly there is still a solution based on XOR! It is a variant of Bloom Filters, which I know everyone loves, but it uses XOR instead of OR to combine things. Basically each word is hashed to three locations and you flip those bits. Then you have to do some probabilistic analysis. https://arxiv.org/abs/2211.03683 Edit: Actually the technique in this paper would take linear…

Paper and discussion history. A mention is not an endorsement.