New cardinality estimation algorithms for HyperLogLog sketches · HackerPapers discussions

New cardinality estimation algorithms for HyperLogLog sketches — mentioned by rahimiali

Sun, 14 Mar 2021 20:58:18 GMT

based on the comments it appears to rely on this paper: https://arxiv.org/abs/1702.01284

Paper and discussion history. A mention is not an endorsement.

New cardinality estimation algorithms for HyperLogLog sketches — mentioned by stochastic_monk

Wed, 14 Nov 2018 20:08:33 GMT

In my benchmarks, HyperLogLogs unilaterally outperform MinHash* by a constant factor, while Bloom Filters of the same size become more accurate than hyperloglogs in the 100-500kB range (though this number increases with the cardinalities of sets being compared). The biggest advantage a HyperLogLog has over MinHash and bloom filters is that it works well regardless of the sizes of the sets being compared, while MinHash is sensitive to differences in set size and bloom filters perform terribly in the small sketch domain. (These results aren't yet public, but they will be soon.) * Part of this…

Paper and discussion history. A mention is not an endorsement.

New cardinality estimation algorithms for HyperLogLog sketches — mentioned by stochastic_monk

Mon, 08 Jan 2018 19:22:05 GMT

Great work! I haven't messed around with the count-min augmentation for HLLs, but I've had a lot of success in practice. I've experimented with using HLL sketches for set operations for genome comparisons. The great thing about these operations is that the cost of operations scales with the size of the sketch, not the size of the dataset being sketched. Add SIMD acceleration, and you have an extremely fast, compact, accurate way to perform approximate set operations. Unfortunately, the naive intersection operation (a sketch consisting of an element-wise 'min' of the counts in both sketches)…

Paper and discussion history. A mention is not an endorsement.

New cardinality estimation algorithms for HyperLogLog sketches — mentioned by jbapple

Mon, 26 Jun 2017 14:45:32 GMT

How does the Xiao et al. paper (which introduces TailCut, "Better with Fewer Bits: Improving the Performance of Cardinality Estimation of Large Data Streams", <http://cse.seu.edu.cn/PersonalPage/csqjxiao/csqjxiao_files/p...) compare with Ertl's "New cardinality estimation algorithms for HyperLogLog sketches", <https://arxiv.org/abs/1702.01284>?

Paper and discussion history. A mention is not an endorsement.