Computing Extremely Accurate Quantiles Using t-Digests · HackerPapers discussions

Computing Extremely Accurate Quantiles Using t-Digests — mentioned by cyral

Tue, 14 Sep 2021 16:51:24 GMT

There are some newer data structures that take this to the next level such as T-Digest[1], which remains extremely accurate even when determining percentiles at the very tail end (like 99.999%) [1]: https://arxiv.org/pdf/1902.04023.pdf / https://github.com/tdunning/t-digest

Paper and discussion history. A mention is not an endorsement.

Computing Extremely Accurate Quantiles Using t-Digests — mentioned by jmacd333

Fri, 20 Mar 2020 17:43:57 GMT

The Circllhist algorithm is interesting, but I get an uneasy feeling from the use of a _relative error_ measure to evaluate the performance of quantile estimation. Note that other authors don't use this measure. Dunning uses Mean Absolute Error in his latest T-digest paper: https://arxiv.org/pdf/1902.04023.pdf Cohen uses Normalized Root-Mean-Squared Error to evaluate sampling schemes, which are equally capable of estimating latency quantiles: https://dl.acm.org/doi/abs/10.1145/3234338 The problem with Relative Error as a measure of accuracy is that it depends on the location of the…

Paper and discussion history. A mention is not an endorsement.

Computing Extremely Accurate Quantiles Using t-Digests — HN submission

Fri, 15 Feb 2019 12:43:04 GMT

Computing extremely accurate quantiles using t-digests

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.