arXiv:2010.03090cs.DB
The majority of text is stored in UTF-8, which must be validated on ingestion. We present the lookup algorithm, which outperforms UTF-8 validation routines used in many libraries and languages by more than 10 times using commonly available SIMD instructions. To ensure reproducibility, our work is freely available as open source software.
Linked by 3 identifiable commenters in 3 comments across 3 threads, from Sep 2022 to May 2024. Found in 2 HN submissions from Oct 2020 to Oct 2020. Open submission discussions.
From https://github.com/websockets/utf-8-validate/tree/master/dep... > This C++ library is part of the JavaScript package utf-8-validate. The utf-8-validate package is routinely downloaded more than a million times per week. > If you are using Node JS (19.4.0 or better), you…
Modern CPUs can look ahead in the instruction stream and run 4 or more instructions simultaneously. This isn't easy: you have to respect data dependencies where one instruction depends on the output of a previous one. When something depends on a load instruction that missed in…
Connected through shared HN threads, not inferred similarity or a recommendation.
Good paper on UTF-8 validation performance: https://arxiv.org/pdf/2010.03090 The relatively simple algorithm (lookup) can be several times faster than conventional algorithms at a common task using nothing more than the instructions available on commodity processors. It requires…
3pmView on HNParent context