HackerPapers

arXiv:1902.08318cs.DB

Parsing Gigabytes of JSON per Second

Geoff Langdale, Daniel Lemire

Abstract

JavaScript Object Notation or JSON is a ubiquitous data exchange format on the Web. Ingesting JSON documents can become a performance bottleneck due to the sheer volume of data. We are thus motivated to make JSON parsing as fast as possible. Despite the maturity of the problem of JSON parsing, we show that substantial speedups are possible. We present the first standard-compliant JSON parser to process gigabytes of data per second on a single core, using commodity processors. We can use a quarter or fewer instructions than a state-of-the-art reference parser like RapidJSON. Unlike other validating parsers, our software (simdjson) makes extensive use of Single Instruction, Multiple Data (SIMD) instructions. To ensure reproducibility, simdjson is freely available as open-source software under a liberal license.

arXiv comments
software: https://github.com/lemire/simdjson
Journal reference
The VLDB Journal, 28(6), 2019
DOI
doi:10.1007/s00778-019-00578-5
Subjects
cs.DBDatabases cs.PFPerformance

On Hacker News

Linked by 5 identifiable commenters in 6 comments across 6 threads, from Mar 2019 to Sep 2026. Found in 4 HN submissions from May 2020 to Oct 2026. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments6 threads

  1. Bitap: My favorite string matching algorithm 1 comment

    • I like burntsushi's string-searching work because it's well documented, split modularly into libraries/applications, and runs the gamut from low- to high-level (his blog posts and comments online are extremely helpful too). I use these three tools which he maintains: Low level…

      RendelloView on HNParent context

  2. Condor's Cuzco RISC-V Core at Hot Chips 2025 1 comment

    • Bit manipulation instructions are great for high-performance code, too, because they allow conditional computing without branching. Some real-world examples in simdjson: https://arxiv.org/pdf/1902.08318

      0x000xca0xfeView on HNParent context

  3. 1BRC merykitty's magic SWAR: 8 lines of code explained in 3k words 1 comment

    • If people like stuff like this, the simdjson paper uses similar techniques, is extremely well-written and has great examples. Paper: https://arxiv.org/abs/1902.08318 Github: https://github.com/simdjson/simdjson

      winwangView on HNParent context

  4. Word-Aligned Bloom Filters 1 comment

    • I find these optimizations fascinating. Anyone familiar with Lemire likely knows about this, but I listened to a podcast episode[1] with him a few days ago and learned about `simdjson`, the tool he authored that parses JSON at 25x the speed of the standard C++ library.[2][3]…

      RendelloView on HNParent context

  5. How fast can a BufferedReader read lines in Java? 1 comment

    • Lemire is one of the leading experts on string matching and the author of several core libraries you probably use every day. edit fine, so instead maybe click on the links in the post to see that this article is just one of a series. He's probably tired of copy-pasting the specs…

      aristusView on HNParent context

  6. An Intel Programmer Jumps over Wall: First Impressions of ARM SIMD Programming 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.