ParPaRaw: Massively Parallel Parsing of Delimiter-Separated Raw Data · HackerPapers discussions

ParPaRaw: Massively Parallel Parsing of Delimiter-Separated Raw Data — mentioned by criticaltinker

Wed, 29 Dec 2021 05:56:50 GMT

DFAs and NFAs may seem like academic novelties but they are amazingly powerful for certain problems. For example, naively searching a string of length N for P different substrings will yield an algorithm that has roughly O(N * P) worst case time complexity. But implementing that algorithm as a DFA yields an efficient construction that has O(N) worst case time complexity. This results in considerable speedups for complex pattern recognition tasks on large inputs, and yields other useful properties. As a real world example, a CSV file can be parsed in parallel on a GPU by using a DFA based…

Paper and discussion history. A mention is not an endorsement.

ParPaRaw: Massively Parallel Parsing of Delimiter-Separated Raw Data — mentioned by ejbs

Fri, 28 Jun 2019 08:22:08 GMT

> They did not include the time it took to transfer the input file from memory to the GPU To give some context: PCIe 3.0 x16 achieves about 11-12 GB/s. So, transfers for benchmarked 53MB file would add ~4.8 ms (per direction, assuming pinned host memory) Now, they could also pursue a streaming approach, processing in "micro-batches". I.e., pipelining input transfers (HOST->GPU), GPU-based processing, and transfer of results (GPU->HOST), as pursued by [1] (disclaimer: I'm a co-author). This lower's processing latency. Since the interconnect is full-duplex, it would just limit the processing…

Paper and discussion history. A mention is not an endorsement.