A performance comparison of Dask and Apache Spark for data-intensive neuroimaging pipelines · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/1907.13030/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTA performance comparison of Dask and Apache Spark for data-intensive neuroimaging pipelines — mentioned by peterthehackerhttps://news.ycombinator.com/item?id=26136282urn:hackerpapers:mention:1907.13030:26136282Sun, 14 Feb 2021 21:09:29 GMT<p>I would be interested in seeing some spark v dask benchmarks too. Haven’t seen any yet though. In my experience, Dask really shines when you implement custom numpy computations that could only be done in spark UDFs. We saw a decent performance difference there, but for common built-in computations I’d imagine that spark has better performance. Edit: after some googling I found this paper with benchmarks. https://arxiv.org/pdf/1907.13030.pdf &gt; Results show that despite slight differences between Spark and Dask, both engines perform comparably. However, Dask pipelines risk being limited by…</p><p><a href="https://hackerpaper.kaipfister.de/p/1907.13030/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.DC