SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems · HackerPapers discussions

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by joe_the_user

Tue, 18 Feb 2025 19:56:58 GMT

Related: a few years ago, various people were exploring locality sensitive hashing as an alternative to neural networks(SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems [1]). This never took off. I don't if it didn't work or it was a matter of the "bitter lesson" ideology pushing people to prefer hardware acceleration over smart algorithms. I would note that smart algorithms require hiring (more)smart people and not only can hardware acceleration be cheaper but hardware and data can be more reliably added than smart people, who are short…

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — HN submission

Sat, 04 Jan 2025 04:25:14 GMT

In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale AI

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by rajnathani

Sun, 13 Feb 2022 11:28:23 GMT

> Sparse operations still don’t have good support in any of the generic SIMD instructions (avx2/512, Neon), or in GPU… This is incorrect. For GPUs: Nvidia A100 (launched in 2020) has sparse matrix multiplication support for inference. For CPUs: Check out SLIDE from 2019: https://arxiv.org/pdf/1903.03129.pdf

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by outlace

Tue, 05 Oct 2021 23:36:19 GMT

Implementing this paper < https://arxiv.org/abs/1903.03129 > for a computer vision application.

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by rajnathani

Mon, 23 Aug 2021 07:00:55 GMT

Sparse neural networks can be optimized by implementing them using Locality Sensitive Hashing (2019 paper): https://arxiv.org/pdf/1903.03129.pdf Fun fact, last week one of the main authors of the above 2019 paper spun off a company for the above approach: https://techcrunch.com/2021/08/13/thirdai-raises-6m-to-democ...

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by guywhocodes

Fri, 09 Apr 2021 00:39:12 GMT

It's not the first similar claim, there was one that large CPU's can beat GPUs by creating a very sparse matrix using spatial hashing. Preserving the gradient but finding it with vastly less computation by simply not multiplying elements that don't have much or any impact. EDIT: orginial work afaik https://arxiv.org/abs/1903.03129

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by exegete

Fri, 09 Apr 2021 00:33:22 GMT

Paper: https://arxiv.org/abs/1903.03129 Code: https://github.com/keroro824/HashingDeepLearning Edit: Here is the new paper referred to https://arxiv.org/abs/2103.10891 Here is the new code: https://github.com/RUSH-LAB/SLIDE The paper I linked above was the first paper of which this is supposed to be an improvement.

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by exegete

Fri, 09 Apr 2021 00:31:50 GMT

https://arxiv.org/abs/1903.03129 https://github.com/keroro824/HashingDeepLearning

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by mlthoughts2018

Sun, 20 Dec 2020 18:19:00 GMT

I found it very weird that the SLIDE algorithm from early 2019 isn’t mentioned. Maybe I missed it or maybe it is compared just deeper in the referenced publications? SLIDE seems way, way superior to any of the listed solutions or approaches, as far as I could tell on a first read through. https://arxiv.org/abs/1903.03129

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by cztomsik

Sat, 07 Mar 2020 08:07:38 GMT

No idea, it was probably sponsored by them but the concept should be the same for AMD and others too. EDIT: here's a bit more context (they've been contacted by Intel after publishing initial PoC) source: https://news.rice.edu/2020/03/02/deep-learning-rethink-overc... paper: https://arxiv.org/pdf/1903.03129.pdf repo: https://github.com/keroro824/HashingDeepLearning

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by signa11

Fri, 06 Mar 2020 10:29:31 GMT

actual paper: https://arxiv.org/abs/1903.03129

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — HN submission

Fri, 06 Mar 2020 09:19:27 GMT

Smart Algorithms beat Hardware Acceleration for Large-Scale Deep Learning

355 points; 135 discussion replies on HN. Paper and source history. A submission is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — HN submission

Thu, 05 Mar 2020 09:04:48 GMT

Slide: In Defense of Smart Algorithms over Hardware Accelerated Deep Learning

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — mentioned by yayr

Wed, 04 Mar 2020 20:01:07 GMT

source: https://github.com/keroro824/HashingDeepLearning paper: https://arxiv.org/abs/1903.03129

Paper and discussion history. A mention is not an endorsement.

SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems — HN submission

Mon, 11 Mar 2019 08:34:43 GMT

In Defense of Smart Algorithms over Hardware Acceleration for Deep Learning

18 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.