ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models — HN submission

Thu, 20 Feb 2025 14:05:59 GMT

ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models(2023)

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models — mentioned by keivanalizadeh

Wed, 20 Dec 2023 17:01:42 GMT

By the time we put our paper on arxiv this paper was not out so we were not aware of, but it is similar in some ways. Both this paper and us are reliant on our previous paper https://arxiv.org/abs/2310.04564 and dejavu https://arxiv.org/abs/2310.17157. They are targeting limited gpu memory and limit cpu to gpu memory transfer. I don't know how it could be useful on Macs because MacBooks have a unified memory and you don't need to do that transfer necessarily.

Paper and discussion history. A mention is not an endorsement.

ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models · HackerPapers discussions

ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models — HN submission

ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models — mentioned by keivanalizadeh