Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time — mentioned by keivanalizadeh

Wed, 20 Dec 2023 17:01:42 GMT

By the time we put our paper on arxiv this paper was not out so we were not aware of, but it is similar in some ways. Both this paper and us are reliant on our previous paper https://arxiv.org/abs/2310.04564 and dejavu https://arxiv.org/abs/2310.17157. They are targeting limited gpu memory and limit cpu to gpu memory transfer. I don't know how it could be useful on Macs because MacBooks have a unified memory and you don't need to do that transfer necessarily.

Paper and discussion history. A mention is not an endorsement.

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time · HackerPapers discussions

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time — mentioned by keivanalizadeh