LLM-Pruner: On the Structural Pruning of Large Language Models — mentioned by renonce

Thu, 18 Apr 2024 04:50:03 GMT

Essentially the algorithm is about pruning parameters on the fly and making the weight matrix sparse by setting least significant weights to zero, where "least significant" is defined in terms of the rank of the absolute value of the pruned weight within its group? A search for model pruning turns out many results, including https://arxiv.org/abs/2305.11627 which discusses "magnitude pruning" as a baseline, and refers to https://arxiv.org/pdf/2301.00774.pdf, which asserts in the introduction: > First, as shown in Figure 1, SparseGPT can induce uniform layerwise sparsity of up to 60% in e.g.…

Paper and discussion history. A mention is not an endorsement.

LLM-Pruner: On the Structural Pruning of Large Language Models · HackerPapers discussions

LLM-Pruner: On the Structural Pruning of Large Language Models — mentioned by renonce