Sparse Fine-tuning for Inference Acceleration of Large Language Models — mentioned by jillbore

Fri, 20 Oct 2023 16:45:31 GMT

That is the accuracy of the non-finetuned model i.e. mpt-7b. Once finetuned, it can get around 30% accuracy on 0-shot. Not sure what 8-shot performance is for the finetuned model. https://arxiv.org/pdf/2310.06927.pdf

Paper and discussion history. A mention is not an endorsement.

Sparse Fine-tuning for Inference Acceleration of Large Language Models — HN submission

Tue, 17 Oct 2023 13:09:26 GMT

8x Acceleration for LLM Inference on CPUs

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Sparse Fine-tuning for Inference Acceleration of Large Language Models · HackerPapers discussions

Sparse Fine-tuning for Inference Acceleration of Large Language Models — mentioned by jillbore

Sparse Fine-tuning for Inference Acceleration of Large Language Models — HN submission