INT2.1: Towards Fine-Tunable Quantized Large Language Models with Error Correction through Low-Rank Adaptation — mentioned by woadwarrior01

Wed, 13 Sep 2023 15:17:35 GMT

This paper[1] attempts to do LoRA fine tuning on quantised LLMs. The LoRa weights themselves are full precision, but they're much smaller. Although, I haven't had a chance to try it yet. [1]: https://arxiv.org/abs/2306.08162

Paper and discussion history. A mention is not an endorsement.

INT2.1: Towards Fine-Tunable Quantized Large Language Models with Error Correction through Low-Rank Adaptation · HackerPapers discussions

INT2.1: Towards Fine-Tunable Quantized Large Language Models with Error Correction through Low-Rank Adaptation — mentioned by woadwarrior01