AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression — mentioned by yowlingcat

Thu, 17 Sep 2026 23:56:52 GMT

That's correct. I think there can certainly be issues even with 4bpw with naive quantization (IE you'll notice far better results from a QAT 4bpw vs a naive 4bpw). One such method that I've been meaning to look into further is Tencent's AngelSlim QAT/PTQ approach. They did a Hy4 preview release thats an STQ_1_0 at 2.38 bpw: https://huggingface.co/AngelSlim/Hy4-preview-GGUF https://arxiv.org/abs/2602.21233 Of course, it's still 213g of VRAM I'd need so it's somewhat out of the range of what I can run locally. In contrast, this new Bonsai is nice because the original was already exciting for…

Paper and discussion history. A mention is not an endorsement.

AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression · HackerPapers discussions

AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression — mentioned by yowlingcat