QTIP: Quantization with Trellises and Incoherence Processing — mentioned by acd

Thu, 10 Sep 2026 14:12:52 GMT

glq is a re implementation gpl port of QTIP Trellis encoding of large language models as vLLM plugin. glq is efficient between 2-4 bits per word where it has an edge over scalar quantization. Fit larger LLMs on consumer hardware GPUs. glq extents QTIP up to 8 bits per word by running the original algorithm two times with a residual. 1.Incoherence processing — two-sided randomized Hadamard transform (via FWHT) makes weights ~i.i.d. Gaussian 2. BlockLDLQ rounding — sequential sweep with Hessian error feedback 3. Viterbi encoding — bitshift trellis + computed Gaussian code 4. Decode — stateless…

Paper and discussion history. A mention is not an endorsement.

QTIP: Quantization with Trellises and Incoherence Processing · HackerPapers discussions

QTIP: Quantization with Trellises and Incoherence Processing — mentioned by acd