GoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2407.12077/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTGoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression — mentioned by pico_creatorhttps://news.ycombinator.com/item?id=42572451urn:hackerpapers:mention:2407.12077:42572451Thu, 02 Jan 2025 07:19:28 GMT<p>Not an MoE, but we have already done hybrid models. And found it to be highly performant (as per the training budget) https://arxiv.org/abs/2407.12077</p><p><a href="https://hackerpaper.kaipfister.de/p/2407.12077/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL