A Closer Look at the Limitations of Instruction Tuning · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2402.05119/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTA Closer Look at the Limitations of Instruction Tuning — mentioned by DanyWinhttps://news.ycombinator.com/item?id=39699670urn:hackerpapers:mention:2402.05119:39699670Thu, 14 Mar 2024 01:25:27 GMT<p>You are exactly right! As I wanted to have a solution that works with many LLMs out of the box, I focused on chain of thoughts and few shot learnings. Lots of paper show that fine-tuning only helps with steerability and form (https://arxiv.org/abs/2402.05119), therefore I thought it would be sufficient to provide just the right examples and it did work! We do intend to create a decentralized dataset to further train models and have maybe a 2b or 7b model working well</p><p><a href="https://hackerpaper.kaipfister.de/p/2402.05119/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL