Training Language Models on Synthetic Edit Sequences Improves Code Synthesis — mentioned by twotwotwo

Thu, 01 May 2025 02:05:55 GMT

It's kind of weird to think that in a coding assistant, an LLM is regularly asked to produce a valid block of code top to bottom, or repeat a section of code with changes, when that's not what we do. (There are other intuitively odd things about this, like the amount of compute spent generating 'easy' tokens, e.g. repeating unchanged code.) Some of that might be that models are just weird and intuition doesn't apply. But maybe the way we do it--jumping around, correcting as we go, etc.--is legitimately an efficient use of effort, and a model could do its job better, with less effort, or both…

Paper and discussion history. A mention is not an endorsement.

Training Language Models on Synthetic Edit Sequences Improves Code Synthesis · HackerPapers discussions

Training Language Models on Synthetic Edit Sequences Improves Code Synthesis — mentioned by twotwotwo