It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs — mentioned by mmargenot

Wed, 14 Oct 2026 17:19:17 GMT

LLMs can be pre-trained on LLM outputs, and there have been a few recent interesting implementations of this. - https://arxiv.org/abs/2609.37891 - https://arxiv.org/pdf/2508.10975 - https://www.datologyai.com/blog/eating-the-rainbow-a-primer-... As well as recent model releases that used large fractions of synthetic data (Arcee Trinity): - https://arxiv.org/pdf/2602.17004v1 It's mostly a matter of increasingly complex and sophisticated sampling methodologies to sift for the valuable synthetic tokens. There have also been some interesting results about models that use the same data converging…

Paper and discussion history. A mention is not an endorsement.

It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs · HackerPapers discussions

It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs — mentioned by mmargenot