RAFT: Adapting Language Model to Domain Specific RAG — mentioned by gopalv

Wed, 28 Oct 2026 21:31:53 GMT

> On-policy distillation: The expanded student model learns token-specific feedback from teacher distributions generated on self-sampled sequences. The weird thing is to see this based off the KV-Cache in the diagrams. Completely different from the Databricks RAFT [1]. My team built "Dreamtime" earlier this year [2], which used a different mechanism to distill out learnings across multiple sessions. This didn't pull from the token layer, but from user nudges on unsatisfactory responses. This works like a sawtooth wave where we accumulate context, check for dissonances (a user gets to say "It…

Paper and discussion history. A mention is not an endorsement.

RAFT: Adapting Language Model to Domain Specific RAG — HN submission

Mon, 18 Mar 2024 15:12:18 GMT

Raft: Adapting Language Model to Domain Specific RAG

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

RAFT: Adapting Language Model to Domain Specific RAG · HackerPapers discussions

RAFT: Adapting Language Model to Domain Specific RAG — mentioned by gopalv

RAFT: Adapting Language Model to Domain Specific RAG — HN submission