Source-Aware Training Enables Knowledge Attribution in Language Models · HackerPapers discussions

Source-Aware Training Enables Knowledge Attribution in Language Models — mentioned by DoctorOetker

Sun, 18 Oct 2026 17:32:45 GMT

@DeepSeek teams, why not train your next series of LLM models with source-aware training? https://arxiv.org/abs/2404.01019 LLM's would be able to answer questions, delineating exactly which facts/assumptions/reasonings are prior work and from where (this URL, that book, etc.), and which parts are it's own deductions, or the user's.

Paper and discussion history. A mention is not an endorsement.

Source-Aware Training Enables Knowledge Attribution in Language Models — mentioned by DoctorOetker

Mon, 28 Sep 2026 13:55:19 GMT

I don't like the term 'hallucination' to be honest, not because it anthropomorphizes, but because it lacks a formal definition in the context of machine learning. Suppose parents tell their children that there exists this man called "Santa Claus" who comes down the chimney to deliver presents. Now consider a scientist talking to this child, should the scientist call these confidently expressed beliefs surrounding "Santa Claus" hallucinations ? I don't think so, most would call the epistemological behavior of the child naive (because it blindly believes what its parents say, without direct…

Paper and discussion history. A mention is not an endorsement.

Source-Aware Training Enables Knowledge Attribution in Language Models — mentioned by DoctorOetker

Mon, 11 May 2026 21:01:33 GMT

In this case I think you'd want to use Source-Aware-Training [0] to associate a "timestamp" vector to each native context chunk (perhaps overlapping) of conversation, probably the weights using a kind of Gray code so that the LLM has the immediate out-of-native context history can be retrieved through the nearby gray code of 1, 2, etc steps ago compared to the current timestep gray code. https://arxiv.org/abs/2404.01019

Paper and discussion history. A mention is not an endorsement.

Source-Aware Training Enables Knowledge Attribution in Language Models — mentioned by DoctorOetker

Thu, 12 Feb 2026 06:19:27 GMT

To a large extent both "hallucinations" and "plagiarism" can be addressed with the same training method: source-aware training. https://arxiv.org/abs/2404.01019 At the frontier of science we have speculations, which until proper measurements become possible, are unknown to be true or false (or even unknown to be equivalent with other speculations etc. regardless of their being true or false, or truer or falser). Once settled we may call earlier but wrong speculations as "reasonable wrong guesses". In science it is important that these guesses or suspicions are communicated as it drives the…

Paper and discussion history. A mention is not an endorsement.

Source-Aware Training Enables Knowledge Attribution in Language Models — mentioned by DoctorOetker

Mon, 08 Dec 2025 13:31:19 GMT

Less than 800 words, but more if you follow the link :) https://arxiv.org/abs/2404.01019 "Source-Aware Training Enables Knowledge Attribution in Language Models"

Paper and discussion history. A mention is not an endorsement.

Source-Aware Training Enables Knowledge Attribution in Language Models — mentioned by DoctorOetker

Mon, 22 Sep 2025 03:09:53 GMT

> The article you shared raises an interesting point by comparing human memory with LLMs, but I think the analogy can only go so far. The analogy may be much less wishy-washy than you imagine: check out "Source-aware training" https://arxiv.org/abs/2404.01019 Everybody quickly learns to take note of who-claims-what: in kindergarten kids learn to detect each others lies, and learn to attach more credibility to the teachers / caretakers than dubious claims "I am allowed to hit you", to the point that precisely because they learn it they start crafting insidious lies "the teacher said I'm…

Paper and discussion history. A mention is not an endorsement.

Source-Aware Training Enables Knowledge Attribution in Language Models — mentioned by numeri

Fri, 20 Sep 2024 16:59:19 GMT

Sort of like this? It does help: Source-Aware Training Enables Knowledge Attribution in Language Models (https://arxiv.org/abs/2404.01019) From the abstract: > ... To give LLMs such ability, we explore source-aware training -- a recipe that involves (i) training the LLM to associate unique source document identifiers with the knowledge in each document, followed by (ii) an instruction-tuning stage to teach the LLM to cite a supporting pretraining source when prompted.

Paper and discussion history. A mention is not an endorsement.

Source-Aware Training Enables Knowledge Attribution in Language Models — HN submission

Thu, 18 Jul 2024 16:00:29 GMT

Training LLMs to cite the pretraining data

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.