Generalization through Memorization: Nearest Neighbor Language Models — mentioned by jpfed

Mon, 28 Aug 2023 16:24:09 GMT

The most promising work along these lines centers around augmenting LLMs with an external data store ("retrieval-augmented LLM"s). I think this started with Facebook's KNN-LLM ( https://arxiv.org/pdf/1911.00172.pdf ). Legal conflict may force the industry to move towards vector DBs as the predominant method by which facts are "stored" rather than model parameters ( https://arxiv.org/pdf/2308.04430.pdf ) , with the happy side effect of update-ability over time.

Paper and discussion history. A mention is not an endorsement.

Generalization through Memorization: Nearest Neighbor Language Models · HackerPapers discussions

Generalization through Memorization: Nearest Neighbor Language Models — mentioned by jpfed