Augmenting Language Models with Long-Term Memory — mentioned by filterfiber

Thu, 30 Nov 2023 17:54:42 GMT

Yeah that's a fairly well studied one. Most of these techniques are rather "lossy" compared to extending the context window. The most likely "real solution" is going to be using various tricks and finetuning on higher context lengths to just extend the context window. Here's a bunch of other related methods, Summarizing context - https://arxiv.org/abs/2305.14239 continuous finetuning - https://arxiv.org/pdf/2307.02839.pdf retrieval augmented generation - https://arxiv.org/abs/2005.11401 knowledge graphs - https://arxiv.org/abs/2306.08302 augmenting the network a side network…

Paper and discussion history. A mention is not an endorsement.

Augmenting Language Models with Long-Term Memory — HN submission

Tue, 13 Jun 2023 19:52:07 GMT

Augmenting Language Models with Long-Term Memory

32 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Augmenting Language Models with Long-Term Memory · HackerPapers discussions

Augmenting Language Models with Long-Term Memory — mentioned by filterfiber

Augmenting Language Models with Long-Term Memory — HN submission