Memory Transformer — mentioned by thomasahle

Fri, 02 Jun 2023 15:11:58 GMT

Have you tried something like Memory Transformers https://arxiv.org/abs/2006.11527 where you move the k/v pairs that don't fit in the context window to a vector db? Seems like a more general approach, but I have tested then against each other.

Paper and discussion history. A mention is not an endorsement.

Memory Transformer · HackerPapers discussions

Memory Transformer — mentioned by thomasahle