LM2: Large Memory Models · HackerPapers discussions

LM2: Large Memory Models — mentioned by viraptor

Sat, 15 Feb 2025 12:06:36 GMT

https://arxiv.org/abs/2502.06049 But for most practical use cases we don't want the persistent memory. We already have opportunistic fact extraction like chatgpt profile building if we want that. We also have graph builders. This mostly covers the private usage. Public version was attempted and just ends up like this: https://en.wikipedia.org/wiki/Tay_(chatbot)

Paper and discussion history. A mention is not an endorsement.

LM2: Large Memory Models — mentioned by jmward01

Fri, 14 Feb 2025 23:52:24 GMT

I have an internal repo that does guided window attn. I figured out that One Weird Trick to get the model to learn how to focus so that you can move a fixed window around instead of full attn. I also built NNMemory (but that appears to be an idea others hae had now too [1]) and I have a completely bonkers mechanism for non-determanistic exit logic so that the model can spin until it thinks it has a good answer. I also built scale free connections between layers to completely remove residual connections. Plus some crazy things on sacrificial training (adding parameters that are removed after…

Paper and discussion history. A mention is not an endorsement.

LM2: Large Memory Models — HN submission

Thu, 13 Feb 2025 23:21:21 GMT

LM2: Large Memory Models

110 points; 30 discussion replies on HN. Paper and source history. A submission is not an endorsement.