Hungry Hungry Hippos: Towards Language Modeling with State Space Models · HackerPapers discussions

Hungry Hungry Hippos: Towards Language Modeling with State Space Models — mentioned by jwilber

Tue, 29 Apr 2025 19:45:06 GMT

LLM/state space models have been popular for some years now, see: https://arxiv.org/abs/2212.14052 More recently, hybrid architectures that utilize attention plus other operators are gaining traction. See https://arxiv.org/abs/2503.01868

Paper and discussion history. A mention is not an endorsement.

Hungry Hungry Hippos: Towards Language Modeling with State Space Models — mentioned by throwaway4aday

Sun, 05 Mar 2023 13:34:52 GMT

Exactly, we're seeing the time horizon shrink rapidly. There have already been numerous threads ruminating on the fact that even a few months of not paying attention will result in you lagging behind in this technology. I can't wait to see what happens when state space models are developed further[0]. If the prompt and output length can be increased significantly then large chunks of the work of building software could be offloaded to these tools. [0] https://arxiv.org/abs/2212.14052

Paper and discussion history. A mention is not an endorsement.

Hungry Hungry Hippos: Towards Language Modeling with State Space Models — mentioned by dm3

Wed, 01 Feb 2023 09:21:15 GMT

People are already making progress on this, e.g. the H3 project[1]. [1] https://arxiv.org/abs/2212.14052

Paper and discussion history. A mention is not an endorsement.

Hungry Hungry Hippos: Towards Language Modeling with State Space Models — HN submission

Wed, 25 Jan 2023 23:12:56 GMT

Towards Language Modeling With State Space Models

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Hungry Hungry Hippos: Towards Language Modeling with State Space Models — HN submission

Tue, 24 Jan 2023 15:42:15 GMT

Hungry Hungry Hippos: Towards Language Modeling with State Space Models

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.