Dynamic Chunking for End-to-End Hierarchical Sequence Modeling · HackerPapers discussions

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — mentioned by psb217

Wed, 22 Oct 2025 09:34:28 GMT

Yes, you can get good compression of a long sequence of "base" text tokens into a shorter sequence of "meta" text tokens, where each meta token represents the information from multiple base tokens. But, grouping a fixed number of base tokens into each meta token isn't ideal, since that won't align neatly with sensible semantic boundaries, like words, phrases, sentences, etc. So, the trick is how decide which base tokens should be grouped into each meta token.... This sort of "dynamic chunking" of low-level information, perhaps down to the level of raw bytes, into shorter sequences of meta…

Paper and discussion history. A mention is not an endorsement.

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — mentioned by cubefox

Fri, 10 Oct 2025 19:53:31 GMT

This sounds a bit like H-Net [1] or Byte Latent Transformer [2]. 1: https://arxiv.org/abs/2507.07955 2: https://arxiv.org/abs/2412.09871

Paper and discussion history. A mention is not an endorsement.

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — mentioned by spmurrayzzz

Tue, 29 Jul 2025 14:24:02 GMT

Do you have a citation for the paper on that? IME, that's not really something you see used in practice, at least not after 2022 or so. Without some form positional adjustment, transformer-based LLMs have no way to differentiate from "The dog bit the man." and "The man bit the dog." given the token ids are nearly identical. You just end up back in the bag-of-words problem space. The self-attention mechanism is permutation-invariant, so as long as it remains true that the attention scores are computed as an unordered set, you need some way to model the sequence. Long context is almost always…

Paper and discussion history. A mention is not an endorsement.

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — HN submission

Sun, 27 Jul 2025 21:27:53 GMT

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — HN submission

Thu, 24 Jul 2025 01:01:21 GMT

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — mentioned by lukebechtel

Tue, 15 Jul 2025 20:38:56 GMT

https://arxiv.org/pdf/2507.07955 paper

Paper and discussion history. A mention is not an endorsement.

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — HN submission

Sun, 13 Jul 2025 06:37:48 GMT

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling

10 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — HN submission

Sat, 12 Jul 2025 11:54:21 GMT

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling — HN submission

Fri, 11 Jul 2025 18:36:28 GMT

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.