Reformer: The Efficient Transformer · HackerPapers discussions

Reformer: The Efficient Transformer — mentioned by in-silico

Wed, 06 May 2026 05:32:27 GMT

I wonder how different their method actually is from other sub-quadratic sparse attention methods like Reformer [1] and Routing Transformer [2]. [1]: https://arxiv.org/abs/2001.04451 [2]: https://arxiv.org/abs/2003.05997

Paper and discussion history. A mention is not an endorsement.

Reformer: The Efficient Transformer — mentioned by sinenomine

Sun, 11 Oct 2020 18:10:26 GMT

One can use this to express a large part of a deep ML model as a reversible function, thus drastically lowering memory requirements for backpropagation. The same technique, but applied by hand was recently used to speed up training for transformer language model: https://arxiv.org/abs/2001.04451 I think in the future we will see a trend of expressing most of DL model as reversible computation, with minimal irreversible module in the end and in the beginning.

Paper and discussion history. A mention is not an endorsement.

Reformer: The Efficient Transformer — mentioned by alevskaya

Fri, 31 Jan 2020 07:28:57 GMT

It's still early days for JAX, but there's neural tangents https://arxiv.org/abs/1912.02803 and reformer https://arxiv.org/abs/2001.04451 from iclr.

Paper and discussion history. A mention is not an endorsement.

Reformer: The Efficient Transformer — HN submission

Sun, 19 Jan 2020 11:57:01 GMT

Reformer: The Efficient Transformer

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.