Perceiver: General Perception with Iterative Attention · HackerPapers discussions

Perceiver: General Perception with Iterative Attention — mentioned by visarga

Thu, 23 Feb 2023 05:11:44 GMT

Well, it depends. Perceiver can handle big context. https://arxiv.org/abs/2103.03206

Paper and discussion history. A mention is not an endorsement.

Perceiver: General Perception with Iterative Attention — mentioned by visarga

Sat, 07 Jan 2023 20:11:53 GMT

Maybe you are interested in this paper: > Perceiver: General Perception with Iterative Attention Biological systems perceive the world by simultaneously processing high dimensional inputs from modalities as diverse as vision, audition, touch, proprioception, etc. Perceiver is a deep learning model that can process multiple modalities, such as images, point clouds, audio, and video, simultaneously. It is based on the transformer architecture and uses an asymmetric attention mechanism to distill a large number of inputs into a smaller latent bottleneck. This allows it to scale to handle very…

Paper and discussion history. A mention is not an endorsement.

Perceiver: General Perception with Iterative Attention — mentioned by gwern

Sun, 28 Nov 2021 18:04:17 GMT

I think if you wanted to do this at scale, you would prefer Perceiver-IO ( https://arxiv.org/abs/2103.03206#deepmind https://arxiv.org/abs/2107.14795#deepmind ).

Paper and discussion history. A mention is not an endorsement.

Perceiver: General Perception with Iterative Attention — mentioned by dunefox

Mon, 13 Sep 2021 15:20:13 GMT

The 'general perceiver' is such a model that is somewhat distanced from the modality of its input: https://arxiv.org/abs/2103.03206

Paper and discussion history. A mention is not an endorsement.