Emergence of grid-like representations by training recurrent neural networks to perform spatial localization · HackerPapers discussions

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization — mentioned by soraki_soladead

Sun, 16 Jun 2024 14:12:50 GMT

You might enjoy this paper[0] which shows that recurrent position encodings recover grid cell representations and maps to path integration found in a popular model of the hippocampus. This isn't terribly surprising since RNNs have shown this before[1, 2] but its an interesting connection. [0] https://arxiv.org/abs/2112.04035 [1] https://arxiv.org/abs/1803.07770 [2] https://www.nature.com/articles/s41586-018-0102-6

Paper and discussion history. A mention is not an endorsement.

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization — mentioned by ad404b8a372f2b9

Sat, 02 Apr 2022 18:00:10 GMT

Neuroscience is not my specialty, I only understand the ML side of it. My understanding is that when our machine learning architectures are specially crafted to model specific types of data, we observe similar neurons as those in the brain because they converge to the same optimal solution to handle this data. For example, taking into account the translational invariance of image data with convolutional layers leads to similar neurons as the V1 cortex. Training recurrent layers on navigation tasks lead to grid-cells similar to those in the brain. Some references…

Paper and discussion history. A mention is not an endorsement.

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization — mentioned by jimfleming

Mon, 11 Feb 2019 20:14:18 GMT

In addition, multiple works[0][1] have discovered that grid cell representations arise from regularized recurrent networks when provided relative inputs to predict absolute outputs. [0] https://arxiv.org/abs/1803.07770 [1] https://deepmind.com/blog/grid-cells/

Paper and discussion history. A mention is not an endorsement.

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization — mentioned by baylearn

Sat, 12 May 2018 07:28:27 GMT

This DeepMind work has been gathering a lot of hype, but it is very similar to this work by Cueva and Wei: Published on Arxiv on March 31: https://arxiv.org/abs/1803.07770 Published at ICLR 2018, a well-known machine learning conference: https://openreview.net/forum?id=B17JTOe0- It is rather disheartening that they didn't acknowledge the -- seems like a deliberate oversight. The Cueva and Wei paper is well written, explains concepts clearly, without the hype, and published in an open access venue.

Paper and discussion history. A mention is not an endorsement.

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization — mentioned by jimfleming

Wed, 09 May 2018 21:49:37 GMT

I think you're referring to this paper: "Emergence of grid-like representations by training recurrent neural networks to perform spatial localization". https://arxiv.org/abs/1803.07770 It appears to be from Columbia vs DeepMind with different authors.

Paper and discussion history. A mention is not an endorsement.