Relating transformers to models and neural representations of the hippocampal formation · HackerPapers discussions

Relating transformers to models and neural representations of the hippocampal formation — mentioned by triclops200

Mon, 03 Nov 2025 20:05:13 GMT

As a researcher in these fields: this reasoning is tired, overblown, and just wrong. We have a lot of understanding of how the brain works overall. You don't. Go read the active inference book by Friston et. al. for some of the epistemological and behavioral mechanics (Yes, this applies to llms as well, they easily satisfy the requirements to be considered the mathematical object described as a markov blanket). And, yes, if you could somehow freeze a human's current physical configuration at some time, you would absolutely, in principle, given what we know about the universe, be able to…

Paper and discussion history. A mention is not an endorsement.

Relating transformers to models and neural representations of the hippocampal formation — mentioned by triclops200

Thu, 12 Sep 2024 15:50:30 GMT

The (semi) automatic simplification algorithm provided in the paper for KANs seem, to me, like they're solving a similar problem to https://arxiv.org/pdf/2112.04035, but with the additional constraint of forward functional interpretability as the goal instead of just a generalized abstraction compressor.

Paper and discussion history. A mention is not an endorsement.

Relating transformers to models and neural representations of the hippocampal formation — mentioned by soraki_soladead

Sun, 16 Jun 2024 14:12:50 GMT

You might enjoy this paper[0] which shows that recurrent position encodings recover grid cell representations and maps to path integration found in a popular model of the hippocampus. This isn't terribly surprising since RNNs have shown this before[1, 2] but its an interesting connection. [0] https://arxiv.org/abs/2112.04035 [1] https://arxiv.org/abs/1803.07770 [2] https://www.nature.com/articles/s41586-018-0102-6

Paper and discussion history. A mention is not an endorsement.

Relating transformers to models and neural representations of the hippocampal formation — mentioned by utopcell

Wed, 03 Jan 2024 23:34:48 GMT

Here [1] are some "frauds" from Stanford University, Oxford University and University College London telling you exactly that. From their abstract: ``One of the most exciting and promising novel architectures, the Transformer neural network, was developed without the brain in mind. In this work, we show that transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably place and grid cells. Furthermore, we show that this result is no surprise since it is closely related to current hippocampal…

Paper and discussion history. A mention is not an endorsement.

Relating transformers to models and neural representations of the hippocampal formation — mentioned by macrolime

Sat, 10 Dec 2022 22:50:18 GMT

The neuromatch computational neuroscience course also seems quite interesting, though maybe less of practical use. https://compneuro.neuromatch.io/ Recent research like "Relating transformers to models and neural representations of the hippocampal formation" might make it more relevant though (https://arxiv.org/abs/2112.04035v2) quote from the abstract of that paper: "Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the…

Paper and discussion history. A mention is not an endorsement.

Relating transformers to models and neural representations of the hippocampal formation — HN submission

Mon, 23 May 2022 20:20:21 GMT

Relating transformers to neural representations of hippocampal formation

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Relating transformers to models and neural representations of the hippocampal formation — mentioned by gfodor

Wed, 13 Apr 2022 03:48:23 GMT

That’s just begging the question. We don’t know if DALL-E is creative in ways isomorphic to humans. It might be. How would one go about structuring a hypothesis? Have you read the original DALL-E paper? Worth considering: https://arxiv.org/abs/2112.04035

Paper and discussion history. A mention is not an endorsement.

Relating transformers to models and neural representations of the hippocampal formation — HN submission

Tue, 12 Apr 2022 17:24:40 GMT

Relating Transformers to Neural Representations of the Hippocampal Formation

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Relating transformers to models and neural representations of the hippocampal formation — HN submission

Mon, 11 Apr 2022 22:50:57 GMT

Relating Transformers to Modes and Neural Representations of the Hippocampal

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.