The Platonic Representation Hypothesis

Thank you for expanding! I come from more of an ML background so still learning a lot on these topics: Agreed that the neocortex uses fewer layers because of looping - I also suspect it's partly because neurons are more complex than the neurons used in ANNs, so in principle they…

ainchJul 7, 2026View on HNParent context

Platonic representation hypothesis? https://arxiv.org/abs/2405.07987 Maybe in the same way fourier/wavelet basis is just the most natural way to work with certain signals or images, there's a certain representation for language and a representation for "thinking" that's natural.…

krackersMar 28, 2026View on HNParent context

Fair push back, but I do think the LSTM vs Transformers point kinda supports my position in the limit, not refutes. Once the compute bottleneck is removed, LSTMs scale favourably. https://arxiv.org/pdf/2510.02228 (I believe there's similar work done on vanilla LSTMs, but I'd…

jpcompartirMar 24, 2026View on HNParent context

https://arxiv.org/abs/2505.12540 https://arxiv.org/abs/2405.07987 These, other papers, and the lottery ticket phenomenon; what it boils down to is that any neural network like system which encodes some common mapping of a phenomenon in the context of the world - not necessarily…

observationistMar 19, 2026View on HNParent context

>> I would argue this is deeply false I am making the case that this is distinctly and specifically true, for these types of models. They're eliciting many of the underlying functions and processes that brought about the data; transformers are able to model the higher degrees of…

observationistMar 2, 2026View on HNParent context

> They're going to diverge if the underlying manner in which data gets memorized and encoded, such as with RNNs, like RWKV. In the original paper (https://arxiv.org/abs/2405.07987) the authors also compared the representations of transformer-based LLMs to convolution-based image…

in-silicoJan 8, 2026View on HNParent context

Same hat, except 18 months later, assuming it survives peer review, reproduction, etc. (or: "The newer one proposes evidence that appears to support the older one.") https://arxiv.org/abs/2405.07987

altairprimeDec 9, 2025View on HNParent context

Also related: https://arxiv.org/abs/2405.07987 As a resident Max Stirner fan, the idea that platonism is physically present in reality and provably correct is upsetting indeed.

Der_EinzigeAug 15, 2025View on HNParent context

Apologies if this comes across as too abstract, but I think your comment raises really important questions. (1) While studying the properties of the mathematical objects produced is important, I don't think we should understand the situation you describe as a problem to be…

agentcoopsJul 28, 2025View on HNParent context

This is called the The Platonic Representation Hypothesis https://arxiv.org/abs/2405.07987 We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multiple…

UltraSaneJul 19, 2025View on HNParent context

Very fair! > Does this extend to being able to analytically determine which concepts are encodable in one embedding but not another? An embedding from a deft tiny stories LLM presumably cannot encode concepts about RNA replication. Yeah, this is a great point. We're mostly…

jxmorris12May 21, 2025View on HNParent context

>I am not a neuroscientist, but I think it's likely that LLMs (with 10s of billions of parameters) and the human brain (with 1-2 orders of magnitude more neural connections[1]) process language in analogous ways Related is the platonic representation hypothesis where models…

sam_ezehMar 4, 2025View on HNParent context

In fact, it looks like the model is doing those things internally. We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multiple domains, the ways by which…

thomashopOct 31, 2024View on HNParent context

We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multiple domains, the ways by which different neural networks represent data are becoming more aligned.…

thomashopOct 31, 2024View on HNParent context

The use of this sort of anthropomorphic and "incantation" style prompting is a workaround while mechanistic interpretability and monosemanticity work[1] is done to expose the neuron(s) that have larger impacts on model behavior -- cf Golden Gate Claude. Further, even if…

TastyDucksMay 26, 2024View on HNParent context

Neural network representation spaces seem to converge, regardless of architecture: https://arxiv.org/abs/2405.07987 It would make sense for the human mental latent spaces to also converge. The reason is that the latent space exists to model the environment, which is largely…

canjobearMay 22, 2024View on HNParent context

Thanks! Yeah, it seems like a lot can be done with just text while we wait for multimodal models to catch up. The recent Platonic Representation Hypothesis [1] also suggests that different models, regardless of modality, build the same internal representations of the world. [1]…

KhoomeiKMay 15, 2024View on HNParent context