On the Turing Completeness of Modern Neural Network Architectures · HackerPapers discussions

On the Turing Completeness of Modern Neural Network Architectures — mentioned by rlupi

Tue, 04 Feb 2025 16:36:10 GMT

> No AI can code in Lean 4 reliably I wonder if this is due to the nature of the language. Lean 4 lets you redefine its syntax in ways that most other languages do not allow[1], so effectively you are dealing with a recursive language, that could require a Touring complete token representation system. [1] https://leanprover-community.github.io/lean4-metaprogramming... What other language lets you redefine the meaning of digits? The mix of syntax + macros + elaboration makes it really flexible, but hard to treat reliably. LLMs based on transformers are not Touring complete (nitpick: they are…

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by benzible

Mon, 23 Sep 2024 07:05:36 GMT

It's gibberish. For one thing... https://arxiv.org/abs/1901.03429

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by gwern

Wed, 14 Jun 2023 21:43:59 GMT

2019 version which includes the Neural GPU: https://arxiv.org/abs/1901.03429

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by MacsHeadroom

Sat, 22 Apr 2023 19:51:04 GMT

Transformers are turing complete: https://arxiv.org/abs/1901.03429 Additionally, Transformers are RNNs: https://arxiv.org/abs/2006.16236 Transformers are turing complete RNNs, as are the Neocortex. There are differences like spiking vs non-spiking neurons, universal vs non-universal activation, the effects of brain waves, bicameralism, etc. But these differences may or may not be important depending on the topic of discussion. Additionally, the differences increasingly disappear as transformers are increasingly augmented with functionally equivalent systems.

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by naasking

Sat, 22 Apr 2023 14:40:05 GMT

It's a well known property: https://arxiv.org/abs/1901.03429

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by naasking

Wed, 15 Mar 2023 15:38:32 GMT

> I must have missed the part when it started doing anything algorithmically. I thought it’s applied statistics, with all the consequences of that. This is a common misunderstanding. Transformers are actually Turing complete: * On the Turing Completeness of Modern Neural Network Architectures, https://arxiv.org/abs/1901.03429 * On the Computational Power of Transformers and its Implications in Sequence Modeling, https://arxiv.org/abs/2006.09286

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by naasking

Sun, 12 Mar 2023 14:00:32 GMT

* On the Turing Completeness of Modern Neural Network Architectures, https://arxiv.org/abs/1901.03429 * On the Computational Power of Transformers and its Implications in Sequence Modeling, https://arxiv.org/abs/2006.09286

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by fasterik

Tue, 10 Jan 2023 23:42:04 GMT

Modern neural network architectures are Turing complete [1]. So I don't see any argument for a limit in principle unless you are arguing that a Turing machine can't achieve language understanding. If that's what you're saying, then I wonder who is espousing mysticism here. [1] https://arxiv.org/abs/1901.03429

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by ly3xqhl8g9

Mon, 12 Sep 2022 15:42:19 GMT

Precisely, fun is the operative word. As I was writing I was thinking that perhaps you would still need to run client-side non-layout JS, however, "attention is all you need": it seems that there are some architectures for Turing complete neural networks [1]. I wouldn't worry about performance: Nvidia breaks world records with H100 [2], Intel is going for 6 GHz processors [3], for performance you just have to be patient. [1] 2019, Jorge Pérez et. al, On the Turing Completeness of Modern Neural Network Architectures, https://arxiv.org/pdf/1901.03429.pdf [2]…

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by tsimionescu

Mon, 13 Jun 2022 08:36:08 GMT

That part about "meaningful computations" was indeed a bad phrasing, and, ironically, quite meaningless. Still, it's important to note that "ANNs are Turing Complete" actually means "particular classes of ANNs are Turing Complete" - that is, for any Turing Machine, you can construct an equivalent ANN from that class. Here is a paper [0] showing this for the Transformer architecture that is quite relevant to GPT-3 (not sure if LaMDA is also based on this). Note that even here, there are some caveats - you have to use a positional encoding, and the Transformer must have some particular layers.…

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by riku_iki

Sun, 06 Sep 2020 16:37:52 GMT

Transformer is a function from seq of symbols to seq of symbols. For example, truth table is exactly similar kind of function from variables to [True, False] alphabet. Transformer can represent some very complex logical operations, and per this article is turing complete: https://arxiv.org/abs/1901.03429, meaning any computable function, including theorem prover can be represented as transformer. Another question is if it is feasible/viable/rational to build transformer for this? My intuition says: no.

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — mentioned by bhntr3

Fri, 03 Jul 2020 17:15:30 GMT

Here's an interesting paper on the topic. It seems that only some NN architectures are turing complete: https://arxiv.org/pdf/1901.03429.pdf To the essence of what I think the original question is asking: It seems possible (due to the turing completeness of some NN architectures) that a language model could learn a sub-network that was a turing complete computational framework + problem solver and then use that to solve problems in natural language. But it also seems highly unlikely that it would actually play out that way. The linked paper has a nice summary of attempts to build neural nets…

Paper and discussion history. A mention is not an endorsement.

On the Turing Completeness of Modern Neural Network Architectures — HN submission

Thu, 02 May 2019 12:59:33 GMT

On the Turing Completeness of Modern NN Architectures (Transformer, Neural GPU)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.