Are Transformers universal approximators of sequence-to-sequence functions? · HackerPapers discussions

Are Transformers universal approximators of sequence-to-sequence functions? — mentioned by solid_fuel

Thu, 18 Jun 2026 08:56:00 GMT

1) You’re still wrong, this is prompt injection. 2) You continue to have basic misunderstandings of the issue. That bugs exist in other things does not mean a core design flaw in LLMs can magically be fixed. 3) https://arxiv.org/pdf/1912.10077 This paper doesn’t have any bearing to the question of the separation of user and command data in LLMs. Did you even bother to look at it? 4) Hey you’re the one that made the claim. If you can't event remember why, I can’t help you. 5) Because the world is stateful. 6) Wow so you just decided to add up all the ram after all, huh? If you want to play…

Paper and discussion history. A mention is not an endorsement.

Are Transformers universal approximators of sequence-to-sequence functions? — mentioned by anuramat

Thu, 18 Jun 2026 05:12:05 GMT

> yes it is agree to disagree > every LLM has been vulnerable and every OS had bugs > show the logic https://arxiv.org/pdf/1912.10077 > you are the one asserting mappings existed I know? that's why I'm asking? > no useful system can be a pure function why not? surely you can describe useful systems with qm? evolution operator of a closed system seems pretty pure to me it's almost as if you could reformulate anything such that the state was one of the arguments of the function > you can start adding up the state space for the linux kernel I can give you a lower bound -- (your estimate for…

Paper and discussion history. A mention is not an endorsement.

Are Transformers universal approximators of sequence-to-sequence functions? — mentioned by hodgehog11

Fri, 13 Feb 2026 07:16:12 GMT

Fine, that's fair. I believe the statement that you made is countered by my claim, which is: Theorem. For any tolerance epsilon > 0, there exists a transformer neural network of sufficient size that follows, up to the factor epsilon, the policy that most optimally achieves arbitrary goals in arbitrary stochastic environments. Proof (sketch). For any stochastic environment with a given goal, there exists a model that maximizes expected return under this goal (not necessarily unique, but it exists). From Solomonoff's convergence theorem (Theorem 3.19 in [1]), Bayes-optimal predictors under the…

Paper and discussion history. A mention is not an endorsement.

Are Transformers universal approximators of sequence-to-sequence functions? — mentioned by triclops200

Mon, 03 Nov 2025 20:05:13 GMT

As a researcher in these fields: this reasoning is tired, overblown, and just wrong. We have a lot of understanding of how the brain works overall. You don't. Go read the active inference book by Friston et. al. for some of the epistemological and behavioral mechanics (Yes, this applies to llms as well, they easily satisfy the requirements to be considered the mathematical object described as a markov blanket). And, yes, if you could somehow freeze a human's current physical configuration at some time, you would absolutely, in principle, given what we know about the universe, be able to…

Paper and discussion history. A mention is not an endorsement.

Are Transformers universal approximators of sequence-to-sequence functions? — mentioned by logicchains

Tue, 12 Mar 2024 09:17:21 GMT

We already understand how transformers work: their architecture can learn to approximate a very large class of functions-on-sequences (specifically continuous sequence-to-sequence functions with compact support: https://arxiv.org/abs/1912.10077). It can do it more accurately than previous architectures like RNNs because transformers don't "forget" any information from prior items in the sequence. Training transformers to predict the next item in sequences eventually forces them to learn a function that approximates a world model (or at least a model of how the world behaves in the training…

Paper and discussion history. A mention is not an endorsement.

Are Transformers universal approximators of sequence-to-sequence functions? — mentioned by hackinthebochs

Sat, 13 May 2023 15:56:54 GMT

People really need to update their model of what a "statistical predictor" can accomplish. We know that Transformers are universal approximators of sequence-to-sequence functions[1], and so any structure that can be encoded into a sequence-to-sequence map can be modeled by Transformer layers. It follows that prediction and modeling are not categorically distinct capacities in LLMs, but exist on a continuum. How well the model predicts in a given instance is largely due to the availability of data during training. This is the basis for the beginnings of genuine understanding in LLMs. I talk…

Paper and discussion history. A mention is not an endorsement.