arXiv:2301.04589cs.CL
Memory Augmented Large Language Models are Computationally Universal
Abstract
We show that transformer-based large language models are computationally universal when augmented with an external memory. Any deterministic language model that conditions on strings of bounded length is equivalent to a finite automaton, hence computationally limited. However, augmenting such models with a read-write memory creates the possibility of processing arbitrarily large inputs and, potentially, simulating any algorithm. We establish that an existing large language model, Flan-U-PaLM 540B, can be combined with an associative read-write memory to exactly simulate the execution of a universal Turing machine, $U_{15,2}$. A key aspect of the finding is that it does not require any modification of the language model weights. Instead, the construction relies solely on designing a form of stored instruction computer that can subsequently be programmed with a specific set of prompts.
I do. This is the newest thng i'm aware of: https://www.percepta.ai/blog/can-llms-be-computers But there were papers in 2023 with a different approach requiring external memory https://arxiv.org/abs/2301.04589 too
GomotonoView on HNParent context