Neural Turing Machines · HackerPapers discussions

Neural Turing Machines — mentioned by rvz

Fri, 30 Oct 2026 17:42:13 GMT

HN loves slop meetoo projects without questioning the details. With close inspection it is an overly complicated version of this paper [0] on neural turing machines. This concept was hyped up (because DeepMind) for a while at the time and in reality it amounted to nothing. This is just the second reincarnation of a similar concept but with added extra saturated slop and hype on top. [0] https://arxiv.org/abs/1410.5401

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by rvz

Thu, 29 Oct 2026 01:31:26 GMT

So this is yet another rube goldberg machine. We have gone down this path already in [0] and this is the same thing but overly complicated. This reason is "because LLMs". Why? [0] https://arxiv.org/abs/1410.5401

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — HN submission

Sun, 13 Sep 2026 05:14:09 GMT

Neural Turing Machines (2014)

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Neural Turing Machines — mentioned by cute_boi

Tue, 07 Jul 2026 19:03:23 GMT

No need stupid moving texts. CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/ The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/ Understanding LSTM Networks - https://colah.github.io/posts/2015-08-Understanding-LSTMs/ ImageNet Classification with Deep Convolutional Neural Networks - https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... Deep Residual Learning for Image Recognition - https://arxiv.org/abs/1512.03385 Multi-Scale Context Aggregation by Dilated Convolutions…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by gooob

Tue, 07 Jul 2026 18:54:32 GMT

https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... https://papers.nips.cc/paper_files/paper/2012/file/c399862d3... https://arxiv.org/pdf/1512.03385 https://arxiv.org/pdf/1511.07122 https://arxiv.org/pdf/1603.05027 https://arxiv.org/pdf/1409.2329 https://arxiv.org/pdf/1512.02595 https://arxiv.org/pdf/1409.0473 https://arxiv.org/pdf/1506.03134 https://arxiv.org/pdf/1706.03762 https://nlp.seas.harvard.edu/annotated-transformer/ https://arxiv.org/pdf/1410.5401 https://arxiv.org/pdf/1706.01427 https://arxiv.org/pdf/1806.01822…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by jph00

Thu, 09 Oct 2025 05:34:29 GMT

This is quite a good overview, and parts reflect well how things played out in language model research. It's certainly true that language models and deep learning were not considered particularly promising in NLP, which frustrated me greatly at the time since I knew otherwise! However the article misses the first two LLMs entirely. Radford cited CoVE, ELMo, and ULMFiT as the inspirations for GPT. ULMFiT (my paper with Sebastian Ruder) was the only one which actually fine-tuned the full language model for downstream tasks. https://thundergolfer.com/blog/the-first-llm ULMFiT also pioneered the…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by is_true

Sat, 18 May 2024 12:26:29 GMT

a backup just in case https://nlp.seas.harvard.edu/annotated-transformer/ https://scottaaronson.blog/?p=762 https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://arxiv.org/pdf/1409.2329.pdf https://www.cs.toronto.edu/~hinton/absps/colt93.pdf https://arxiv.org/pdf/1506.03134.pdf https://proceedings.neurips.cc/paper_files/paper/2012/file/c... https://arxiv.org/pdf/1511.06391.pdf https://arxiv.org/pdf/1811.06965.pdf https://arxiv.org/pdf/1512.03385.pdf https://arxiv.org/pdf/1511.07122.pdf https://arxiv.org/pdf/1704.01212.pdf…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by typon

Wed, 10 Apr 2024 11:51:25 GMT

For those unaware of Alex Graves' work, check out the NTM paper: https://arxiv.org/abs/1410.5401 This actually does what this blog post pretends to

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by blackbear_

Sat, 21 Jan 2023 09:12:11 GMT

You seem to be talking about neural Turing machines: https://arxiv.org/abs/1410.5401 Combining these with LLM sounds indeed quite interesting, I don't know why they haven't been used much.

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by blackbear_

Wed, 16 Feb 2022 10:10:45 GMT

A bit disappointed they do not compare with, or at least mention, the memory mechanism of neural turing machines [1]. [1] https://arxiv.org/abs/1410.5401

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by somethingAlex

Sun, 25 Jul 2021 20:24:00 GMT

There's no neural network you could train using supervised learning to solve sorting in the general case. To train a neural network, the training set and test set need to come from a common probability distribution but the inputs of sorting algorithms can be random. There's no relation between what you could possibly train your network on and what you'd ask it to do. Put another way - the input and output of your model would have zero correlation. There's nothing to train the model to pick up on. Interesting related work: Graves[0] trained a "neural Turing machine" to encode instructions for…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by nmfisher

Thu, 04 Jun 2020 08:40:57 GMT

Are you aware of the Neural Turing Machine (https://arxiv.org/abs/1410.5401) or Differentiable Neural Computer (https://deepmind.com/blog/differentiable-neural-computers/)? They're not exactly what you describe, which I suppose is a truly online model that knows how/when to update its own parameters. But those two models do incorporate a similar concept of external memory, whereby the controller is trained via BP to read/write to a tensor (essentially a form of soft-addressable memory available at inference time). As far as I recall, these were never applied beyond toy problems, and it seems…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by stev0lution

Sun, 23 Feb 2020 19:30:38 GMT

"We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples." https://arxiv.org/pdf/1410.5401.pdf

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — HN submission

Tue, 05 Feb 2019 10:01:36 GMT

Neural Turing Machines (2014)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Neural Turing Machines — mentioned by solomatov

Sat, 10 Nov 2018 19:37:37 GMT

- Neural turing machines: https://arxiv.org/abs/1410.5401 - Universal transformers: https://arxiv.org/abs/1807.03819 These are just 2 of most well known.

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by apsec112

Sat, 02 Jun 2018 23:46:42 GMT

While I haven't read those particular papers yet, one common pattern in the ML literature seems to be: 1) Identify a commonly seen problem with deep learning architectures, whether that's large data volumes, lack of transfer learning, etc. 2) Invent a solution to the problem. 3) Test that the solution works on toy examples, like MNIST, simple block worlds, simulated data, etc. 4) Hint that the technique, now proven to work, will naturally be extended to real data sets very soon, so we should consider the problem basically solved now. Hooray! 5) Return to step #1. If anyone applies the…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by unlikelymordant

Mon, 17 Jul 2017 22:42:30 GMT

Input and output sizes don't have to be fixed. E.g. speech recognition doesn't work with fixed sized inputs. Natural language processing deals with many different length sequences. seq2seq networks are explicitly designed to deal with problems that have variable length inputs and outputs that are also variable in length and different from the input. How would you sort integers? using neural turing machines: https://arxiv.org/abs/1410.5401 NMTs and other memory network architectures have explicit memory as state (including stacks!), indeed any recurrent neural net has state. Are NNs Turing…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by deepnet

Wed, 19 Apr 2017 20:31:36 GMT

Neural Turing Machines by Alex Graves, Greg Wayne, Ivo Danihelka https://arxiv.org/abs/1410.5401 A Neural Turing machine is a type of neural network with addressable memory - it learns rather than being explicity programmed. The net also learns to boot, i.e. how to read/ write it's own memory from scratch. Stochastic Gradient Descent learning relies on backpropagation of the error corrections to the individual weights to make a slight improvement - this requires a differentiable neural net. This version innovates with content addressable memory. The authors demonstrate learning from big data…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by cs702

Fri, 30 Dec 2016 16:06:50 GMT

yequalsx: yes, almost by definition, memory made up of bits should be modeled with a field of characteristic 2. That seems the most "natural" way to do it. However, the authors of this paper are thinking about "computers" in a broader sense. Take a look at these two papers: * "Neural Turing Machines:" https://arxiv.org/abs/1410.5401 * "Hybrid computing using a neural network with dynamic external memory:" http://www.nature.com/nature/journal/v538/n7626/full/nature2... and a related blog post at https://deepmind.com/blog/differentiable-neural-computers The memory space of these "neural"…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by sapphireblue

Sun, 31 Jul 2016 16:55:11 GMT

The author proposes a lot of vague ideas in this article (for example "I believe one of the biggest problems is the use of Error Propagation and Gradient Descent") without references or any solid proofs why they are necessary to solve the proposed program (Automate programming using ML?). In fact there is already a lot of solid work just on this subject: * Learning algorithms from examples http://arxiv.org/abs/1511.07275 https://arxiv.org/abs/1410.5401 * Generating source code from natural language description http://arxiv.org/abs/1510.07211 * And, the most closest work to what author…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by sapphireblue

Fri, 22 Jul 2016 20:12:54 GMT

There are signs, if you know were to look for them: http://arxiv.org/abs/1510.07211 https://arxiv.org/abs/1410.4615 http://arxiv.org/abs/1605.06640 https://arxiv.org/abs/1410.5401

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by current_call

Wed, 25 May 2016 01:24:42 GMT

You might enjoy this paper if you haven't already read it. http://arxiv.org/abs/1410.5401 Jump to the experiments section to get an overview of what it can do.

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — HN submission

Sat, 21 May 2016 16:42:47 GMT

Neural Turing Machines (2014)

55 points; 27 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Neural Turing Machines — mentioned by alexbeloi

Fri, 20 May 2016 23:08:35 GMT

Here's the original paper on Neural Turing machines: https://arxiv.org/abs/1410.5401 IIRC, that paper's results focus on the model's ability to extrapolate prediction of sequential data to arbitrary lengths having only been trained on sequences of a fixed length.

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by carapace

Wed, 18 May 2016 01:04:18 GMT

http://arxiv.org/abs/1410.5401

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by andreyk

Wed, 20 Jan 2016 18:24:30 GMT

FYI, RNNs are a form of Deep Learning these days (in particular LSTM and GRU RNNs, very popular for language translation and much more for a few years now). There is also research lately into weirder types of neural nets such as 'Neural Turing Machines' (http://arxiv.org/abs/1410.5401).

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — HN submission

Mon, 18 Jan 2016 05:44:25 GMT

Neural Turing Machines (DeepMind)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Neural Turing Machines — mentioned by sxyuan

Sat, 05 Sep 2015 17:26:39 GMT

One related work that came to mind was Neural Turing Machines, which augments the LSTM network with a (fixed size) memory [1]. I did see a brief reference to the Graves et al. paper in the paper linked to in the original post, but I'm also wondering if someone more knowledgeable could elaborate on the difference. My own guess would be that the models are similar (both using NNs with external memory, trained through backprop), but this has a simpler memory structure and so could be trained more easily. It seems like the choice of regular RNNs vs. an LSTM network doesn't matter much since the…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by deepnet

Mon, 24 Aug 2015 06:19:46 GMT

The OP paper proposes a different architectural approach to the translation task of the 2014 NIPS paper by Ilya Sutskever, Oriol Vinyals & Quoc Le, Sequence to Sequence Learning with Neural Networks http://papers.nips.cc/paper/5346-sequence-to-sequence-learni... which "uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector." The task was English to French. Meng et al.(2015)(OP) translate a Chinese sequence to English, using a network based on Neural Turing…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by kastnerkyle

Thu, 06 Aug 2015 21:31:02 GMT

The literature is mostly image based, but as other users have said voice, text, and video have also seen substantial gains. It has also been used to great effect for taxi destination prediction (heterogenous information) [1], drug discovery [2], and general operations on graphs [3]. There are also lots of examples beyond classification - structured prediction such as image segmentation [4], predicting depth, surface normals, and pixels from only RGB [5], text to text translation [6], image captioning [7], Turing machines [8], speech synthesis [9], question answering (Q/A) [10][11]…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by RoboTeddy

Fri, 17 Jul 2015 22:41:12 GMT

> Our intelligence grows from childhood over many years of training, through our physical and mental interactions with the world. We learn methods along side experiences. It is not about the speed of linear computation, or the amount of memory. There's nothing in principle that prevents machines from learning methods. For example, there's software that can learn some simple algorithms: http://arxiv.org/abs/1410.5401 > Do we know what intelligence is? It's hard to pin down exactly what anything is (unless it's something we constructed from axioms). I'd be hard pressed to tell you exactly what…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by deepnet

Sun, 24 May 2015 08:43:55 GMT

In "Learning to Execute" by Zaremba & Sutskever http://arxiv.org/abs/1410.4615 An RNN learns snippets of python Their next paper is "Reinforcement Learning Neural Turing Machines" http://arxiv.org/abs/1505.00521 based on Graves "Neural Turing Machines" http://arxiv.org/abs/1410.5401, which attempts to infer algorithms from the result. In a lost BBC interview from 1951 Turing reputedly spoke of evolving cpu bitmasks for computation.

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by teraflop

Thu, 21 May 2015 21:41:34 GMT

Currently it doesn't work, but saying it'll never work is pretty strong. This kind of demo shows that deep neural networks can capture the structure of language, if not the semantics, in a very general way. And we have separate evidence that they can (in principle) capture semantic meaning and algorithmic reasoning as well, for example: http://arxiv.org/pdf/1410.5401v2.pdf (the "neural Turing machines" paper from DeepMind)

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by compbio

Mon, 23 Mar 2015 16:19:32 GMT

Deep nets are only loosely inspired by neurobiology. That's why LeCun calls them "convolutional nets" and not "convolutional neural nets" and prefers "nodes" over "neurons". It is, however, possible to have a deep net produce 3D models/images: https://www.youtube.com/watch?v=QCSW4isBDL0 "Learning to Generate Chairs with Convolutional Neural Networks". I also suspect a different part of cognition is used when humans are asked to recreate a "fire truck" than when humans are asked to classify a "fire truck" from a "car". The former seems closer to using memory ("what did the last five fire…

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — mentioned by a-priori

Fri, 13 Feb 2015 18:15:02 GMT

Yes, but computation can be performed with a function approximator where each iteration is a function `f :: (state, input) -> (state', output)`. This is the basis of an architecture called a 'neural turing machine' (http://arxiv.org/abs/1410.5401) and it is, indeed, Turing complete and can be trained through standard neural network training algorithms to perform arbitrary computations.

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — HN submission

Sat, 25 Oct 2014 23:17:07 GMT

Neural Turing Machines by DeepMind [pdf]

8 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Neural Turing Machines — HN submission

Wed, 22 Oct 2014 01:39:58 GMT

Neural Turing Machines

60 points; 7 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Neural Turing Machines — mentioned by nhaehnle

Tue, 21 Oct 2014 17:36:05 GMT

For arXiv submissions, please submit the abstract link; in this case: http://arxiv.org/abs/1410.5401 The advantage is immediate: One gets a plain website with the abstract, which allows the visitor to decide whether the PDF download is worth it. The PDF itself is only one more click away. I personally would also leave out the v1, as that link automatically goes to the latest version in all cases.

Paper and discussion history. A mention is not an endorsement.

Neural Turing Machines — HN submission

Tue, 21 Oct 2014 15:27:10 GMT

Neural Turing Machines [pdf]

75 points; 11 discussion replies on HN. Paper and source history. A submission is not an endorsement.