Visualizing and Understanding Recurrent Networks · HackerPapers discussions

Visualizing and Understanding Recurrent Networks — mentioned by magoghm

Wed, 14 Feb 2024 14:24:52 GMT

I tried to find the where I heard that Radford was inspired by that blog post, but the closest thing I found is that in the "Sentiment Neuron" paper (Learning to Generate Reviews and Discovering Sentiment: https://arxiv.org/pdf/1704.01444.pdf), in the "Discussion and Future Work" section they mention this Karpathy paper from 2015: Visualizing and Understanding Recurrent Networks https://arxiv.org/abs/1506.02078

Paper and discussion history. A mention is not an endorsement.

Visualizing and Understanding Recurrent Networks — mentioned by ninjin

Fri, 08 Jan 2021 14:58:31 GMT

Yikes, you are right… I just linked a private repo. '^^ I have poked the rest of the group and it seems that at least a tweet was made [1] – but not much else remains. Describing it from memory, we ran ELMo and BERT on Wikipedia and then allowed similarity search between a query and showed heat maps to a matched context. Nothing particularly deep compared to yours that go into the transformer “machinery”, but I think it captures very well how most Question Answering models still operate: Embed query and contexts in a high-dimensional space, compare, find semantically plausible span, and done!…

Paper and discussion history. A mention is not an endorsement.

Visualizing and Understanding Recurrent Networks — mentioned by veselin

Sun, 07 Jul 2019 13:56:43 GMT

This one is a character level language model. The Microsoft one is per token. This one should be the closest to the Microsoft model: https://miltos.allamanis.com/publicationfiles/allamanis2018l... Btw, simple neural networks are far from doing the best for code. The kind of accuracy LSTM gives is already achieved with good n-gram models. E.g. see this one on a Linux code prediction: https://arxiv.org/pdf/1506.02078.pdf Also, there are other techniques that improve on top of the n-gram models: https://files.sri.inf.ethz.ch/website/papers/charmodel-iclr2...

Paper and discussion history. A mention is not an endorsement.

Visualizing and Understanding Recurrent Networks — mentioned by fnbr

Wed, 19 Jul 2017 16:11:24 GMT

So, this is definitely good work, and I don't have any suggestions on how to do it better, but I'm ultimately not sure how useful I find it. I have no intuition for what the histograms for `t` should look like, and so seeing the different histograms for ծ vs `not ծ` doesn't tell me much except that they're different (which is trivially true!). Does anyone else find this sort of visualization useful? Maybe I'm just misunderstanding this. I would love to develop more of an intuition for neuron activations in deep nets- I want to better understand how they work, as right now, the only thing that…

Paper and discussion history. A mention is not an endorsement.

Visualizing and Understanding Recurrent Networks — HN submission

Thu, 07 Jan 2016 12:31:16 GMT

Visualizing and understanding recurrent networks

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Visualizing and Understanding Recurrent Networks — HN submission

Wed, 10 Jun 2015 01:12:39 GMT

Visualizing and Understanding Recurrent Networks

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.