Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models · HackerPapers discussions

Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models — mentioned by kastnerkyle

Mon, 10 Oct 2016 22:35:15 GMT

There are a number of papers tackling a similar task [0][1][2] for anyone who is interested. There isn't enough information to tell exactly what is going on with Deepgram, but one way to approach this would be to construct a shared embedding space for words/phrases and speech. These types of embedding spaces are powerful [3][4][5], but not magic. Cool demo, looking forward to seeing more detail about what is going on. However I would quibble with the STT WER quoted above. Maybe in noisy environments with unknown speakers (and no voice normalization) this is accurate, but the kinds of clean…

Paper and discussion history. A mention is not an endorsement.

Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models — mentioned by tfgg

Tue, 18 Nov 2014 15:11:24 GMT

ArXiv abstract links, which are preferable to direct links to the PDF: http://arxiv.org/abs/1411.2539 http://arxiv.org/abs/1410.1090 http://arxiv.org/abs/1411.4389

Paper and discussion history. A mention is not an endorsement.

Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models — mentioned by karpathy

Tue, 18 Nov 2014 01:24:44 GMT

Hi all, I'm the author on the Stanford article, so I can answer any more detailed/technical questions. The article doesn't mention this, but closely related work I've only very recently become aware of (if anyone is interested) are additionally: From UofT: http://arxiv.org/pdf/1411.2539v1.pdf From Baidu/UCLA: http://arxiv.org/pdf/1410.1090v1.pdf From Google: http://googleresearch.blogspot.com/2014/11/a-picture-is-wort... From Stanford (our work): http://cs.stanford.edu/people/karpathy/deepimagesent/ Edit: And seemingly also from Berkeley http://arxiv.org/pdf/1411.4389v1.pdf At least from my…

Paper and discussion history. A mention is not an endorsement.

Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models — mentioned by benanne

Tue, 18 Nov 2014 01:19:05 GMT

Looks like the Toronto group have been working on something very similar as well: http://lanl.arxiv.org/abs/1411.2539 Has anybody been able to find the Google paper? The article says it's on arxiv, but I can't seem to find it there. All that seems to be published so far is this blog post: http://googleresearch.blogspot.be/2014/11/a-picture-is-worth...

Paper and discussion history. A mention is not an endorsement.