Character-Aware Neural Language Models — mentioned by juxtaposicion

Tue, 27 Oct 2015 14:58:11 GMT

I think you're right when thinking about CNNs on words. It's max-pool that's usually combined with CNNs that helps with translational invariance, less so the CNN filters themselves (which, if it were a full convolution, would show up in the complex phase). I think it makes more sense when CNNs are applied at the character level. The filter banks then activate for specific n-gram patterns of characters, like certain prefixes, suffixes, and root words. The higher level LSTMs are then relieved of having to understand that level of structure. Also, tokenization is hard, and might be especially…

Paper and discussion history. A mention is not an endorsement.

Character-Aware Neural Language Models · HackerPapers discussions

Character-Aware Neural Language Models — mentioned by juxtaposicion