An Algorithm for Routing Capsules in All Domains · HackerPapers discussions

An Algorithm for Routing Capsules in All Domains — mentioned by cs702

Wed, 03 Jun 2020 23:59:07 GMT

Have you looked at capsule-routing algorithms that use expectation-maximization to generate layer outputs in deep neural networks? For example, in https://research.google/pubs/pub46653/ and https://arxiv.org/abs/1911.00792 each output capsule in a layer is generated by a Gaussian mixture model and also has an associated "activation value" that increases to the extent the GMM can explain (i.e., generate) its input data data "better" than the GMMs of other output capsules in the same layer. I wonder, do you think these routing algorithms, which are differentiable end-to-end, could be…

Paper and discussion history. A mention is not an endorsement.

An Algorithm for Routing Capsules in All Domains — mentioned by cs702

Sun, 03 May 2020 21:48:41 GMT

I'm surprised there is no mention of capsules and capsule-routing algorithms. Capsules are groups of neurons that represent discrete entities in different contexts. For example, a 4x4 pose matrix is a capsule representing a particular object in different orientations seen from different viewpoints. Similarly, a subword embedding can be seen as a capsule with vector shape representing a particular subword in different natural language contexts. More generally, a capsule can have any shape, but it always represents only one entity in some context. In certain new capsule-routing algorithms…

Paper and discussion history. A mention is not an endorsement.

An Algorithm for Routing Capsules in All Domains — mentioned by cs702

Wed, 11 Dec 2019 00:32:41 GMT

I recently saw a paper (which I reposted here on HN a while back) that perhaps could help with your problem. The paper proposes an approach that sort of induces models to learn representations of objects that are good at predicting "the most agreed-upon" rotations of the inputs. I'm not explaining it well. Anyway, a model in the paper achieved SOTA on a change-of-viewpoint dataset with a really tiny number of parameters. Might be worth a look: https://arxiv.org/abs/1911.00792

Paper and discussion history. A mention is not an endorsement.

An Algorithm for Routing Capsules in All Domains — HN submission

Wed, 06 Nov 2019 16:55:50 GMT

Maximizing ‘bang per bit’ to achieve state of the art results

24 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.