Fast Feedforward Networks · HackerPapers discussions

Fast Feedforward Networks — mentioned by WithinReason

Wed, 22 Nov 2023 13:35:02 GMT

Link to previous paper: https://arxiv.org/abs/2308.14711 An attempt at a summary: They use a sigmoid function to make differentiable "soft" branches, and stack them to construct a binary tree, with the goal of only taking one branch at inference time (but training the whole tree) leading to log(W) instead of W inference cost. They gradually harden the branches so they become hard branches at the end of training. A branch is computed as branch(input, N), with a neural network N computing a scalar c=N(input), then using a sigmoid to do a soft branch by returning the weighted sum of the…

Paper and discussion history. A mention is not an endorsement.

Fast Feedforward Networks — HN submission

Thu, 21 Sep 2023 19:43:52 GMT

Fast Feedforward Networks, 220x faster than feedforward networks

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Fast Feedforward Networks — HN submission

Wed, 20 Sep 2023 08:34:39 GMT

Fast Feedforward Networks

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.