Sparse Spiking Gradient Descent — mentioned by imtringued

Wed, 11 Jan 2023 13:56:46 GMT

Here is a GPU implementation. https://ieeexplore.ieee.org/document/9635657 It is somewhere from 8x to 25x faster than doing dense machine learning. The speedup was higher on the original CPU implementation and the GPU paper mentions that if there isn't enough shared memory on the GPU it will have to switch to an algorithm that has more overhead. By neurons I actually meant "nodes" My comment is effectively a summary of this article: https://www.kdnuggets.com/2020/03/deep-learning-breakthrough... Edit: There is a paper for sparse spiking gradient descent promising a 150x improvement. I am not…

Paper and discussion history. A mention is not an endorsement.

Sparse Spiking Gradient Descent · HackerPapers discussions

Sparse Spiking Gradient Descent — mentioned by imtringued