SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks — mentioned by jegp

Tue, 25 Apr 2023 06:54:00 GMT

If we think about spikes as discretized transfer functions, I would say it's totally viable to "hack" them to represent numerical approximations. In fact, a recent paper demonstrates that exact mappings between ANNs and SNNs exist: https://arxiv.org/abs/2212.12522 There are some pitfalls here, though, and I'm biased against these kinds of methods because they don't use the temporal traces of the neuron integration. Regarding RWKV, someone actually trained a "SpikeGPT": https://arxiv.org/abs/2302.13939 That's a neat insight, which will be great for porting these models onto energy-efficient…

Paper and discussion history. A mention is not an endorsement.

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks — HN submission

Wed, 01 Mar 2023 06:07:05 GMT

SpikeGPT: Generative Pre-Trained Language Model with Spiking Neural Networks

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks · HackerPapers discussions

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks — mentioned by jegp

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks — HN submission