Learning Activation Functions: A new paradigm for understanding Neural Networks — mentioned by porridgeraisin

Thu, 27 Feb 2025 16:12:21 GMT

This idea exists (the broad field is called neural architecture search), although you have to parameterize it somehow to allow gradient descent to happen. Here are examples: https://arxiv.org/abs/2009.04759 https://arxiv.org/abs/1906.09529

Paper and discussion history. A mention is not an endorsement.

Learning Activation Functions: A new paradigm for understanding Neural Networks — mentioned by adamzen

Wed, 08 Feb 2023 19:17:39 GMT

Learned activation functions do seem to be a thing(https://arxiv.org/abs/1906.09529)

Paper and discussion history. A mention is not an endorsement.

Learning Activation Functions: A new paradigm for understanding Neural Networks · HackerPapers discussions

Learning Activation Functions: A new paradigm for understanding Neural Networks — mentioned by porridgeraisin

Learning Activation Functions: A new paradigm for understanding Neural Networks — mentioned by adamzen