Deep Equilibrium Models — mentioned by imtringued

Wed, 08 Oct 2025 09:06:33 GMT

Deep Equilibrium Models >We present a new approach to modeling sequential data: the deep equilibrium model (DEQ). Motivated by an observation that the hidden layers of many existing deep sequence models converge towards some fixed point, we propose the DEQ approach that directly finds these equilibrium points via root-finding. Such a method is equivalent to running an infinite depth (weight-tied) feedforward network, but has the notable advantage that we can analytically backpropagate through the equilibrium point using implicit differentiation. https://arxiv.org/abs/1909.01377 What's…

Paper and discussion history. A mention is not an endorsement.

Deep Equilibrium Models — mentioned by ciaranby

Fri, 12 Feb 2021 17:42:37 GMT

Great post, thanks! I saw Growing Neural Cellular Automata document you describe a strategy to get the model to learn attractor dynamics. I was kind of reminded of Deep Equilibrium Models (https://arxiv.org/abs/1909.01377). Is there a relationship between these models and do you think these root finding and implicit differentiation techniques could be used to train Cellular Automata too?

Paper and discussion history. A mention is not an endorsement.

Deep Equilibrium Models · HackerPapers discussions

Deep Equilibrium Models — mentioned by imtringued

Deep Equilibrium Models — mentioned by ciaranby