Overcoming catastrophic forgetting in neural networks · HackerPapers discussions

Overcoming catastrophic forgetting in neural networks — mentioned by johnsmith1840

Sat, 14 Jun 2025 00:33:06 GMT

AGI likely a combination of these two papers + something new likely along the lines of distillation. 1. Preventing collapse -> model gets "full" https://arxiv.org/pdf/1612.00796 2. Forgetting causes better generalization https://arxiv.org/abs/2307.01163 3. Unknow paper that connects this - allow a "forgetting" model that improves generalization over time. - I tried for a long time to make this but it's a bit difficult Fun implication is that if true this implies AGI will need "breaks" and likely need to consume non task content of high variety much like a person does.

Paper and discussion history. A mention is not an endorsement.

Overcoming catastrophic forgetting in neural networks — mentioned by ilaksh

Sat, 30 Sep 2017 08:02:23 GMT

Yes there are still lots of people trying to take Hinton and other's old approach and rediscovering the many ways it lacks for general intelligence. However, it is also the case that people are making a shitton of progress in overcoming those problems, both while keeping some of those old DL assumptions and by discarding many of them. The pessimistic articles never seem to be aware of research like these: https://arxiv.org/abs/1612.00796 and https://hackernoon.com/feynman-machine-a-new-approach-for-co... .

Paper and discussion history. A mention is not an endorsement.

Overcoming catastrophic forgetting in neural networks — HN submission

Mon, 05 Dec 2016 01:49:04 GMT

Overcoming catastrophic forgetting in neural networks

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.