Memory Aware Synapses: Learning what (not) to forget — mentioned by axelroze

Sat, 28 Aug 2021 15:03:16 GMT

It's not that simple because that way you lose too much information. Actually it is more likely the whole system would fail if weights at any layer are reset. There is a way to selectively unlearn something via Memory Aware Synapses (MAS): - https://arxiv.org/abs/1711.09601 The idea was developed mostly for transfer learning as in learn new stuff on a new domain but do not forget the old stuff as well. For forgetting it could be trained on some old images + all zeros target mask and the MAS to preserve everything else.

Paper and discussion history. A mention is not an endorsement.

Memory Aware Synapses: Learning what (not) to forget · HackerPapers discussions

Memory Aware Synapses: Learning what (not) to forget — mentioned by axelroze