Dataset Distillation — mentioned by gota

Tue, 27 Jul 2021 01:39:20 GMT

I guess I can see some parallel the way you describe it, but it is not that tight of an analogy IMO. Correct me if I'm wrong but the Lottery Ticket Hypothesis mainly means that, for many of our tasks, we start with needlessly huge networks, and thus a good portion of the learning process is just the network learning to "cope with the noise" from the extraneous connections Hence why the lottery ticket network (being a smaller version of the same initial network) can achieve similar accuracy (but not 'better') in much shorter time - because you took away all the confounding stuff. That kind of…

Paper and discussion history. A mention is not an endorsement.

Dataset Distillation · HackerPapers discussions

Dataset Distillation — mentioned by gota