A Cookbook of Self-Supervised Learning · HackerPapers discussions

A Cookbook of Self-Supervised Learning — mentioned by mistrial9

Wed, 12 Jun 2024 21:18:16 GMT

> but the current bottleneck is figuring out how to get a model to train itself against its own output without human intervention this is sort of humorous, because the entire current generation of Transformer-based encoders was built specifically to get "a model to train itself without human intervention" ref. arXiv:2304.12210v2 [cs.LG] 28 Jun 2023 * A Cookbook of Self-Supervised Learning

Paper and discussion history. A mention is not an endorsement.

A Cookbook of Self-Supervised Learning — mentioned by isaacfung

Tue, 19 Sep 2023 03:48:59 GMT

This article was written in 2021 when masked language models have been successfully applied in nlp(Bert, word2vec, glove, etc). However at the time, it was unclear how the same technique could be applied to vision tasks because unlike language which has a limited vocab, you can't explicitly assign a probability to every possible image. Since then researchers have already made significant progress with techniques like contrastive learning(simclr), self distillation (BYOL, DINO), masked image models, etc. A cookbook of self-supervised learning is a good source to learn more about this topic.…

Paper and discussion history. A mention is not an endorsement.

A Cookbook of Self-Supervised Learning — HN submission

Tue, 25 Apr 2023 16:13:29 GMT

A Cookbook of Self-Supervised Learning

178 points; 22 discussion replies on HN. Paper and source history. A submission is not an endorsement.

A Cookbook of Self-Supervised Learning — HN submission

Tue, 25 Apr 2023 08:03:46 GMT

A Cookbook of Self-Supervised Learning

5 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.