DINOv2: Learning Robust Visual Features without Supervision — mentioned by panabee

Tue, 25 Apr 2023 01:11:08 GMT

dino v2 reflipped me, reasoning that text supervision artificially constrains image learning: https://arxiv.org/abs/2304.07193 if unified modeling isn't necessary, or even hurtful, a key advantage of transformers goes away.

Paper and discussion history. A mention is not an endorsement.

DINOv2: Learning Robust Visual Features without Supervision — mentioned by jeeeb

Wed, 19 Apr 2023 23:39:46 GMT

I don’t have any insider insight on this but the GPT3 paper discusses some of their data sets and curation techniques (https://arxiv.org/pdf/2005.14165.pdf). The recent DinoV2 paper is also interesting reading (https://arxiv.org/pdf/2304.07193.pdf), as they particularly focus on techniques for improving the training set. OpenAI also have been open about making heavy use of RL (via PPO) to fine tune the models. For RL it seems they’ve basically developed a second model that can be used to score the quality of responses based on the encoded preferences of human evaluators. I.e you build a…

Paper and discussion history. A mention is not an endorsement.

DINOv2: Learning Robust Visual Features without Supervision · HackerPapers discussions

DINOv2: Learning Robust Visual Features without Supervision — mentioned by panabee

DINOv2: Learning Robust Visual Features without Supervision — mentioned by jeeeb