Multimodal Autoregressive Pre-training of Large Vision Encoders — mentioned by m_ke

Fri, 27 Dec 2024 22:18:16 GMT

The world is a fuzzy place, most things are not binary. I haven't worked in medical imaging in a while but VLMs make for much better diagnostic tools than task specific classifiers or segmentation models which tend to find hacks in the data to cheat on the objective that they're optimized for. The next token objective turns our to give us much better vision supervision than things like CLIP or classification losses. (ex: https://arxiv.org/abs/2411.14402) I spent the last few years working on large scale food recognition models and my multi label classification models had no chance of…

Paper and discussion history. A mention is not an endorsement.

Multimodal Autoregressive Pre-training of Large Vision Encoders · HackerPapers discussions

Multimodal Autoregressive Pre-training of Large Vision Encoders — mentioned by m_ke