PaLI: A Jointly-Scaled Multilingual Language-Image Model · HackerPapers discussions

PaLI: A Jointly-Scaled Multilingual Language-Image Model — mentioned by osmarks

Sun, 14 Apr 2024 16:34:58 GMT

(https://arxiv.org/pdf/2209.06794.pdf page 20.)

Paper and discussion history. A mention is not an endorsement.

PaLI: A Jointly-Scaled Multilingual Language-Image Model — mentioned by famouswaffles

Thu, 22 Feb 2024 15:36:03 GMT

This explanation is wrong as I've already said (256 is not the result of any conversion to text) but no one has to take my word for it. From the Gemini report https://arxiv.org/abs/2312.11805 >The visual encoding of Gemini models is inspired by our own foundational work on Flamingo (Alayrac et al., 2022), CoCa (Yu et al., 2022a), and PaLI (Chen et al.,2022), with the important distinction that the models are multimodal from the beginning and can natively output images using discrete image tokens (Ramesh et al., 2021; Yu et al., 2022b). These are the papers Google say the multimodality in…

Paper and discussion history. A mention is not an endorsement.

PaLI: A Jointly-Scaled Multilingual Language-Image Model — mentioned by reaperman

Wed, 24 May 2023 04:45:06 GMT

This model is vision-only so it can't be SOTA even if it's #1 performing in many of the original categories of benchmarks, which it is (it's a very very good model). We've moved on from ImageNet-style tests "Choose the most appropriate label for this image from 200 possible labels" to much more advanced "Reasoning" tests[0]. PaLI[1] is potentially the SoTA here but BeIT-3[2] may be better example for my thesis. Notice that BeIT-3 is trained on not just images, but also trained in natural language. It outperforms purely image-trained models on even pure-image tasks like Object Detection and…

Paper and discussion history. A mention is not an endorsement.

PaLI: A Jointly-Scaled Multilingual Language-Image Model — mentioned by runnerup

Thu, 04 May 2023 03:07:32 GMT

http://www.incompleteideas.net/IncIdeas/BitterLesson.html Generally we have continued finding that the more "other"/general stuff an AI model is trained on, the better it performs on specific tasks. As in, an AI model trained to identify photos of all animals will perform better than an AI model that is only trained to identify breeds of dogs. Even at identifying breeds of dogs. Taken to the extreme, we've found that training image models with "multi-modal" LLM capabilities improves their ability to identify dogs/etc. A lot of people don't realize that GPT-4 is actually multi-modal...while…

Paper and discussion history. A mention is not an endorsement.