BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models · HackerPapers discussions

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models — mentioned by sitkack

Fri, 05 May 2023 11:44:55 GMT

Given the exposure hn has had to blip-2 it might be good to provide more context. https://arxiv.org/abs/2301.12597 https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... Were you expecting a response?

Paper and discussion history. A mention is not an endorsement.

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models — mentioned by HarHarVeryFunny

Mon, 17 Apr 2023 13:22:10 GMT

It has more in common with GPT-3 than GPT-4 in terms of size, but in reality it's based on Vicuna/Llama which is 10x smaller than either, so as far as the LLM part of it goes its not mini-anything - it's just straight-up Vicuna 13B. The model as a whole is just BLIP-2 with a larger linear layer, and using Vicuna as the LLM. If you look at their code it's literally using the entire BLIP-2 encoder (Salesforce code). https://arxiv.org/pdf/2301.12597.pdf

Paper and discussion history. A mention is not an endorsement.

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models — mentioned by thatsadude

Mon, 17 Apr 2023 13:08:48 GMT

How is this a paper, they simply fine-tune the fully-connected layer of BLIP-2 https://arxiv.org/pdf/2301.12597.pdf

Paper and discussion history. A mention is not an endorsement.

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models — mentioned by f38zf5vdt

Tue, 21 Mar 2023 13:42:46 GMT

You are humanizing token prediction. The multimodal models for text-vision were all established using a scaffold of architectures that unified text-token and vision-token similarity e.g. BLIP2. [1] It's possible that a model using unified representations might be able to establish that the set of visual tokens you are searching for corresponds to some set of text tokens, but only if the pretrained weights for the vision encoder are able to extract the features corresponding to the object to which you are describing to the vision model. And the pretrained vision encoder will have at some point…

Paper and discussion history. A mention is not an endorsement.

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models — HN submission

Tue, 31 Jan 2023 05:45:30 GMT

Blip-2

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.