MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models — mentioned by mk_stjames

Sat, 14 Oct 2023 17:44:41 GMT

The entire GPT4 model was not retrained to utilize images, instead I think it is much more like show in [1] where an extra layer(s) are learned that take another pretrained image model and map that model to the latent space of the LLM. In [1] this was just a single linear layer. I found it pretty shocking how simple it was. I believe OpenAI is doing something similar. [1] https://arxiv.org/pdf/2304.10592.pdf

Paper and discussion history. A mention is not an endorsement.

MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models · HackerPapers discussions

MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models — mentioned by mk_stjames