Gemini: A Family of Highly Capable Multimodal Models · HackerPapers discussions

Gemini: A Family of Highly Capable Multimodal Models — mentioned by andy12_

Sat, 06 Dec 2025 11:15:55 GMT

No, the "large _language_ model" name is a misnomer nowadays. Some time ago it was indeed common to get a pure-text model and inject embeddings from a separately trained image-encoder (which generated "meh" results), but current natively multi-modal models are pre-trained with both text and images from the ground-up. That's why they are so much better at image understanding. > Gemini models are trained on a dataset that is both multimodal and multilingual. Our pre-training dataset uses data from web documents, books, and code, and includes image, audio, and video data.…

Paper and discussion history. A mention is not an endorsement.

Gemini: A Family of Highly Capable Multimodal Models — mentioned by repsak

Fri, 17 May 2024 20:57:39 GMT

I raise you the Gemini paper https://arxiv.org/abs/2312.11805

Paper and discussion history. A mention is not an endorsement.

Gemini: A Family of Highly Capable Multimodal Models — mentioned by Hugsun

Tue, 14 May 2024 22:56:36 GMT

According to the paper[1], Gemini Nano-1 is 1.8B, and (Nano-2) is 3.25B parameters. They are quantized to 4-bits which makes them 0.9GB and 1.6GB respectively. That's a juicy binary blob to ship with Chrome. As this will be on device, it will be interesting to see the inevitable reverse engineering of the model, if Google doesn't just release the weights. [1]: https://arxiv.org/pdf/2312.11805

Paper and discussion history. A mention is not an endorsement.

Gemini: A Family of Highly Capable Multimodal Models — mentioned by espadrine

Tue, 09 Apr 2024 16:47:22 GMT

Huh, that is true in both the model cards of Gemini 1.5 Pro and Gemini 1.0. That feels like it runs counter to this statement from the Gemini 1.0 technical report[0]: > Gemini models are trained to accommodate textual input interleaved with a wide variety of audio and visual inputs, such as natural images, charts, screenshots, PDFs, and videos, and they can produce text and image outputs [0]: https://arxiv.org/pdf/2312.11805.pdf

Paper and discussion history. A mention is not an endorsement.

Gemini: A Family of Highly Capable Multimodal Models — mentioned by PheonixPharts

Fri, 08 Mar 2024 18:37:25 GMT

The benchmarks are so easy to game it's not meaningful unless you see a major difference and even then it can be questionable. For example, did anyone read the Gemini paper[0]? The team details the slight modification they made to the metric which is basically a tweak on maj@32 that worked in their favor. They choose to do the majority vote of 32 samples unless the probability of the highest sample was too low and then they chose greedy, they also had to learn what "too low" was. Sound convoluted and a bit contrived? They even show plots that clearly demonstrate they settled on the metric…

Paper and discussion history. A mention is not an endorsement.

Gemini: A Family of Highly Capable Multimodal Models — mentioned by famouswaffles

Thu, 22 Feb 2024 15:36:03 GMT

This explanation is wrong as I've already said (256 is not the result of any conversion to text) but no one has to take my word for it. From the Gemini report https://arxiv.org/abs/2312.11805 >The visual encoding of Gemini models is inspired by our own foundational work on Flamingo (Alayrac et al., 2022), CoCa (Yu et al., 2022a), and PaLI (Chen et al.,2022), with the important distinction that the models are multimodal from the beginning and can natively output images using discrete image tokens (Ramesh et al., 2021; Yu et al., 2022b). These are the papers Google say the multimodality in…

Paper and discussion history. A mention is not an endorsement.

Gemini: A Family of Highly Capable Multimodal Models — mentioned by famouswaffles

Thu, 22 Feb 2024 14:11:26 GMT

>You can literally convert the tokens returned from a video to text. What do you even think tokens are? Tokens are patches of each image. It's amazing to me how people will confidently spout utter nonsense. It only takes looking at the technical report for the Gemini models to see that you're completely wrong. https://arxiv.org/abs/2312.11805 >The visual encoding of Gemini models is inspired by our own foundational work on Flamingo (Alayrac et al., 2022), CoCa (Yu et al., 2022a), and PaLI (Chen et al.,2022), with the important distinction that the models are multimodal from the beginning and…

Paper and discussion history. A mention is not an endorsement.

Gemini: A Family of Highly Capable Multimodal Models — HN submission

Wed, 20 Dec 2023 21:46:44 GMT

Gemini: A Family of Highly Capable Multimodal Models

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Gemini: A Family of Highly Capable Multimodal Models — HN submission

Wed, 20 Dec 2023 10:34:29 GMT

Gemini: A Family of Highly Capable Multimodal Models

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.