Gemma 3 Technical Report · HackerPapers discussions

Gemma 3 Technical Report — mentioned by h8hawk

Sun, 27 Sep 2026 09:39:45 GMT

and this: https://arxiv.org/abs/2503.19786

Paper and discussion history. A mention is not an endorsement.

Gemma 3 Technical Report — mentioned by canyon289

Wed, 20 Aug 2025 17:55:32 GMT

Thank you! The Gemma3 technical report contains many details on training setup https://arxiv.org/pdf/2503.19786 This was released with the initial batch of Gemma3 so it doesn't contain the 270m details, nonetheless you'll get a good idea of what it takes to build these models.

Paper and discussion history. A mention is not an endorsement.

Gemma 3 Technical Report — mentioned by canyon289

Thu, 14 Aug 2025 21:21:17 GMT

Not rude at all and I'll again share what I can. We ran a bunch of experimental architectures at this size to get a sense of performance at this size, in particular how well it was able to adapt to datasets across some loss measures. For the embedding size it comes from a mix of "hard technical" data, like the loss measures I mentioned above, and for this model it also comes from community considerations such as adaptability across input tokens and consistency with the gemma ecosystem. At this size you are right its a bit funny the embedding is so large. For more details read the Gemma3…

Paper and discussion history. A mention is not an endorsement.

Gemma 3 Technical Report — mentioned by ben_w

Mon, 16 Jun 2025 13:56:30 GMT

> When I read the Gemma 3 paper (https://arxiv.org/html/2503.19786v1) and saw an entire section dedicated to measuring and reducing the memorization rate I was annoyed. How does this benefit end users at all? It benefits users because memorisation is a waste of parameters that would be more useful if they were instead learning rules and generalisations. For short snippets, common idioms and quotations that people recognise, exact quotes can be worth memorising; but the longer the quotations get, the less often it is important to be word-for-word exact — even for just a few paragraphs, I think…

Paper and discussion history. A mention is not an endorsement.

Gemma 3 Technical Report — mentioned by msp26

Mon, 16 Jun 2025 12:31:11 GMT

Agree completely. When I read the Gemma 3 paper (https://arxiv.org/html/2503.19786v1) and saw an entire section dedicated to measuring and reducing the memorization rate I was annoyed. How does this benefit end users at all? I want the language model I'm using to have knowledge of cultural artifacts. Gemma 3 27B was useless at a question related to grouping Berserk characters by potential baldurs gate 3 classes; Claude did fine. The methods used to reduce memorisation rate probably also deteriorate performance in some other ways that don't show up on benchmarks.

Paper and discussion history. A mention is not an endorsement.