Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws · HackerPapers discussions

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws — mentioned by bconsta

Sat, 04 Oct 2025 20:46:21 GMT

There is a study that gives a rule of thumb of ~2 bits per param for a model's memorization capacity: https://arxiv.org/abs/2404.05405

Paper and discussion history. A mention is not an endorsement.

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws — mentioned by kouteiheika

Mon, 16 Jun 2025 06:08:34 GMT

No, assuming that just because it was in the training data it must be memorized is hare brained. LLMs have limited capacity to memorize, under ~4 bits per parameter[1][2], and are trained on terabytes of data. It's physically impossible for them to memorize everything they're trained on. The model memorized chunks of Harry Potter not just because it was directly trained on the whole book, which the article also alludes to: > For example, the researchers found that Llama 3.1 70B only memorized 0.13 percent of Sandman Slim, a 2009 novel by author Richard Kadrey. That’s a tiny fraction of the 42…

Paper and discussion history. A mention is not an endorsement.

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws — mentioned by huac

Fri, 17 Jan 2025 11:21:01 GMT

from an AI research perspective -- it's pretty straightforward to mitigate this attack 1. perplexity filtering - small LLM looks at how in-distribution the data is to the LLM's distribution. if it's too high (gibberish like this) or too low (likely already LLM generated at low temperature or already memorized), toss it out. 2. models can learn to prioritize/deprioritize data just based on the domain name of where it came from. essentially they can learn 'wikipedia good, your random website bad' without any other explicit labels. https://arxiv.org/abs/2404.05405 and also another recent paper…

Paper and discussion history. A mention is not an endorsement.

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws — mentioned by smaddox

Tue, 25 Jun 2024 13:23:48 GMT

Because existing LLMs store no more than 2bits of knowledge per parameter, despite having many more bits of precision: https://arxiv.org/abs/2404.05405

Paper and discussion history. A mention is not an endorsement.

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws — HN submission

Tue, 09 Apr 2024 03:56:53 GMT

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.