Will we run out of data? Limits of LLM scaling based on human-generated data · HackerPapers discussions

Will we run out of data? Limits of LLM scaling based on human-generated data — mentioned by wavewrangler

Wed, 30 Sep 2026 05:52:32 GMT

Geoffrey Hinton thinks AI is conscious. I know he's extremely accomplished and all, but as it pertains to AI as it exists now we're miles apart (not to say that he's not brilliant, but by difference of opinion). I also don't want to rehash the whole what-is-consciousness philosophical debate, and actually it's a separate question anyway, something doesn't have to be conscious to be dangerous. So I'll stick to the danger bit. Do you use AI much? Not a trick question. What is your usage like? Reason I ask is recall OpenAI hyping GPT-2 with the same language as they are these new models. Doom…

Paper and discussion history. A mention is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — mentioned by theptip

Mon, 29 Sep 2025 15:11:48 GMT

I don’t think this is true. See https://arxiv.org/html/2211.04325v2 for example.

Paper and discussion history. A mention is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — mentioned by theptip

Sun, 28 Sep 2025 16:09:34 GMT

> Where are they going to get more data? This is a great question, but note that folks were freaking out about this a year or so ago and we seem to be doing fine. We seem to be making progress with some combination of synthetic training datasets on coding/math tasks, textbooks authored by paid experts, and new tokens (plus preference signals) generated by users of the LLM systems. It wouldn’t surprise me if coding/math turned out to have a dense-enough loss-landscape to produce enough synthetic data to get to AGI - though I wouldn’t bet on this as a highly likely outcome. I have been wanting…

Paper and discussion history. A mention is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — HN submission

Fri, 07 Jun 2024 17:08:29 GMT

Will we run out of data? Limits of LLM scaling based on human-generated data

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — mentioned by hungrigekatze

Wed, 13 Dec 2023 22:59:51 GMT

I have no doubt that this "feature" is a backroom deal worth millions because OpenAI is running out of public internet data with which to improve its models. (See this paper from researchers at MIT and a few other schools which predicts that high-quality text training data will be 'used up' by 2026: https://arxiv.org/abs/2211.04325 ) Think of all of the email, Google Docs, and other data that Alphabet has that it can use to train and improve its models. OpenAI has limited ways to get non-public text data unless Microsoft is giving them some data from Office users, Hotmail users. Just my two…

Paper and discussion history. A mention is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — HN submission

Thu, 03 Aug 2023 12:53:23 GMT

Will we run out of data to train LLMs?

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — HN submission

Thu, 05 Jan 2023 10:48:59 GMT

Will we run out of data? An analysis of the limits of scaling datasets in ML

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — HN submission

Fri, 23 Dec 2022 01:17:13 GMT

Will we run out of data?

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — HN submission

Thu, 22 Dec 2022 03:22:37 GMT

An analysis of the limits of scaling datasets in Machine Learning

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — mentioned by ren_engineer

Mon, 05 Dec 2022 14:25:06 GMT

there's a paper that basically shows these LLMs are pretty much tapped out in terms of quality data to train on, not sure if there are any legit architecture improvements in the works to improve results quality but it seems brute force training with more data may be about over - https://arxiv.org/pdf/2211.04325.pdf As for SEO, Google is going to have to stop being lazy and relying so much on backlinks and domain authority. They might have to consider allowing searchers to essentially vote on what they consider good. I don't see any other way low quality AI content doesn't dominate SERPs…

Paper and discussion history. A mention is not an endorsement.

Will we run out of data? Limits of LLM scaling based on human-generated data — HN submission

Tue, 29 Nov 2022 07:03:52 GMT

We might run out of language data to train ML models soon [pdf]

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.