Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data · HackerPapers discussions

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — mentioned by konmok

Sun, 13 Sep 2026 03:54:44 GMT

> The latest studies demonstrate model collapse is not a given Which studies? [edit: I'll assume you mean these two given by @dorolow: https://arxiv.org/abs/2404.01413 https://arxiv.org/abs/2406.07515] > It can't be used as the -only- data source of course, but that's not how it is being used Right, so human data creation would also have to scale up exponentially, and that's not gonna happen. > because there's no difference between synthetic data and the data people can create I mean, that's obviously false, otherwise model collapse wouldn't exist. The difference is statistical, but it's…

Paper and discussion history. A mention is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — mentioned by dorolow

Sun, 13 Sep 2026 03:52:33 GMT

We use large amounts of synthetic data for training at work and have not observed any sort of model collapse when done properly. Edit: https://arxiv.org/abs/2404.01413 https://arxiv.org/abs/2406.07515

Paper and discussion history. A mention is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — mentioned by dwohnitmok

Wed, 26 Aug 2026 23:40:09 GMT

That paper has had a pretty turbulent reception and looks pretty conclusively wrong at this point. It used an incorrect theoretical framing that assumed that data was being replaced rather than accumulated as a result of more training (see https://arxiv.org/abs/2404.01413 which explores this). This is incorrect because this simply isn't how real-world datasets are created via synthetic data generation (which generally accumulate more data over time rather than replace their data). As a result most of the theoretical results were invalid. Empirical evidence has also cast a considerable amount…

Paper and discussion history. A mention is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — mentioned by ainch

Wed, 26 Aug 2026 08:47:59 GMT

"Model collapse" is often overstated, as this paper demonstrates: https://arxiv.org/pdf/2404.01413 The original model collapse paper assumes you train networks on 100% synthetic data produced by the previous generation. But if you maintain some portion of real data then the problem is mitigated.

Paper and discussion history. A mention is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — mentioned by tkgally

Sat, 07 Dec 2024 07:38:52 GMT

This paper was first published in May 2023 and discussed on HN the following month: https://news.ycombinator.com/item?id=36319076 Some research since seems to add nuance to its conclusions: https://arxiv.org/abs/2404.01413

Paper and discussion history. A mention is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — HN submission

Wed, 24 Jul 2024 19:42:56 GMT

Avoiding Model Collapse via Accumulating Real and Synthetic Data

5 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — mentioned by groby_b

Thu, 27 Jun 2024 21:23:22 GMT

You're confusing it with data poisoning. Model collapse itself is(was?) a fairly serious research topic: https://arxiv.org/abs/2305.17493 We've by now reached a "probably not inevitable" - https://arxiv.org/abs/2404.01413 argues there's a finite upper bound to error - but I'd also point out that that paper assumes training data cardinality increases with the number of training generations and is strictly accumulative. To a first order, that means you better have a pre-2022 dataset to get started, and have archived it well. but it's probably fair to say current SOTA is still more or less "it's…

Paper and discussion history. A mention is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — HN submission

Sun, 02 Jun 2024 09:48:29 GMT

Is Model Collapse Inevitable? Breaking the Curse of Recursion

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — HN submission

Thu, 02 May 2024 12:36:02 GMT

Is Model Collapse Inevitable?

5 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — HN submission

Mon, 08 Apr 2024 18:47:58 GMT

Is Model Collapse Inevitable? Breaking the Curse with Real and Synthetic Data

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.