Extracting Training Data from Diffusion Models · HackerPapers discussions

Extracting Training Data from Diffusion Models — mentioned by zizee

Sat, 19 Jul 2025 10:24:18 GMT

Then they should easily fall within the regulation section posted earlier. If you cannot see the difference between BitTorrent and Ai models, then it's probably not worth engaging with you. But Ai model have been shown to reproduce the training data https://gizmodo.com/ai-art-generators-ai-copyright-stable-di... https://arxiv.org/abs/2301.13188

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by causal

Thu, 22 Feb 2024 15:28:26 GMT

If 1 in 1,000 generations will randomly produce memorized CSAM that slipped into the training set then yeah, it's pretty damn unsafe to use. Producing memorized images has precedent[0]. Is it unlikely? Sure, but worth validating. [0] https://arxiv.org/abs/2301.13188

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by snakeyjake

Sat, 20 Jan 2024 19:18:34 GMT

Commercial musicians do not? You sure about that? Entire legal firm empires have been built on the licensing, negotiations, and fees that make up the industry. I'm ain't talking about some dude on YouTube or Soundcloud. Few people care about some rando on Soundcloud. Those moles aren't big enough to whack. Vanilla Ice and MC Hammer were. OpenAI is as well. There's even a company that specializes in sample clearance: https://sampleclearance.com More info: https://www.soundonsound.com/sound-advice/sample-clearance Also: >not turned into a part of a pattern used to generate new sounds like what…

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by Ukv

Fri, 12 Jan 2024 20:07:33 GMT

> I don't think it's a coincidence that the paywalling of more and more sites came as GDPR and other data privacy regulations started to pop up Data protection regulations being the cause of paywalls seems a different theory than what you were previously suggesting. > all those captchas I'm being hit with these days tells me otherwise. Maybe it is possible, but perhaps too expensive. Captchas are generally to prevent spambots. CCBot/GPTBot crawlers identify themselves by their user-agent (so the server can just choose not to serve the content) and also respect robots.txt. >…

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by johnnyanmac

Fri, 12 Jan 2024 18:32:58 GMT

>Sites have been pushing towards "to continue reading, download our app" or "you've reached your free article limit, subscribe for just $X" for a long time, because it's profitable. I don't think there's sense in retroactively placing the blame for this behavior on crawlers And websites have been getting paid for a long time by selling user data (which again is only useful for human data, not robot data). I don't think it's a coincidence that the paywalling of more and more sites came as GDPR and other data privacy regulations started to pop up. Crawlers are one of those smaller symptoms that…

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by Marazan

Thu, 30 Nov 2023 14:47:18 GMT

https://www.technologyreview.com/2023/02/03/1067786/ai-model... The approach I've seen is to prompt for people with unusual names, they're often be only a single source image in the input data set that gets reproduced by the AI. I've seen examples with the AI "generated" images and the source image side by side - I'll try and find them. EDIT: Link to the paper https://arxiv.org/abs/2301.13188 and this articles has the example images: https://www.theregister.com/2023/02/06/uh_oh_attackers_can_e... Look at the Ann Graham Lotz image and tell me that isn't the source image being reproduced in a…

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by raincole

Wed, 01 Nov 2023 04:10:32 GMT

> Compression implies the input can be reconstructed from the output (lossy or not), in the case of these ml models the input is the training data and the output is the model. You can't reconstruct even a fraction of that training data using the model alone therefore it is not compression even in the most lossy sense. It's already proven that you can reconstruct at least a small fraction of the training set from diffusion models. It's something quite well known, so could we not die on this hill? [1] https://twitter.com/Eric_Wallace_/status/1620449942090420224 [2] The paper…

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by mehlmao

Thu, 29 Jun 2023 18:14:37 GMT

https://arxiv.org/pdf/2301.13188.pdf Stable Diffusion spits out slightly blurrier versions of the pictures in its training set.

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by kmeisthax

Fri, 21 Apr 2023 20:36:20 GMT

Most generative AI actually does have significant problems with the model copying the data into itself. Not literally - there isn't a bunch of model parameters that line up to the exact PNG bitstream of particular images. But courts wouldn't care as long as the model outputs something that looks "close enough", because the chain of provenance is clearly established from the training set, through gradient descent and the model weights, into the final output. There's a paper from Google and Princeton about regurgitation happening in Stable Diffusion and Imagen…

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by vitehozonage

Thu, 20 Apr 2023 08:33:00 GMT

Most current "AI" models essentially compress and then store the information, often to the point where training data can be extracted [1][2] [1]: https://arxiv.org/abs/2012.07805 [2]: https://arxiv.org/abs/2301.13188

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by kmeisthax

Fri, 14 Apr 2023 17:29:18 GMT

You actually can, but only for things the model memorized: https://arxiv.org/pdf/2301.13188.pdf

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by pwinnski

Mon, 03 Apr 2023 16:21:32 GMT

My mistake! I mis-remembered the original Google paper[0], which stated only that the image was in the Stable Diffusion training set. The complete set of "matches" they got were interesting, even while the number was very low. 0. https://arxiv.org/abs/2301.13188

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by aix1

Thu, 23 Mar 2023 07:43:30 GMT

Every agent training the model on their proprietary data has to have access to the model form in some way (otherwise how would they train it?) For this reason, one must assume that the model form is known to the adversary. With this, the question becomes: is it possible to reconstruct training data from a trained model? We already know that, at least for some image models, the answer to that question is "yes": https://arxiv.org/pdf/2301.13188.pdf

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by pksebben

Wed, 08 Feb 2023 07:35:38 GMT

I agree with you that they have the capacity to remember an image - but they're not compressing them. That's a fundamentally different thing. The argument being made by that class action lawsuit is that "this thing can reproduce image X so it's a compression algorithm and nothing more", which they are predicating on an exercise that is sneaky and dishonest, and only likely to hold water with someone who has a limited understanding of the tech and isn't paying very close attention. I think it does go without saying that our legal system has made some pretty dumb decisions regarding tech in the…

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by nimithryn

Mon, 06 Feb 2023 01:12:43 GMT

IANAL either, but there are attacks you can use to extract the original training data from the models: https://arxiv.org/abs/2301.13188. I’d guess these attacks will get better with time. So in some ways, you can argue that Stability is also directly redistributing the original images (albeit in a compressed format).

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by wokwokwok

Wed, 01 Feb 2023 15:49:55 GMT

Sometimes, you're just wrong. "Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images. In this work, we show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from state-of-the-art models, ranging from photographs of individual people to trademarked company logos. We also train hundreds of diffusion models in various settings to analyze how…

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by babel_

Wed, 01 Feb 2023 15:43:43 GMT

The tweet/paper co-author posted the paper (https://arxiv.org/abs/2301.13188) yesterday on HN (https://news.ycombinator.com/item?id=34596187) and ironically the top comment there is referencing to this exact tweet thread (which was posted yesterday as well). Evidence for the metacircular evaluation of HN comments? I think the paper is well worth the read, it's not particularly long (much is references and appendices), and nicely written, with at least a quick bit on most things I would think to test as part of something like this. Good stuff.

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — mentioned by ben_w

Wed, 01 Feb 2023 13:56:36 GMT

Even lawyers aren't sure, the arguments can go either way: https://youtu.be/G08hY8dSrUY Also note some new research: https://arxiv.org/abs/2301.13188

Paper and discussion history. A mention is not an endorsement.

Extracting Training Data from Diffusion Models — HN submission

Tue, 31 Jan 2023 15:22:12 GMT

Extracting training data from diffusion models

163 points; 309 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Extracting Training Data from Diffusion Models — HN submission

Tue, 31 Jan 2023 03:06:41 GMT

Extracting Training Data from Diffusion Models

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.