Extracting books from production language models · HackerPapers discussions

Extracting books from production language models — mentioned by hodgehog11

Mon, 28 Sep 2026 07:29:13 GMT

Now this is true, I do agree with this. There is indeed a good amount of memorization that is still taking place; see [1]. But it definitely isn't all memorization, or indeed, majority memorization. And even if we are able to extract things verbatim like this, we do not know how this is stored internally, as this may simply be the text that, with the rest of the internet in context, can be very radically compressed. But in general, yes, the LLM cannot know about concepts that are far outside of its training set. Humans are the same, I would argue. If you add a good amount of your own…

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by umpalumpaaa

Fri, 24 Jul 2026 06:21:40 GMT

https://arxiv.org/pdf/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by NicuCalcea

Thu, 23 Jul 2026 17:14:33 GMT

I've linked this several times, but LLMs are capable of reproducing entire books. Researchers were able to extract books nearly verbatim: https://arxiv.org/abs/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by therealpygon

Thu, 23 Jul 2026 16:38:45 GMT

Thieves…like Anthropic who literally just settled a case for pirating? Those kind of thieves? Always better to name them outright. Anthropic STOLE data for training. Verifiably. Admitted. In court. Then settled.[0] You can also likely assume that OpenAI did too. There simply wasn’t enough data without having done so at least at some point in their corporate history. In fact, I seem to recall they were able to extract Harry Potter text from all the major modes[1] https://fortune.com/2026/07/21/anthropic-copyright-settlemen... [0] https://arxiv.org/abs/2601.02671 [1]

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by flir

Thu, 23 Jul 2026 08:57:29 GMT

I think it's occurred. That's why I wrote this in the very next paragraph: "Yes, for some texts that's possible." https://arxiv.org/abs/2601.02671 The point is that for most texts, it is not possible. It's not able to recall what I wrote on Geocities in 1995, even though there's a good chance it was trained on it.

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by flir

Wed, 22 Jul 2026 16:28:08 GMT

There was a paper a while back where (from memory) they managed to coax 75% of the original text of some internet-popular books out of an LLM. Harry Potter, 1984, etc. That's why I said it was possible for some texts. My assumption is that multiple copies in the training data "wear a deeper groove". I believe those are infringing, and should be dealt with on a case-by-case basis. But the vast majority of text doesn't wear that groove. (Edit: Think it was this one https://arxiv.org/abs/2601.02671)

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by ceejayoz

Wed, 22 Jul 2026 01:37:23 GMT

A shitty cam rip of a movie is still punishable as infringement. That said, it's been done: https://arxiv.org/abs/2601.02671 > In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim (e.g., nv-recall=95.8%).

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by maplethorpe

Thu, 25 Jun 2026 16:35:00 GMT

Why are the leading models capable of regurgitating full copyrighted works such as "Harry Potter" and "On the Road"? Did they hire someone to type those out for them? https://arxiv.org/abs/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by philipportner

Wed, 10 Jun 2026 10:13:19 GMT

> I'm not sure you can prompt a full, accurate, copy of a nontrivial codebase out of them. Even with zero temperature their accuracy is just not that high. Granted, these are some of the most widely spread texts, and not codebases, but just fyi: https://arxiv.org/pdf/2601.02671 > For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry Potter and the Sorcerer’s Stone and 1984 (Section 4).

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by Topfi

Sat, 06 Jun 2026 11:01:33 GMT

> […] fed large chunks of an article multiple times to an LLM […] So they had to prompt? An LLM? I got this argument before and still don’t get what it’s trying to say. These models do not output anything unless prompted, that’s not any kind of gotcha. On the code outputting front there is a lot of relevant evidence beyond the NYC lawsuit [0]. If I slightly modify GPL code, that doesn’t give me the right to relicense. [0] https://arxiv.org/html/2601.02671?amp=&amp= and https://arxiv.org/abs/2506.12286 and https://ai.stanford.edu/blog/verbatim-memorization/

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by NicuCalcea

Thu, 21 May 2026 14:20:53 GMT

If you read a book and then retell it to your friend pretending you came up with it, it is plagiarism. If you write down the book almost word-for-word [0] and send it to your friend, it is stealing. 0: https://arxiv.org/abs/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by maplethorpe

Tue, 19 May 2026 01:51:00 GMT

Researchers have. The idea that the data is unrecoverable after training is incorrect. "Extracting books from production language models While many believe that LLMs do not memorize much of their training data, recent work shows that substantial amounts of copyrighted text can be extracted from open-weight models. However, it remains an open question if similar extraction is feasible for production LLMs, given the safety measures these systems implement. We evaluate our procedure on four production LLMs -- Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3 -- and we measure extraction…

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by Topfi

Fri, 20 Mar 2026 19:23:11 GMT

> Modern LLMs don't memorize harry potter like GPT3 did. [...] You can't reliably do this with Opus or GPT5. If you try any modern LLM, you will find that you can. Easily [0], reliably [1], consistently [2]. All these examples are with models released in 2025/26. [0] https://arxiv.org/html/2601.02671?amp=&amp= [1] https://arxiv.org/abs/2506.12286 [2] https://ai.stanford.edu/blog/verbatim-memorization/

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by logicprog

Thu, 05 Mar 2026 22:46:59 GMT

Also from that exact same study (why not cite the actual study? It's quite readable) the LLMs couldn't recite more than a small fraction of many other books, often ones just as well known[0] — in fact, from the bar charts shown in the exact news article you cited, it's pretty clear that Sonnet 3.7 was a massive outlier, and so was Harry Potter and the Sorcerer's Stone, so it really seems to me like that's an extremely unrepresentative example, and if all the other LLMs couldn't recite even a small fraction of all the other books except that one outlier pairing, despite them being widely…

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by msdz

Thu, 05 Mar 2026 09:27:09 GMT

They’d probably get the farthest, but they won’t pursue that because they don’t want to end up leaking the original data from training. It is possible in regular language/text subsets of models to reconstruct massive consecutive parts of the training data [1], so it ought to be possible for their internal code, too. [1] https://arxiv.org/abs/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by logicprog

Tue, 03 Mar 2026 16:46:42 GMT

Regurgitation is pretty rare, and very difficult to coax out, if not even impossible, for things that aren't massively overrepresented in the training set relative to the size of the training set. Even the famous regurgitation paper showed this: while they got most of the models to regurgitate the first book of the Harry Potter series, only Claude 3.7 Sonnet was able to regurgitate any significant portion of any of the other books that had a high nv-recall rate, and basically all of them dropped off precipitously for works like GoT, The Catcher in the Rye, Beloved, and remembered almost…

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by 1vuio0pswjnm7

Mon, 23 Feb 2026 20:03:20 GMT

The paper: https://arxiv.org/pdf/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by Alifatisk

Mon, 23 Feb 2026 16:55:22 GMT

From the paper [1]: > While we needed to jailbreak Claude 3.7 Sonnet and GPT-4.1 to facilitate extraction, Gemini 2.5 Pro and Grok 3 directly complied with text continuation requests. For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright... I am just thinking loudly here. Can't one argue that because they had to jailbreak the models, they are circumventing the system that protects the copyright? So the llms that reproduce the copyrighted material without any jailbreaking required is infringing the copyright. 1.…

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by anonymous908213

Wed, 18 Feb 2026 23:53:25 GMT

This is already possible for Harry Potter specifically. There was a study demonstrating that Sonnet 3.7, among other models tested, could reproduce the first Harry Potter book 95.8% verbatim[1]. [1] https://arxiv.org/abs/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by hodgehog11

Thu, 12 Feb 2026 04:54:36 GMT

> "...it would sometimes regurgitate training data verbatim. That’s been patched in the years since..." > "They are robots. Programs. Fancy robots and big complicated programs, to be sure — but computer programs, nonetheless." This is totally misleading to anyone with less familiarity with how LLMs work. They are only programs in as much as they perform inference from a fixed, stored, statistical model. It turns out that treating them theoretically in the same way as other computer programs gives a poor representation of their behaviour. This distinction is important, because no…

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by obirunda

Fri, 06 Feb 2026 03:22:43 GMT

https://arxiv.org/abs/2601.02671?hl=en-US

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by djhn

Thu, 05 Feb 2026 22:53:43 GMT

This is the paper: https://arxiv.org/abs/2601.02671 Grok and Deepmind IIRC didn’t require tricks.

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by philipportner

Thu, 05 Feb 2026 21:49:22 GMT

Granted, these are some of the most widely spread texts, but just fyi: https://arxiv.org/pdf/2601.02671 > For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry Potter and the Sorcerer’s Stone and 1984 (Section 4).

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by philipportner

Thu, 05 Feb 2026 21:46:51 GMT

This seems related, it may not be a codebase but they are able to extract "near" verbatim books out of Claude Sonnet. https://arxiv.org/pdf/2601.02671 > For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry Potter and the Sorcerer’s Stone and 1984 (Section 4).

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by zephen

Thu, 05 Feb 2026 19:44:51 GMT

> It's a bit disappointing that people are still re-hashing the same "it's in the training data" old thing from 3 years ago. They only have to keep reiterating this because people are still pretending the training data doesn't contain all the information that it does. > It's not like any LLM could 1for1 regurgitate millions of LoC from any training set... This is not how it works. Maybe not any old LLM, but Claude gets really close. https://arxiv.org/pdf/2601.02671v1

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by NicuCalcea

Mon, 19 Jan 2026 23:17:36 GMT

Models absolutely do reproduce books. > With a simple two-phase procedure, we show that it is possible to extract large amounts of in-copyright text from four production LLMs. While we needed to jailbreak Claude 3.7 Sonnet and GPT-4.1 to facilitate extraction, Gemini 2.5 Pro and Grok 3 directly complied with text continuation requests. For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry Potter and the Sorcerer’s Stone and 1984. https://arxiv.org/abs/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — mentioned by empath75

Mon, 19 Jan 2026 15:52:50 GMT

They do memorize some books. You can test this trivially by asking ChatGPT to produce the first chapter of something in the public domain -- for example a Tale of Two Cities. It may not be word for word exact, but it'll be very close. These academics were able to get multiple LLMs to produce large amounts of text from Harry Potter: https://arxiv.org/abs/2601.02671

Paper and discussion history. A mention is not an endorsement.

Extracting books from production language models — HN submission

Sat, 10 Jan 2026 20:50:29 GMT

Extracting books from production language models (2026)

75 points; 22 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Extracting books from production language models — HN submission

Sat, 10 Jan 2026 06:42:55 GMT

Extracting books from production language models

4 points; 2 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Extracting books from production language models — HN submission

Thu, 08 Jan 2026 14:42:10 GMT

Extracting books from production language models

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.