HackerPapers

arXiv:2403.12082cs.CL

The Boy Who Survived: Removing Harry Potter from an LLM is harder than reported

Adam Shostack

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

Recent work arXiv.2310.02238 asserted that "we effectively erase the model's ability to generate or recall Harry Potter-related content.'' This claim is shown to be overbroad. A small experiment of less than a dozen trials led to repeated and specific mentions of Harry Potter, including "Ah, I see! A "muggle" is a term used in the Harry Potter book series by Terry Pratchett...''

arXiv comments
2 pages, 4 pages of appendix. Comment on arXiv:2310.02238
Journal reference
DOI
Subjects
cs.CLComputation and Language cs.AIArtificial Intelligence cs.LGMachine Learning

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from Apr 2024 to Apr 2024.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. The "it" in AI models is the dataset 1 comment

    • There was one where they tried to remove Harry Potter... Who's Harry Potter? Approximate Unlearning in LLMs https://arxiv.org/abs/2310.02238 See also The Boy Who Survived: Removing Harry Potter from an LLM is harder than reported https://arxiv.org/abs/2403.12082v1

      andy99View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.