The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks · HackerPapers discussions

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — mentioned by astrange

Wed, 06 Jan 2021 05:06:07 GMT

Models can unpredictably memorize sensitive input data, so there can be a real copyright issue here, I think. https://arxiv.org/abs/1802.08232 Worse, sometimes the input data is illegal to distribute for other reasons than copyright.

Paper and discussion history. A mention is not an endorsement.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — mentioned by Imnimo

Tue, 05 Jan 2021 19:10:08 GMT

Point 1 might be less clear cut in some settings. Obviously, if you're just doing a simple regression, it's no big deal, but more powerful models can memorize and expose parts of their training data. See for example: https://arxiv.org/abs/1802.08232

Paper and discussion history. A mention is not an endorsement.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — mentioned by fyp

Fri, 15 Nov 2019 21:41:32 GMT

We probably need a technical rather than legal solution for this. The problem is that generative algorithms are susceptible to accidental memorization so you can't guarantee that the output will be transformative. For example, play with https://talktotransformer.com to see how many well known pieces of text it can spew out verbatim. It is very prone to derailing into a harry potter fan fiction regardless of prompt. Other than copyright there are privacy considerations too. For example Gmail's Smart Compose is trained on users' private messages so you don't want it to memorize "private"…

Paper and discussion history. A mention is not an endorsement.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — mentioned by craffel

Fri, 25 Oct 2019 14:21:18 GMT

It actually can be more pernicious than that: https://arxiv.org/abs/1802.08232 However note that the dataset used to train GPT-2 is about 20x smaller than C4. I'm not 100% sure how many times the training set was repeated over the course of training for GPT-2, but it was likely many times. I stand by my statement (that memorization is unlikely with SGD and no repetition of training data) but I would be happy to be proven otherwise.

Paper and discussion history. A mention is not an endorsement.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — mentioned by evrydayhustling

Thu, 16 May 2019 13:51:38 GMT

This is probably a "publishing in the press" translation issue, but the notion that neural networks have no memory is dead wrong. DNNs encode incredibly specific histories of training data in their weights, which is both a privacy/security risk [1] and a phenomena that is actively being utilized in many dnn architectures. Overall, this has a "brand the framework before proving exceptional capabilities" feel. [1] https://arxiv.org/abs/1802.08232

Paper and discussion history. A mention is not an endorsement.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — HN submission

Fri, 26 Oct 2018 20:03:37 GMT

Measuring Unintended Neural Network Memorization and Extracting Secrets

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — HN submission

Mon, 05 Mar 2018 13:31:46 GMT

The Secret Sharer: Measuring Unintended NN Memorization and Extracting Secrets

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — HN submission

Wed, 28 Feb 2018 01:03:04 GMT

The Secret Sharer: Measuring Unintended NN Memorization and Extracting Secrets

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — HN submission

Tue, 27 Feb 2018 01:38:41 GMT

Measuring Unintended Neural Network Memorization and Extracting Secrets [pdf]

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.