Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop — mentioned by TradingPlaces

Sat, 09 Dec 2023 22:36:12 GMT

"Large language models (LLM) have become state of the art in many benchmarks and conversational LLM applications like ChatGPT are now widely used by the public. Those LLMs can be used to generate large amounts of content which is posted on the internet to various platforms. As LLMs are trained on datasets usually collected from the internet, this LLM-generated content might be used to train the next generation of LLMs. Therefore, a self-consuming training loop emerges in which new LLM generations are trained on the output from the previous generations. We empirically study this self-consuming…

Paper and discussion history. A mention is not an endorsement.

Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop · HackerPapers discussions

Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop — mentioned by TradingPlaces