We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs · HackerPapers discussions

We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs — mentioned by simonw

Sat, 15 Nov 2025 19:35:37 GMT

Here's another one that went un-cited: > When you ask AI to generate code with dependencies, it hallucinates non-existent packages 19.7% of the time. One. In. Five. > Researchers generated 2.23 million packages across various prompts. 440,445 were complete fabrications. Including 205,474 unique packages that simply don’t exist. That looks like this report from June 2024: https://arxiv.org/abs/2406.10279 Here's the thing: the quoted numbers are totals across 16 early-2024 models, and most of those hallucinations came from models with names like CodeLlama 34B Python and WizardCoder 7B Python…

Paper and discussion history. A mention is not an endorsement.

We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs — mentioned by dijksterhuis

Sat, 12 Apr 2025 13:06:27 GMT

socket article seems to mostly be a review of this arXiv preprint paper: https://arxiv.org/pdf/2406.10279 there’s also some info from Python software foundation folks in the register article, so it’s not just a socket pitch article.

Paper and discussion history. A mention is not an endorsement.

We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs — HN submission

Tue, 01 Oct 2024 01:06:50 GMT

A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs

31 points; 9 discussion replies on HN. Paper and source history. A submission is not an endorsement.