On the Challenges of Fuzzing Techniques via Large Language Models · HackerPapers discussions

On the Challenges of Fuzzing Techniques via Large Language Models — mentioned by westurner

Sun, 06 Jul 2025 23:47:17 GMT

Is there an established benchmark for building a full product? - SWE-bench leaderboard: https://www.swebench.com/ - Which metrics for e.g. "SWE-Lancer: a benchmark of freelance software engineering tasks from Upwork"? https://news.ycombinator.com/item?id=43101314 - MetaGPT, MGX: https://github.com/FoundationAgents/MetaGPT : > Software Company as Multi-Agent System > MetaGPT takes a one line requirement as input and outputs user stories / competitive analysis / requirements / data structures / APIs / documents, etc. Internally, MetaGPT includes product managers / architects / project managers…

Paper and discussion history. A mention is not an endorsement.

On the Challenges of Fuzzing Techniques via Large Language Models — mentioned by westurner

Sun, 18 Feb 2024 06:06:27 GMT

"Automated Unit Test Improvement using Large Language Models at Meta" (2024) https://arxiv.org/abs/2402.09171 : > This paper describes Meta's TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. TestGen-LLM verifies that its generated test classes successfully clear a set of filters that assure measurable improvement over the original test suite, thereby eliminating problems due to LLM hallucination. [...] We believe this is the first report on industrial scale deployment of LLM-generated code backed by such assurances of code improvement. Coverage-guided…

Paper and discussion history. A mention is not an endorsement.