eyeballvul: a future-proof benchmark for vulnerability detection in the wild · HackerPapers discussions

eyeballvul: a future-proof benchmark for vulnerability detection in the wild — HN submission

Mon, 06 Jul 2026 11:53:55 GMT

Ask HN: Are there good security benchmarks for LLMs?

I'm looking for this myself but figured it's good to have an actual discussion about this. I'm pretty new to the benchmarking side of LLMs. For example, I looked at eyeballvull [1]. It seems promising but I don't see wide support for example. I respect the author for still committing. A benchmark where an agent scans a repo in full is what I'm looking for. But then I also wondered: maybe there are others out there that I haven't been aware of yet. And, at the risk of potentially being flooded, for people that are curious about security aware software engineering with agents (or without them…

7 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

eyeballvul: a future-proof benchmark for vulnerability detection in the wild — mentioned by summarity

Sat, 04 Apr 2026 11:11:03 GMT

Already happend: https://arxiv.org/abs/2407.08708

Paper and discussion history. A mention is not an endorsement.

eyeballvul: a future-proof benchmark for vulnerability detection in the wild — HN submission

Thu, 01 Aug 2024 17:57:03 GMT

Eyeballvul: A future-proof benchmark for vulnerability detection in the wild

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.