Are You Human? An Adversarial Benchmark to Expose LLMs — mentioned by refulgentis

Fri, 20 Dec 2024 20:15:37 GMT

- "Are You Human" https://arxiv.org/pdf/2410.09569 is designed to be directly on target, i.e. cross cutting set of questions that are easy for humans, but challenging for LLMs, Instead of one type of visual puzzle. Much better than ARC for the purpose you're looking for. - SimpleBench https://simple-bench.com/ (similar to above; great landing page w/scores that show human / ai gap) - PIQA (physical question answering, i.e. "how do i get a yolk out of a water bottle", common favorite of local llm enthusiasts in /r/localllama https://paperswithcode.com/dataset/piqa - Berkeley Function-Calling…

Paper and discussion history. A mention is not an endorsement.

Are You Human? An Adversarial Benchmark to Expose LLMs · HackerPapers discussions

Are You Human? An Adversarial Benchmark to Expose LLMs — mentioned by refulgentis