SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/1905.00537/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTSuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems — mentioned by RcouF1uZ4gsChttps://news.ycombinator.com/item?id=21351253urn:hackerpapers:mention:1905.00537:21351253Fri, 25 Oct 2019 02:23:15 GMT<p>I think classifying this as human level is misleading. Look at the sub-scores on the page. One score that looks very different from humans is AX-b. The SuperGlue paper provides more context about AX-b https://arxiv.org/pdf/1905.00537.pdf AX-b &quot;is the broad-coverage diagnostic task, scored using Matthews’ correlation (MCC). &quot; This is how the paper describes this test &quot; Analyzing Linguistic and World Knowledge in Models GLUE includes an expert-constructed, diagnostic dataset that automatically tests models for a broad range of linguistic, commonsense, and world knowledge. Each example in this…</p><p><a href="https://hackerpaper.kaipfister.de/p/1905.00537/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL