Measuring Massive Multitask Language Understanding · HackerPapers discussions

Measuring Massive Multitask Language Understanding — mentioned by thomasliao

Wed, 24 Jun 2026 09:31:00 GMT

(Author) It's short for "evaluation", a test for an AI model. Specifically, an AI evaluation comprises (1) a dataset of prompts (as questions / tasks / queries), (2) some way to score model performance on each prompt, like a set of correct answers or a grading rubric that you can use with an LLM autograder, and (3) a metric, such as accuracy¹. (If you're already familiar with the term "benchmark", it's the same thing; for some reason the former has become the term of art in the past few years). For example, a simple eval is a dataset of multiple-choice questions, which each have one correct…

Paper and discussion history. A mention is not an endorsement.

Measuring Massive Multitask Language Understanding — HN submission

Sat, 20 Apr 2024 19:28:40 GMT

Measuring Multitask Language Understanding

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Measuring Massive Multitask Language Understanding — mentioned by gwern

Mon, 29 Mar 2021 17:59:17 GMT

Well... https://www.eduref.net/features/what-grades-can-ai-get-in-co... https://arxiv.org/abs/2009.03300

Paper and discussion history. A mention is not an endorsement.

Measuring Massive Multitask Language Understanding — HN submission

Mon, 14 Sep 2020 15:54:06 GMT

Measuring Multitask Language Understanding

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Measuring Massive Multitask Language Understanding — HN submission

Wed, 09 Sep 2020 05:41:47 GMT

Measuring Multitask Language Understanding

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.