MiniF2F: a cross-system benchmark for formal Olympiad-level mathematics — mentioned by gowld

Tue, 14 Mar 2023 18:38:14 GMT

> It's AMC-12 scores aren't awful. A blank test scores 37.5 The best score 60 is 5 correct answers + 20 blank answers; or 6 correct, 4 correct random guesses, and 15 incorrect random guesses. (20% chance of correct guess) The 5 easiest questions are relatively simple calculations, once the parsing task is achieved. (Example: https://artofproblemsolving.com/wiki/index.php/2022_AMC_12A_... ) so the main factor in that score is how good GPT is at refusing to answer a question, or doing a bit better to overcome the guessing penalty. > It's AMC 10 score being dramatically lower is pretty bad…

Paper and discussion history. A mention is not an endorsement.

MiniF2F: a cross-system benchmark for formal Olympiad-level mathematics · HackerPapers discussions

MiniF2F: a cross-system benchmark for formal Olympiad-level mathematics — mentioned by gowld