Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad · HackerPapers discussions

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — mentioned by bwfan123

Sat, 19 Jul 2025 16:53:17 GMT

Based on the past history with frontier-math & AIME 2025 [1],[2] I would not trust announcements which cant be independently verified. I am excited to try it out though. Also, the performance of LLMs on imo 2025 was not even bronze [3]. Finally, this article shows that LLMs were just mostly bluffing [4] on usamo 2025. [1] https://www.reddit.com/r/slatestarcodex/comments/1i53ih7/fro... [2] https://x.com/DimitrisPapail/status/1888325914603516214 [3] https://matharena.ai/imo/ [4] https://arxiv.org/pdf/2503.21934

Paper and discussion history. A mention is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — mentioned by bwfan123

Sat, 19 Jul 2025 15:22:42 GMT

> At this point, I think it can only be explained by ignorance, bad faith, or fear of becoming irrelevant. Based on the past history with frontier-math & AIME 2025 [1],[2] I would not trust announcements which cant be independently verified. I am excited to try it out though. Also, the performance of LLMs was not even bronze [3]. Finally, this article shows that LLMs were just mostly bluffing [4]. [1] https://www.reddit.com/r/slatestarcodex/comments/1i53ih7/fro... [2] https://x.com/DimitrisPapail/status/1888325914603516214 [3] https://matharena.ai/imo/ [4] https://arxiv.org/pdf/2503.21934

Paper and discussion history. A mention is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — mentioned by bwfan123

Sat, 19 Jul 2025 15:01:36 GMT

on the surface this is a great achievement - if it holds . alpha-geometry required 1) human formalization of the question and 2) a solver for geometry If the questions were given as-is (without a human formalizing it) and the llm didnt need domain solvers, and the llm was not trained on it already (which happened with frontier math) - I would be impressed. Based on the past history with frontier math [1][2] I remain skeptical. The skeptic in me says that this happens prior to big announcements (GPT-5) to create the hype. Finally, this article shows that LLMs were just bluffing in the usamo…

Paper and discussion history. A mention is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Tue, 01 Jul 2025 10:39:57 GMT

Ask HN: The Proof or Bluff paper. Can "AI" do math?

Over the past 12 months I've seen lots of comments all over the place (here, X, legacy media, blogs etc.) making the case for "AI" performance on the IMO (Math Olympiad) being evidence for continued rapid increases in LLM performance. I've heard my friends who work in AI safety quote these results pretty often whenever they encounter scepticism about the coming AI singularity. It seems to me that these comments stem from the DeepMind results from last summer[0] and February this year[1]. As I understand it, the models they're using for these tasks are very specialised to the task and also…

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Thu, 01 May 2025 01:19:59 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — mentioned by WhitneyLand

Tue, 29 Apr 2025 01:01:33 GMT

China is doing a great job raising doubt about any lead the major US labs may still have. This is solid progress across the board. The new battlefront may be to take reasoning to the level of abstraction and creativity to handle math problems without a numerical answer (for ex: https://arxiv.org/pdf/2503.21934). I suspect that kind of ability will generalize well to other areas and be a significant step toward human level thinking.

Paper and discussion history. A mention is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Wed, 23 Apr 2025 13:20:24 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Tue, 08 Apr 2025 15:34:58 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Mon, 07 Apr 2025 12:20:49 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — mentioned by BlanketLogic

Mon, 07 Apr 2025 05:08:21 GMT

Not the OP but USAMO : USA Math Olympiad. Referred here https://arxiv.org/pdf/2503.21934v1 IMO : International Math Olympiad SOTA : State of the Art OP is probably referring to this referred to this paper here https://arxiv.org/pdf/2503.21934v1. The paper explains out how a rigorous testing revealed abysmal performance of LLMs (results that are at odds with how they are hyped about).

Paper and discussion history. A mention is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — mentioned by simonw

Sun, 06 Apr 2025 21:06:41 GMT

I had to look up these acronyms: - USAMO - United States of America Mathematical Olympiad - IMO - International Mathematical Olympiad - ICPC - International Collegiate Programming Contest Relevant paper: https://arxiv.org/abs/2503.21934 - "Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad" submitted 27th March 2025.

Paper and discussion history. A mention is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — mentioned by AIPedant

Sun, 06 Apr 2025 20:52:07 GMT

Yes, here's the link: https://arxiv.org/abs/2503.21934v1 Anecdotally, I've been playing around with o3-mini on undergraduate math questions: it is much better at "plug-and-chug" proofs than GPT-4, but those problems aren't independently interesting, they are explicitly pedagogical. For anything requiring insight, it's either: 1) A very good answer that reveals the LLM has seen the problem before (e.g. naming the theorem, presenting a "standard" proof, using a much more powerful result) 2) A bad answer that looks correct and takes an enormous amount of effort to falsify. (This is the secret…

Paper and discussion history. A mention is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Sun, 06 Apr 2025 20:08:24 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Sun, 06 Apr 2025 06:33:37 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Thu, 03 Apr 2025 12:21:47 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Tue, 01 Apr 2025 22:44:42 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

6 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Tue, 01 Apr 2025 13:50:35 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad — HN submission

Mon, 31 Mar 2025 23:09:51 GMT

Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad

13 points; 2 discussion replies on HN. Paper and source history. A submission is not an endorsement.