HackerPapers

arXiv:2501.12948cs.CL

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, Zhen Zhang

Abstract

General reasoning represents a long-standing and formidable challenge in artificial intelligence. Recent breakthroughs, exemplified by large language models (LLMs) and chain-of-thought prompting, have achieved considerable success on foundational reasoning tasks. However, this success is heavily contingent upon extensive human-annotated demonstrations, and models' capabilities are still insufficient for more complex problems. Here we show that the reasoning abilities of LLMs can be incentivized through pure reinforcement learning (RL), obviating the need for human-labeled reasoning trajectories. The proposed RL framework facilitates the emergent development of advanced reasoning patterns, such as self-reflection, verification, and dynamic strategy adaptation. Consequently, the trained model achieves superior performance on verifiable tasks such as mathematics, coding competitions, and STEM fields, surpassing its counterparts trained via conventional supervised learning on human demonstrations. Moreover, the emergent reasoning patterns exhibited by these large-scale models can be systematically harnessed to guide and enhance the reasoning capabilities of smaller models.

arXiv comments
Journal reference
Nature volume 645, pages 633-638 (2025)
DOI
doi:10.1038/s41586-025-09422-z
Subjects
cs.CLComputation and Language cs.AIArtificial Intelligence cs.LGMachine Learning

On Hacker News

Linked by 31 identifiable commenters in 31 comments across 28 threads, from Jan 2025 to Sep 2026. Found in 1 HN submission from Jan 2025 to Jan 2025. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments28 threads

  1. DeepSeek v4.1 Flash 1 comment

    • OpenAI released their first reasoning model (o1-preview) https://openai.com/index/introducing-openai-o1-preview/ several months before DeepSeek's R1 https://arxiv.org/abs/2501.12948

      yorwbaView on HNParent context

  2. Alibaba to ban Claude Code in workplace over alleged backdoor risks, source says 1 comment

    • Thanks, I wasn't aware of that paper! I think the popular use of distill (one-hot rather than logits) is not due to Anthropic but due to the DeepSeek-R1 tech report: https://arxiv.org/abs/2501.12948

      wren6991View on HNParent context

  3. How LLMs work 1 comment

    • DeepSeek research: - V3 https://arxiv.org/abs/2412.19437 - V2 https://arxiv.org/abs/2405.04434 - R1 https://arxiv.org/abs/2501.12948 (RL applied to ML models was well-known beforehand, but they show it in the open, at scale, on big models) Then, there's the incentive analysis.…

      gobdovanView on HNParent context

  4. There Will Be a Scientific Theory of Deep Learning 1 comment
  5. Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs 1 comment

    • This is a gross simplification of the process - you would typically use order(s) of magnitude more data and compute, and a substantial amount of online reinforcement learning to elicit emergent tool use capabilities. Many recent OSS models have great tech reports where you can…

      desideratumView on HNParent context

  6. Nanolang: A tiny experimental language designed to be targeted by coding LLMs 1 comment

    • Not exactly sure what you are looking for here. That GRPO works? > Group Relative Policy Optimization (GRPO), a variant reinforcement learning (RL) algorithm of Proximal Policy Optimization (PPO) (Schulman et al., 2017). GRPO foregoes the critic model, instead estimating the…

      nlView on HNParent context

  7. Poison Fountain 1 comment

    • Coding and reasoning skills can be improved using machine-driven reinforcement learning. https://arxiv.org/abs/2501.12948

      mrtesthahView on HNParent context

  8. DeepSeek-R1 paper updated from 22 pages to 86 with additional details 1 comment

    • Apologies for the Reddit link, I couldn't find a better source. Direct arxiv link: https://arxiv.org/abs/2501.12948

      ksymphView on HNParent context

  9. The inefficiency of RL, and implications for RLVR progress 1 comment

    • Article talks about all of this and references DeepSeek R1 paper[0], section 4.2 (first bullet point on PRM) on why this is much trickier to do than it appears. [0]: https://arxiv.org/abs/2501.12948

      thegeomasterView on HNParent context

  10. The Bitter Lesson Is Misunderstood 1 comment

    • > We cannot add more compute to a given compute budget C without increasing data D to maintain the relationship. > We must either (1) discover new architectures with different scaling laws, and/or (2) compute new synthetic data that can contribute to learning (akin to dreams).…

      charleshnView on HNParent context

  11. Ask HN: Can anybody clarify why OpenAI reasoning now shows non-English thoughts? 2 comments

    • LLM's aren't humans and there's no reason to expect their "thinking"[1] to behave exactly - or even much - like human thinking. In particular, they don't need to "think" in one language. More concretely, in the DeepSeek R1 paper[2] they observed this "thought language mixing"…

      mindcrimeView on HNParent context

    • This isn’t entirely surprising. Language-model “reasoning” is basically the model internally exploring possibilities in token-space. These models are trained on enormous multilingual datasets and optimized purely for next-token prediction, not language purity. When reasoning…

      diwankView on HNParent context

  12. The Gentle Singularity 1 comment

    • They definitely do strain the neurology and thinking metaphors in that article. But the Dijkstra's algorithm and A* comparisons are the flipside of that same coin. They aren't trying to make it more effective. And definitely not trying to argue for anything AGI related. Either…

      helloplanetsView on HNParent context

  13. In Two Moves, AlphaGo and Lee Sedol Redefined the Future (2016) 1 comment

    • > completely different technology that has effectively faded into the background. If by that you mean reinforcement learning, that's not the case; e.g. see https://arxiv.org/abs/2501.12948

      mastaziView on HNParent context

  14. S1: The $6 R1 Competitor? 1 comment
  15. RLHF Book 1 comment

    • DeepSeek-R1 had an RLHF step in their post-training pipeline (section 2.3.4 of their technical report[1]). In addition, the "reasoning-oriented reinforcement learning" step (section 2.3.2) used an approach that is almost identical to RLHF in theory and implementation. The main…

      alexhutchesonView on HNParent context

  16. Llama.cpp supports Vulkan. why doesn't Ollama? 1 comment

    • What do you mean there is no such thing as R1-1.5b? DeepSeek released a distilled version based on a 1.5B Qwen model with the full name DeepSeek-R1-Distill-Qwen-1.5B, see chapter 3.2 on page 14 of their research article [0]. [0] https://arxiv.org/abs/2501.12948

      dingocatView on HNParent context

  17. Cognitive Reasoning Agents and the Extended Information Filter 1 comment

    • My bet is models can already do all these things. Useful math such as this is likely to emerge and be refined as part of training, assuming they’re discoverable via gradient descent or whatever the cool kids are using these days. Reinforcement learning should then retain and…

      cadamsdotcomView on HNParent context

  18. Exposed DeepSeek database leaking sensitive information, including chat history 2 comments

    • >To build DeepSeek probably requires at least a $1B+ budget. Zero evidence that the above statement is true, and weak evidence (authors' claims) that it is false. Have you read their papers even? https://arxiv.org/html/2412.19437v1#abstract https://arxiv.org/pdf/2501.12948

      fulladderView on HNParent context

    • https://arxiv.org/abs/2501.12948

      kgwgkView on HNParent context

  19. Berkeley Researchers Replicate DeepSeek R1's Core Tech for Just $30: A Small Mod 1 comment

    • There's a lot of conflicting information here. There are two papers: https://arxiv.org/pdf/2412.19437v1 for DeepSeek-V3-Base, which makes claims about 2.8M H800 GPU hours, and https://arxiv.org/pdf/2501.12948 for DeepSeek-R1, which was based on V3-Base but doesn't seem to quote…

      btownView on HNParent context

  20. The Illustrated DeepSeek-R1 1 comment

    • You’re not the only one. It’s not as impressive at coding compared to O1 as people make it out to be and it’s explicitly spelled out in DeepSeek’s R1 paper that they had trouble with improving over DeepSeek-V3: > Software Engineering Tasks: Due to the long evaluation times…

      throwup238View on HNParent context

  21. The impact of competition and DeepSeek on Nvidia 1 comment

    • The R1 paper (https://arxiv.org/pdf/2501.12948) emphasizes their success with reinforcement learning without requiring any supervised data (unlike RLHF for example). They note that this works well for math and programming questions with verifiable answers. What's totally unclear…

      highfrequencyView on HNParent context

  22. How Deepseek R1 Was Trained 1 comment
  23. DeepSeek releases Janus Pro, a text-to-image generator [pdf] 1 comment

    • One thing I'd love to hear opinions on from someone with more free time to read these papers from DeepSeek is: am I right to feel like they're... publishing all their secret sauce? The paper for R1 (1) seems to be pretty clear how they got such good results with so little…

      erulabsView on HNParent context

  24. DeepSeek temporarily limits registration to Chinese mobile numbers 1 comment

    • They seem to have matched OpenAI model capability on a fraction of the resources. R1 is roughly as good as o1 and can be run locally. There are some interesting contributions in the paper too. So I see it as two parts, (1) the money, lots invested, wondering if they will ever…

      verdvermView on HNParent context

  25. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL 1 comment

    • Here [1] is the leaderboard from chabot arena, where users vote on the output of two anonymous models. Deepseek R1 needs more data points- but it already climbed to No 1 with Style control ranking, which is pretty impressive. Link [2] to the result on more standard LLM…

      shihabView on HNParent context

  26. Emerging reasoning with reinforcement learning 1 comment

    • If you check failure section of their paper, they also tried other methods like MCTS and PRM which is what other labs have been obsessing about but couldn't move on from (that includes bigshots). Only team which I am aware which tried verifiable rewards is tulu but they didn't…

      attentionmechView on HNParent context

  27. Citations on the Anthropic API 1 comment

    • This is exactly what we have had working in Langroid since at least a year, so I don't quite get the buzz around this. Langroid's `DocChatAgent` produces granular markdown-style citations, and works with practically any (good enough) LLM. E.g. try running this example script on…

      d4rkp4tternView on HNParent context

  28. DeepSeek and the Effects of GPU Export Controls 2 comments

    • Their paper goes into the details: https://arxiv.org/abs/2501.12948

      GaggiXView on HNParent context

    • I think you haven't been looking too hard in that case. Here is the R1 paper: https://arxiv.org/abs/2501.12948 You can find more papers from the attached author: https://arxiv.org/search/cs?searchtype=author&query=DeepSeek... or title…

      lhlView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.