DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning · HackerPapers discussions

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by yorwba

Thu, 10 Sep 2026 09:32:15 GMT

OpenAI released their first reasoning model (o1-preview) https://openai.com/index/introducing-openai-o1-preview/ several months before DeepSeek's R1 https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by wren6991

Sun, 12 Jul 2026 07:23:43 GMT

Thanks, I wasn't aware of that paper! I think the popular use of distill (one-hot rather than logits) is not due to Anthropic but due to the DeepSeek-R1 tech report: https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by gobdovan

Sat, 06 Jun 2026 07:10:00 GMT

DeepSeek research: - V3 https://arxiv.org/abs/2412.19437 - V2 https://arxiv.org/abs/2405.04434 - R1 https://arxiv.org/abs/2501.12948 (RL applied to ML models was well-known beforehand, but they show it in the open, at scale, on big models) Then, there's the incentive analysis. If you can see that these models empirically get better with scale, why would you swap the main architecture? Those events will be pretty rare. I'm not saying there's noone cooking a new architecture, just that it is a pretty rare event. And it would have to come from some researchers that would be happy to not publish…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by olalonde

Sat, 25 Apr 2026 10:50:38 GMT

This one is impressive too: https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by desideratum

Sun, 05 Apr 2026 20:26:37 GMT

This is a gross simplification of the process - you would typically use order(s) of magnitude more data and compute, and a substantial amount of online reinforcement learning to elicit emergent tool use capabilities. Many recent OSS models have great tech reports where you can learn more about these kind of things: Kimi 2.5 https://github.com/MoonshotAI/Kimi-K2.5/blob/master/tech_rep... GLM 5 https://arxiv.org/abs/2602.15763 DeepSeek R1 https://arxiv.org/pdf/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by nl

Wed, 21 Jan 2026 06:03:10 GMT

Not exactly sure what you are looking for here. That GRPO works? > Group Relative Policy Optimization (GRPO), a variant reinforcement learning (RL) algorithm of Proximal Policy Optimization (PPO) (Schulman et al., 2017). GRPO foregoes the critic model, instead estimating the baseline from group scores, significantly reducing training resources. By solely using a subset of English instruction tuning data, GRPO obtains a substantial improvement over the strong DeepSeekMath-Instruct, including both in-domain (GSM8K: 82.9% → 88.2%, MATH: 46.8% → 51.7%) and out-of-domain mathematical tasks (e.g.…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by mrtesthah

Sun, 11 Jan 2026 19:11:05 GMT

Coding and reasoning skills can be improved using machine-driven reinforcement learning. https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by ksymph

Wed, 07 Jan 2026 18:26:09 GMT

Apologies for the Reddit link, I couldn't find a better source. Direct arxiv link: https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by thegeomaster

Sun, 30 Nov 2025 13:39:28 GMT

Article talks about all of this and references DeepSeek R1 paper[0], section 4.2 (first bullet point on PRM) on why this is much trickier to do than it appears. [0]: https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by charleshn

Wed, 03 Sep 2025 22:53:44 GMT

> We cannot add more compute to a given compute budget C without increasing data D to maintain the relationship. > We must either (1) discover new architectures with different scaling laws, and/or (2) compute new synthetic data that can contribute to learning (akin to dreams). Of course we can, this is a non issue. See e.g. AlphaZero [0] that's 8 years old at this point, and any modern RL training using synthetic data, e.g. DeepSeek-R1-Zero [1]. [0] https://en.m.wikipedia.org/wiki/AlphaZero [1] https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by diwank

Fri, 13 Jun 2025 02:53:51 GMT

This isn’t entirely surprising. Language-model “reasoning” is basically the model internally exploring possibilities in token-space. These models are trained on enormous multilingual datasets and optimized purely for next-token prediction, not language purity. When reasoning traces or scratchpads are revealed directly (as OpenAI occasionally does with o-series models or DeepSeek-R1-zero), it’s common to see models slip into code-switching or even random language fragments, simply because it’s more token-efficient in their latent space. For example, the DeepSeek team explicitly reported this…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by mindcrime

Fri, 13 Jun 2025 02:32:46 GMT

LLM's aren't humans and there's no reason to expect their "thinking"[1] to behave exactly - or even much - like human thinking. In particular, they don't need to "think" in one language. More concretely, in the DeepSeek R1 paper[2] they observed this "thought language mixing" and did some experiments on suppressing it... and the model results got worse. So I wouldn't personally think of it as an "error", but rather as just an artifact of how these things work. [1]: By this I mean "whatever it is they do that can be thought of as sorta kind roughly analogous to what we generally call…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by helloplanets

Wed, 11 Jun 2025 11:35:10 GMT

They definitely do strain the neurology and thinking metaphors in that article. But the Dijkstra's algorithm and A* comparisons are the flipside of that same coin. They aren't trying to make it more effective. And definitely not trying to argue for anything AGI related. Either way: They're tampering with the inference process, by turning circuits in the LLM on and off, in an attempt to prove that those circuits are related with a specific function. [0] They noticed that circuits related to a token that is only relevant ~8 tokens forward were already activated on the newline token. Instead of…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by mastazi

Thu, 17 Apr 2025 07:07:44 GMT

> completely different technology that has effectively faded into the background. If by that you mean reinforcement learning, that's not the case; e.g. see https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by janalsncm

Wed, 05 Feb 2025 20:48:08 GMT

Section 4.2: Unsuccessful attempts https://arxiv.org/pdf/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by alexhutcheson

Sun, 02 Feb 2025 01:10:46 GMT

DeepSeek-R1 had an RLHF step in their post-training pipeline (section 2.3.4 of their technical report[1]). In addition, the "reasoning-oriented reinforcement learning" step (section 2.3.2) used an approach that is almost identical to RLHF in theory and implementation. The main difference is that they used a rule-based reward system, rather than a model trained on human preference data. If you want to train a model like DeepSeek-R1, you'll need to know the fundamentals of reinforcement learning on language models, including RLHF. [1] https://arxiv.org/pdf/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by dingocat

Fri, 31 Jan 2025 14:02:45 GMT

What do you mean there is no such thing as R1-1.5b? DeepSeek released a distilled version based on a 1.5B Qwen model with the full name DeepSeek-R1-Distill-Qwen-1.5B, see chapter 3.2 on page 14 of their research article [0]. [0] https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by kgwgk

Thu, 30 Jan 2025 12:53:39 GMT

https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by cadamsdotcom

Thu, 30 Jan 2025 00:43:36 GMT

My bet is models can already do all these things. Useful math such as this is likely to emerge and be refined as part of training, assuming they’re discoverable via gradient descent or whatever the cool kids are using these days. Reinforcement learning should then retain and refine anything useful once discovered. If something like a Kalman filter or Gaussian filter is discovered by the training process, it’d be put to use and refined, and maybe this is part of “grokking” phenomenon - a decent explanation is here: https://medium.com/generative-ai-revolution-ai-native-transf.... DeepSeek…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by fulladder

Thu, 30 Jan 2025 00:27:47 GMT

>To build DeepSeek probably requires at least a $1B+ budget. Zero evidence that the above statement is true, and weak evidence (authors' claims) that it is false. Have you read their papers even? https://arxiv.org/html/2412.19437v1#abstract https://arxiv.org/pdf/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by btown

Tue, 28 Jan 2025 18:57:10 GMT

There's a lot of conflicting information here. There are two papers: https://arxiv.org/pdf/2412.19437v1 for DeepSeek-V3-Base, which makes claims about 2.8M H800 GPU hours, and https://arxiv.org/pdf/2501.12948 for DeepSeek-R1, which was based on V3-Base but doesn't seem to quote the cost of its RL steps. https://newsletter.languagemodels.co/p/the-illustrated-deeps... is a really fascinating overview of what those new steps were! The innovation here seems to be that the creation of a cutting-edge reasoning model could be made from a relatively inexpensive base model, regardless of how much RL…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by throwup238

Tue, 28 Jan 2025 07:13:26 GMT

You’re not the only one. It’s not as impressive at coding compared to O1 as people make it out to be and it’s explicitly spelled out in DeepSeek’s R1 paper that they had trouble with improving over DeepSeek-V3: > Software Engineering Tasks: Due to the long evaluation times, which impact the efficiency of the RL process, large-scale RL has not been applied extensively in software engineering tasks. As a result, DeepSeek-R1 has not demonstrated a huge improvement over DeepSeek-V3 on software engineering benchmarks. Future versions will address this by implementing rejection sampling on software…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by highfrequency

Mon, 27 Jan 2025 21:14:01 GMT

The R1 paper (https://arxiv.org/pdf/2501.12948) emphasizes their success with reinforcement learning without requiring any supervised data (unlike RLHF for example). They note that this works well for math and programming questions with verifiable answers. What's totally unclear is what data they used for this reinforcement learning step. How many math problems of the right difficulty with well-defined labeled answers are available on the internet? (I see about 1,000 historical AIME questions, maybe another factor of 10 from other similar contests). Similarly, they mention LeetCode - it looks…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by mfi

Mon, 27 Jan 2025 21:12:08 GMT

Deepseek R1 paper that the blogpost is written around: https://arxiv.org/pdf/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by erulabs

Mon, 27 Jan 2025 20:03:42 GMT

One thing I'd love to hear opinions on from someone with more free time to read these papers from DeepSeek is: am I right to feel like they're... publishing all their secret sauce? The paper for R1 (1) seems to be pretty clear how they got such good results with so little horsepower (see: 'Group Relative Policy Optimization'). Is it not likely that Facebook, OpenAI, etc will just read these papers and implement the tricks? Am I missing something? 1. https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by verdverm

Mon, 27 Jan 2025 16:55:58 GMT

They seem to have matched OpenAI model capability on a fraction of the resources. R1 is roughly as good as o1 and can be run locally. There are some interesting contributions in the paper too. So I see it as two parts, (1) the money, lots invested, wondering if they will ever make it back now (2) China showing impressive results while being handicapped by the West https://arxiv.org/abs/2501.12948 Keep an eye on the effort to reproduce here: https://github.com/huggingface/open-r1 We will see if the (over?) reaction matches reality in time. Media sure loves to whipsaw us all around

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by shihab

Sun, 26 Jan 2025 20:01:16 GMT

Here [1] is the leaderboard from chabot arena, where users vote on the output of two anonymous models. Deepseek R1 needs more data points- but it already climbed to No 1 with Style control ranking, which is pretty impressive. Link [2] to the result on more standard LLM benchmarks. They conveniently placed the results on the first page of the paper. [1] https://lmarena.ai/?leaderboard [2] https://arxiv.org/pdf/2501.12948 (PDF)

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by attentionmech

Sun, 26 Jan 2025 12:05:07 GMT

If you check failure section of their paper, they also tried other methods like MCTS and PRM which is what other labs have been obsessing about but couldn't move on from (that includes bigshots). Only team which I am aware which tried verifiable rewards is tulu but they didn't scaled it up and just left it there. This sort of thing imo is similar to what openAI did with transformer architecture i.e. google invented it but couldn't scale it in the right direction and deepmind got busy with atari games. They had all the pieces still openai could do it. It seems to be it comes down to research…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — HN submission

Sat, 25 Jan 2025 18:39:49 GMT

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

1351 points; 1056 discussion replies on HN. Paper and source history. A submission is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by d4rkp4ttern

Fri, 24 Jan 2025 13:54:41 GMT

This is exactly what we have had working in Langroid since at least a year, so I don't quite get the buzz around this. Langroid's `DocChatAgent` produces granular markdown-style citations, and works with practically any (good enough) LLM. E.g. try running this example script on the DeepSeek R1 paper: https://github.com/langroid/langroid/blob/main/examples/docq... uv run examples/docqa/chat.py https://arxiv.org/pdf/2501.12948 Sample output here: https://gist.github.com/pchalasani/0e2e54cbc3586aba60046b621... https://gist.github.com/pchalasani/0e2e54cbc3586aba60046b621...

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by lhl

Thu, 23 Jan 2025 14:02:09 GMT

I think you haven't been looking too hard in that case. Here is the R1 paper: https://arxiv.org/abs/2501.12948 You can find more papers from the attached author: https://arxiv.org/search/cs?searchtype=author&query=DeepSeek... or title https://arxiv.org/search/?query=DeepSeek&searchtype=title&ab... and go through citations for more. Of course, you could just search by some of the attached authors as well. Daya Guo, the lead author for the R1 paper has 36 papers on Arxiv: https://arxiv.org/search/cs?query=Guo%2C+Daya&searchtype=aut... Besides the papers, DeepSeek has an active Github…

Paper and discussion history. A mention is not an endorsement.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — mentioned by GaggiX

Thu, 23 Jan 2025 14:01:30 GMT

Their paper goes into the details: https://arxiv.org/abs/2501.12948

Paper and discussion history. A mention is not an endorsement.