Decision Transformer: Reinforcement Learning via Sequence Modeling · HackerPapers discussions

Decision Transformer: Reinforcement Learning via Sequence Modeling — mentioned by radarsat1

Sun, 05 Oct 2025 21:46:26 GMT

> By treating the outcome reward as a predictable label, we reformulate the RLVR problem into a supervised learning task over a score function parameterized by the policy model and optimized using cross-entropy loss. Isn't this how the Decision Transformer works? I don't see it in the references, so I'll be curious to compare the papers in more depth. https://arxiv.org/abs/2106.01345 > By conditioning an autoregressive model on the desired return (reward), past states, and actions, our Decision Transformer model can generate future actions that achieve the desired return. Lately it has…

Paper and discussion history. A mention is not an endorsement.

Decision Transformer: Reinforcement Learning via Sequence Modeling — mentioned by isaacimagine

Sun, 15 Jun 2025 15:30:59 GMT

No mention of Decision Transformers or Trajectory Transformers? Both are offline approaches that tend to do very well at long-horizon tasks, as they bypass the credit assignment problem by virtue of having an attention mechanism. Most RL researchers consider these approaches not to be "real RL", as they can't assign credit outside the context window, and therefore can't learn infinite-horizon tasks. With 1m+ context windows, perhaps this is less of an issue in practice? Curious to hear thoughts. DT: https://arxiv.org/abs/2106.01345 TT: https://arxiv.org/abs/2106.02039

Paper and discussion history. A mention is not an endorsement.

Decision Transformer: Reinforcement Learning via Sequence Modeling — mentioned by visarga

Thu, 08 Aug 2024 15:31:14 GMT

> if you include the desired score in your prompt, the model will now strive to produce an answer that is consistent with that score But you need a model to generate score from answer, and then fine-tune another model to generate answer conditioned on score. The first time the score is at the end and the second time at the beginning. It's how DecisionTransformer works too, it constructs a sequence of (reward, state, action) where reward conditions on the next action. https://arxiv.org/pdf/2106.01345 By the same logic you could generate tags, including style, author, venue and date. Some will…

Paper and discussion history. A mention is not an endorsement.

Decision Transformer: Reinforcement Learning via Sequence Modeling — mentioned by famouswaffles

Mon, 16 Oct 2023 03:36:31 GMT

>they've solved reinforcement learning? Transformers can do Reinforcement Learning yes. https://arxiv.org/abs/2106.01345 https://arxiv.org/abs/2205.14953 >they can handle continuous domains, like robot motion? Yes they can handle it just fine. Excellently in fact. https://www.deepmind.com/blog/scaling-up-learning-across-man... https://tidybot.cs.princeton.edu/ https://general-pattern-machines.github.io/ https://wayve.ai/thinking/lingo-natural-language-autonomous-... I don't know if anyone is saying they're the best at or have "solved" everything but they can damn near do anything.

Paper and discussion history. A mention is not an endorsement.

Decision Transformer: Reinforcement Learning via Sequence Modeling — HN submission

Thu, 24 Nov 2022 07:43:57 GMT

Decision Transformer: Reinforcement Learning via Sequence Modeling

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Decision Transformer: Reinforcement Learning via Sequence Modeling — mentioned by isaacimagine

Wed, 14 Jul 2021 18:06:49 GMT

PPO is awesome, but so is GPT-style reward-trajectory prediction! http://arxiv.org/pdf/2106.01345v1. As a RL hobbyist, I'd love to see some sort of hybrid approach. Thoughts?

Paper and discussion history. A mention is not an endorsement.

Decision Transformer: Reinforcement Learning via Sequence Modeling — mentioned by sdenton4

Sat, 10 Jul 2021 18:08:33 GMT

The new hotness this month just says 'screw it; treat RL as a supervised sequence problem with causal transformers.' https://arxiv.org/abs/2106.01345 'We consider the following shift in paradigm: instead of training a policy through conventional RL algorithms like temporal difference (TD) learning [6], we will train transformer models on collected experience using a sequence modeling objective. This will allow us to bypass the need for bootstrapping for long term credit assignment – thereby avoiding one of the “deadly triad” [6] known to destabilize RL. It also avoids the need for discounting…

Paper and discussion history. A mention is not an endorsement.

Decision Transformer: Reinforcement Learning via Sequence Modeling — mentioned by T-A

Sat, 03 Jul 2021 17:35:14 GMT

https://arxiv.org/abs/2106.01345

Paper and discussion history. A mention is not an endorsement.

Decision Transformer: Reinforcement Learning via Sequence Modeling — HN submission

Thu, 10 Jun 2021 16:54:16 GMT

Decision Transformer: ReinforcementLearning via Sequence Modeling

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.