Stabilizing Transformers for Reinforcement Learning — mentioned by ctoth

Mon, 10 Feb 2020 20:54:28 GMT

There's interesting research out of Deepmind on two parts of this, which are using the Transformer model in Reinforcement Learning contexts[0] and creating textual GANs[1]. As you are probably aware, GANs are one of the important tools that have driven forward image synthesis and until recently it was impossible to apply them to text, so I expect this to push us quite a bit forward. There's also ongoing work in the selection of the metric to use to evaluate the generated text, and discriminate between human and machine-generated text. [0] https://arxiv.org/abs/1910.06764 [1]…

Paper and discussion history. A mention is not an endorsement.

Stabilizing Transformers for Reinforcement Learning — mentioned by lucidrains

Wed, 27 Nov 2019 18:53:35 GMT

Another work in the opposite direction, introducing gating in Transformer-xl https://arxiv.org/abs/1910.06764

Paper and discussion history. A mention is not an endorsement.

Stabilizing Transformers for Reinforcement Learning · HackerPapers discussions

Stabilizing Transformers for Reinforcement Learning — mentioned by ctoth

Stabilizing Transformers for Reinforcement Learning — mentioned by lucidrains