RoBERTa: A Robustly Optimized BERT Pretraining Approach · HackerPapers discussions

RoBERTa: A Robustly Optimized BERT Pretraining Approach — mentioned by ninjin

Mon, 27 Nov 2023 03:50:47 GMT

There is a lot of interesting discussion already, but I think there is also some (perhaps unnecessary) disagreements regarding licensing arising from what I believe are the very different backgrounds in the community. Thus, I will attempt to summarise my own perspective as someone that has been active in the artificial intelligence community for nearly 15 years at this point. Firstly, I will state that I am in full agreement with LeCun on what he states in the video about the need for AI systems such as LLMs to be open [1]. I am not sure if I entirely agree with his very high optimism…

Paper and discussion history. A mention is not an endorsement.

RoBERTa: A Robustly Optimized BERT Pretraining Approach — mentioned by vikashrungta

Wed, 08 Feb 2023 16:19:46 GMT

I posted a list of papers on twitter, and will be posting a summary for each of them as well. here is the list https://twitter.com/vrungta/status/1623343807227105280 Unlocking the Secrets of AI: A Journey through the Foundational Papers by @vrungta (2023) 1. "Attention is All You Need" (2017) - https://arxiv.org/abs/1706.03762 (Google Brain) 2. "Generative Adversarial Networks" (2014) - https://arxiv.org/abs/1406.2661 (University of Montreal) 3. "Dynamic Routing Between Capsules" (2017) - https://arxiv.org/abs/1710.09829 (Google Brain) 4. "Unsupervised Representation Learning with Deep…

Paper and discussion history. A mention is not an endorsement.

RoBERTa: A Robustly Optimized BERT Pretraining Approach — HN submission

Mon, 29 Jul 2019 12:30:18 GMT

RoBERTa: A Robustly Optimized Bert Pretraining Approach

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.