REALM: Retrieval-Augmented Language Model Pre-Training · HackerPapers discussions

REALM: Retrieval-Augmented Language Model Pre-Training — mentioned by shawntan

Mon, 06 Feb 2023 20:41:38 GMT

Like the earlier responses said, if it is retrieving information from an external data source, that would be a much easier way to update things. REALM (https://arxiv.org/abs/2002.08909) comes to mind as an example of active work in this area. Updating a monolithic LLM seems like a harder problem at the moment, and probably would fall under the umbrella of "continual learning", though that sub-field would have their own subdivisions of different methods. As for how they would potentially perform, those results would be mostly empirically measured, but neither approach would completely remove…

Paper and discussion history. A mention is not an endorsement.

REALM: Retrieval-Augmented Language Model Pre-Training — mentioned by kelseyfrog

Wed, 14 Dec 2022 22:53:41 GMT

See REALM[1] for some older(2 years) work on this idea. 1. https://arxiv.org/abs/2002.08909

Paper and discussion history. A mention is not an endorsement.

REALM: Retrieval-Augmented Language Model Pre-Training — mentioned by skybrian

Sun, 16 Aug 2020 21:21:26 GMT

Darn, based on the title, I was hoping for an overview of recent research. Lots of people are having fun playing with GPT-3 or AI Dungeon, myself included, but it seems like there is other interesting research going on like the REALM paper [1], [2]. What should I be reading? Why aren't people talking about REALM more? I'm no expert, but it seems like keeping the knowledge base outside the language model has a lot going for it? [1] https://ai.googleblog.com/2020/08/realm-integrating-retrieva... [2] https://arxiv.org/abs/2002.08909

Paper and discussion history. A mention is not an endorsement.

REALM: Retrieval-Augmented Language Model Pre-Training — HN submission

Fri, 21 Feb 2020 19:10:01 GMT

Realm: Retrieval-Augmented Language Model Pre-Training

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.