Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks · HackerPapers discussions

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by seanhunter

Tue, 07 Jul 2026 13:58:11 GMT

Don't blame me, blame the people who invented the concept and wrote the original RAG paper. https://arxiv.org/pdf/2005.11401

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by peter_d_sherman

Wed, 16 Oct 2024 02:04:19 GMT

I think the key quote is: >"Claude's extensive context window has also transformed their approach to handling large codebases. When the 200K context window was released, Hedley notes they "ripped out the entire RAG and just put it in the context window instead and it went from 60 percent accuracy to 98." RAG = Retrieval Augmented Generation Related: What is retrieval-augmented generation, and what does it do for generative AI?: https://github.blog/ai-and-ml/generative-ai/what-is-retrieva... Retrieval Augmented Generation: https://en.wikipedia.org/wiki/Retrieval-augmented_generation…

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — HN submission

Wed, 08 May 2024 15:19:40 GMT

Show HN: I made a better Perplexity for developers

Hi HN, I am Jiayuan, and I'm here to introduce a tool we've been building over the past few months: Devv (https://devv.ai). In simple terms, it is an AI-powered search engine specifically designed for developers. Now, you might ask, with so many AI search engines already available—Perplexity, You.com, Phind, and several open-source projects—why do we need another one? We all know that Generative Search Engines are built on RAG (Retrieval-Augmented Generation)[1] combined with Large Language Models (LLMs). Most of the products mentioned above use indexes from general search engines (like…

185 points; 75 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — HN submission

Tue, 07 May 2024 16:46:18 GMT

Show HN: Devv – AI search engine for devs, built on a custom search index

6 points; 7 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by _akhe

Sun, 31 Mar 2024 14:21:36 GMT

Ragdoll is based on RAG - the knowledge is scoped to documents you define, and is uncensored, even if the model you're using has censorship. For example, if you provided some raunchy URLs as Lara Croft's source of knowledge, you could have realistic conversations about the content there, however sexually unshackled they may be. To test it, I just searched for some known banned words in character.ai and tried them in character.ai. It refused to answer, tried to convince me to not talk about it, and even clarified there's a policy against these words, breaking the 4th wall. I asked the same…

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by regularfry

Sun, 31 Dec 2023 10:01:20 GMT

Sometimes the terminology is around but not well distributed. I'm not sure this is the paper that coined it, but here's an example from 2020: https://arxiv.org/abs/2005.11401

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — HN submission

Fri, 01 Dec 2023 19:54:43 GMT

RAG: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by filterfiber

Thu, 30 Nov 2023 17:54:42 GMT

Yeah that's a fairly well studied one. Most of these techniques are rather "lossy" compared to extending the context window. The most likely "real solution" is going to be using various tricks and finetuning on higher context lengths to just extend the context window. Here's a bunch of other related methods, Summarizing context - https://arxiv.org/abs/2305.14239 continuous finetuning - https://arxiv.org/pdf/2307.02839.pdf retrieval augmented generation - https://arxiv.org/abs/2005.11401 knowledge graphs - https://arxiv.org/abs/2306.08302 augmenting the network a side network…

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by wazer5

Sun, 05 Nov 2023 06:26:47 GMT

Retrieval-augmented generation https://arxiv.org/abs/2005.11401

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by simonw

Thu, 26 Oct 2023 04:04:10 GMT

> You could just sum it up for us all rather than do a divert to your blog? Why? Have links gone out of fashion? I even linked directly to the relevant section rather than linking to the top of the page. The paper that coined the term used the hyphen, though I think I prefer it without: https://arxiv.org/abs/2005.11401

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by sprobertson

Fri, 06 Oct 2023 17:22:08 GMT

Because "RAG" is shorter than "use a vector db/embeddings coupled with LLM to generate better answers" But better answer: "retrieval augmented" has been used in papers since at least 2018 [1], the full "RAG" acronym since at least 2020 [2], and it was recently repopularized when the GPT3+ hype wave realized they were doing the same thing. [1] https://arxiv.org/abs/1808.03430 [2] https://arxiv.org/abs/2005.11401

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by ingridpan

Fri, 25 Aug 2023 18:24:05 GMT

Research: https://arxiv.org/abs/2005.11401 An implementation on Hugging Face: https://huggingface.co/docs/transformers/model_doc/rag

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by chank

Tue, 15 Aug 2023 12:10:50 GMT

this concept already exists and is in practice at many companies that require knowledge driven results. https://arxiv.org/abs/2005.11401

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by lqhl

Mon, 24 Jul 2023 02:24:51 GMT

This idea is a simplified version of Retrieval-Augmented Generation (RAG), and RAG has been studied in various research papers, such as the one available at https://arxiv.org/abs/2005.11401

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by nonfamous

Wed, 21 Jun 2023 02:57:55 GMT

I was taken aback by the terminology too. Isn’t Retrieval-Augmented Generation the standard nomenclature for this pattern? https://arxiv.org/abs/2005.11401

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by HarHarVeryFunny

Thu, 11 May 2023 20:23:54 GMT

Pretty sure it's not "real" (model) context width. Another wide context model is MosaicML's MPT-7B-StoryWriter-65k+ which they are describing as having a context width of 65k, but then give a bit more detail to say they are using ALiBi - a type of positional encoding that allows longer contexts at inference time than training (i.e beyond the real context width of the model). For these types of "extended context" models to actually reason over inputs longer than the native context width of the model, I assume that there is indeed some sort of vector DB trickery - maybe paging thru the input to…

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by pyth0

Thu, 11 May 2023 17:41:04 GMT

This more or less is already a thing and it's called RAG [1][2]. It essentially allows you to have a database of embeddings (in this case your book) from which a model can pull knowledge from while producing answers. As for the standard operation of these generative models, the context window is the only working memory it has and so it must see the entire text each time. [1] https://arxiv.org/abs/2005.11401 [2] https://huggingface.co/docs/transformers/model_doc/rag

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by salamo

Mon, 23 Jan 2023 02:57:17 GMT

There was a 2020 paper called Retrieval Augmented Generation that describes this technique too [1]. That paper used BERT embeddings which are a little cheaper and uses BART as a generator. One of the common failure modes of RAG (and I assume your technique as well) is hallucination: basically, making stuff up that isn't in any of the docs [2]. [1] https://arxiv.org/abs/2005.11401 [2] https://parl.ai/projects/hallucination/

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — mentioned by janalsncm

Tue, 10 Jan 2023 23:51:06 GMT

Thats true for the physics and math but some of the things are updated all the time. For example you can get a current weather report. And they have structured data about movies, TV shows, music, and notable people [1]. Every time any country has an election you’re going to retrain your model? That gets really expensive really fast. On top of that, the training process isn’t that trustworthy. There’s no guarantee your model won’t accidentally say that e.g. Obama is the president. All of this is to say that the best path forward is to translate questions into queries to an auditable knowledge…

Paper and discussion history. A mention is not an endorsement.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — HN submission

Sun, 14 Mar 2021 19:47:01 GMT

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.