RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture · HackerPapers discussions

RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture — mentioned by sigmoid10

Mon, 15 Apr 2024 07:43:45 GMT

See e.g. this paper from Microsoft: https://arxiv.org/abs/2401.08406

Paper and discussion history. A mention is not an endorsement.

RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture — mentioned by rasbt

Sun, 28 Jan 2024 16:03:46 GMT

That's a good point. I may briefly mention RAG-like systems and add some literature references on this, but I am bit hesitant to give general advice because it's heavily project-dependent in my opinion. It usually also comes down in what form the client has the data and whether referencing from a database or documentation is desired or not. The focus of chapter 6+7 is also instruction-finetuning and alignment rather than finetuning for knowledge. The latter goal is best achieved done via pretraining (as opposed to finetuning) imho. In any case, I just read this interesting case study last…

Paper and discussion history. A mention is not an endorsement.

RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture — mentioned by ignoramous

Mon, 22 Jan 2024 19:54:47 GMT

From what I gather, fine-tuning is unreasonably effective [0] because in-context learning really depends on how powerful the underlying model is and just how you do RAG (process queries, retrieve embeddings, rank outcomes, etc [1]). Per this paper I read, fine-tuning may add new domain knowledge (but as another commenter pointed out, knowledge is better represented from data of the pre-training stage) or boost specific knowledge; while RAG is limited to boosting only; nevertheless, both techniques turn out to be similarly capable with different trade-offs [2]. -- [0] Fast.ai: Can Models learn…

Paper and discussion history. A mention is not an endorsement.

RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture — HN submission

Thu, 18 Jan 2024 22:54:38 GMT

RAG vs. Fine-Tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture — HN submission

Thu, 18 Jan 2024 12:33:37 GMT

RAG vs. Fine-Tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.