Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge — mentioned by armcat

Sun, 10 Mar 2024 17:54:55 GMT

I've worked with EXACT this type of problem and for me RAG works perfectly well - it may seem "clumsy" as you put it in terms of trying to engineer or optimize the indexing, augmentation, and retrieval techniques, but it's worth it. I would claim that going down the finetuning route lends to a significantly higher probability of going astray at a MUCH larger cost. There have been some attempts to compare RAG vs FT approaches, I would recommend this paper: https://arxiv.org/abs/2403.01432

Paper and discussion history. A mention is not an endorsement.

Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge · HackerPapers discussions

Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge — mentioned by armcat