Fine-Tuning LLaMA for Multi-Stage Text Retrieval — mentioned by thatjoeoverthr

Fri, 29 Aug 2025 10:39:44 GMT

There's good reasons to do this. Embedding similarity is _not_ a reliable method of determining relevance. I did some measurements and found you can't even really tell if two documents are "similar" or not. Here: https://joecooper.me/blog/redundancy/ One common way is to mix approaches. e.g. take a large top-K from ANN on embeddings as a preliminary shortlist, then run a tuned LLM or cross encoder to evaluate relevance. I'll link here these guys' paper which you might find fun: https://arxiv.org/pdf/2310.08319 At the end of the day you just want a way to shortlist and focus information that's…

Paper and discussion history. A mention is not an endorsement.

Fine-Tuning LLaMA for Multi-Stage Text Retrieval · HackerPapers discussions

Fine-Tuning LLaMA for Multi-Stage Text Retrieval — mentioned by thatjoeoverthr