Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting · HackerPapers discussions

Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting — mentioned by anishathalye

Mon, 15 Sep 2025 14:55:29 GMT

That was a small self-contained example that fit above the fold in the README (and fwiw even last year’s models like GPT-4o give the right output there). That `sort` is based on pairwise comparisons, which is one of the best ways you can do it in terms of accuracy (Qin et al., 2023: https://arxiv.org/abs/2306.17563). I think there are many real use cases where you might want a semantic sort / semantic data processing in general, when there isn’t a deterministic way to do the task and there is not necessarily a single right answer, and some amount of error (due to LLMs being imperfect) is…

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting — mentioned by markbergz

Mon, 10 Mar 2025 20:40:25 GMT

For anyone interested in these LLM pairwise sorting problems, check out this paper: https://arxiv.org/abs/2306.17563 The authors discuss the person 1 / doc 1 bias and the need to always evaluate each pair of items twice. If you want to play around with this method there is a nice python tool here: https://github.com/vagos/llm-sort

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting — mentioned by vagozino

Thu, 13 Feb 2025 18:35:38 GMT

This technique might be more efficient but can be highly correlated to the order of the input text. The paper [1] I mention in the repo touches upon such methods briefly. [1]: https://arxiv.org/abs/2306.17563

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting — mentioned by dollo_7

Sun, 14 Apr 2024 18:11:51 GMT

I'm not sure if that is a metric you can rely on. LLMs are very sensitive to the position of your item lists along the context, paying extra attention at the beginning and the end of those list. See the listwise approach at "Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting", https://arxiv.org/abs/2306.17563

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting — HN submission

Sat, 08 Jul 2023 01:47:34 GMT

Large Language Models Are Effective Text Rankers with Pairwise Ranking Prompting

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.