RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering — mentioned by mskar

Wed, 05 Mar 2025 15:59:20 GMT

We measured PaperQA2 (https://github.com/Future-House/paper-qa) against the science portion of the RAG-Arena benchmark (https://arxiv.org/abs/2407.13998), it's the first time we've compared PaperQA2 against other systems based on Cohere or Contextual.ai. PaperQA2 achieves a 12.4% higher score than Contextual.ai on the same dataset (1,404 questions and 1.7M documents). We're thrilled about this because it's open source, and getting better every day -- check out the code to reproduce this result in our cookbook here: https://futurehouse.gitbook.io/futurehouse-cookbook/paperqa/....

Paper and discussion history. A mention is not an endorsement.

RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering · HackerPapers discussions

RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering — mentioned by mskar