Quantum-Inspired Pairwise Ranking Aggregation via Active LLM Feedback
What it is
This paper introduces LLM-IRA, a method for document reranking that uses Large Language Models to make pairwise judgments on high-disagreement document pairs to learn fusion weights. It then applies these weights to perform a fast, pointwise linear fusion of scores, avoiding the need for expensive full pairwise matrices or listwise prompting.
The problem it solves
Existing ranking aggregation methods often require expensive full pairwise comparisons or listwise prompting, which are computationally costly and slow, while static fusion baselines may not fully exploit the potential of LLMs for dynamic weight learning.
What you could use it for
- Improving retrieval accuracy in RAG systems by aggregating scores from multiple retrievers
- Cost-effective document reranking using limited LLM calls
- Enhancing search engine results on benchmarks like BEIR and TREC Deep Learning
Why it matters
Provides a principled, low-cost method to improve retrieval quality in RAG and search systems by intelligently leveraging LLMs only where necessary, achieving performance comparable to static fusion baselines with significantly fewer LLM calls.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.