Compact and Efficient Indexes for Learned Sparse Retrieval
What it is
This paper proposes methods to significantly reduce the memory footprint of learned sparse retrieval indexes while maintaining high efficiency. It introduces techniques like using medoids for inverted index summaries and compressing forward indexes via vocabulary reordering, bit-packing, and quantization.
The problem it solves
Learned sparse retrieval indexes consume excessive memory and suffer from slower query processing times compared to traditional or optimized dense retrieval methods.
What you could use it for
- Deploying large-scale sparse retrieval models in memory-constrained environments
- Optimizing search latency for learned sparse retrieval systems
- Improving the speed-space trade-off in semantic search engines
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.