Multi-vector embedding models provide high expressive power for retrieval but create significant storage overhead due to the volume of document vectors. Token pooling reduces this size by merging vectors, and pooling-aware training allows encoders to preserve crucial information during this compression. The approach significantly improves retrieval performance at constrained storage budgets compared to naive pooling.
Key points
Multi-vector models improve search quality by mapping individual tokens to vectors, but they scale poorly in storage requirements.
Token pooling compresses document representations by merging vectors, which allows for smaller index sizes.
Pooling-aware training optimizes the encoder to maintain retrieval accuracy despite vector reduction.
Experimental results show that training-aware pooling achieves higher nDCG scores than baseline models at the same vector compression ratio.
Combining pooling with existing compression methods can further optimize storage efficiency without significant quality loss.