Sentence Transformers has added support for multi‑vector (late interaction) embedding models. In this setup each token of an input sequence is encoded into its own vector representation by the underlying transformer encoder. At retrieval time similarity between a query and a document is computed using a late‑interaction function such as MaxSim, which compares every query token vector against all document token vectors and aggregates the highest matches. This contrasts with the traditional approach of pooling token vectors into a single dense embedding before similarity scoring.
Key technical details remain aligned with the existing Sentence Transformers framework: the model architecture reuses standard transformer backbones (e.g., BERT‑base, RoBERTa‑large) and therefore preserves the typical 512‑token context window. The output is a set of token‑level embeddings rather than a single pooled vector, and the licensing follows that of the base model (commonly Apache 2.0 or MIT).
- Multi‑vector representation per token
- Late‑interaction similarity (MaxSim) computed at inference
- No change to maximum sequence length (512 tokens)
- Compatible with existing Sentence Transformers API
- License inherited from the underlying transformer model
Why this matters
The late‑interaction mechanism allows the model to capture fine‑grained token‑level alignment between query and document, which has been shown to improve retrieval effectiveness on benchmarks such as MS MARCO without requiring the computational overhead of full cross‑encoders. By integrating this approach into the widely used Sentence Transformers library, practitioners can access a stronger dense‑retrieval baseline while retaining the simplicity of a single‑model API and the same licensing terms as the base transformer. This bridges the gap between pure dense vectors and more expensive sparse or cross‑encoder methods, offering a practical trade‑off for large‑scale semantic search applications.