NovaSearch/stella_en_1.5B_v5: A Multi-Dimensional Embedding Model

NovaSearch/stella_en_1.5B_v5 is a 1.5 billion parameter embedding model developed by NovaSearch, leveraging the architectures of Alibaba-NLP/gte-large-en-v1.5 and Alibaba-NLP/gte-Qwen2-1.5B-instruct. It is specifically trained using the MRL method, which enables it to generate embeddings in multiple dimensions: 512, 768, 1024, 2048, 4096, and 8192.

Key Capabilities & Features

• Flexible Embedding Dimensions: Users can choose from various output dimensions, with 1024d offering a strong balance of performance and efficiency, showing only a 0.001 MTEB score difference compared to 8192d.
• Simplified Prompting: The model provides two standardized prompts for common tasks:
  • s2p (sentence-to-passage): "Instruct: Given a web search query, retrieve relevant passages that answer the query.\nQuery: {query}"
  • s2s (sentence-to-sentence): "Instruct: Retrieve semantically similar text.\nQuery: {query}"
• Standard Integration: Compatible with SentenceTransformers and transformers libraries for easy text encoding.
• Optimized for Retrieval: Primarily designed for tasks requiring semantic similarity and passage retrieval.

Usage Considerations

• The recommended sequence length for input is 512 tokens, as the model was trained on datasets of this length.
• While higher dimensions generally yield better performance, 1024d is often sufficient for most applications.
• A technology report and code for Jasper and Stella models are available, detailing their distillation and SOTA embedding techniques (arXiv:2412.19048, GitHub).