Bia/CORAL
CORAL is a 3 billion parameter multi-modal embedding model developed by Bia, built upon Qwen2.5-3B-Instruct. It is designed for interleaved multi-condition semantic retrieval queries, trained using the novel MERIT dataset. The model's loss function incorporates Contrastive Learning, Vision Reconstruction, and Masked Language Modeling to reconstruct both queries and positive samples. CORAL excels at generating embeddings for complex multi-modal search tasks, enabling advanced product and image retrieval based on combined textual and visual conditions.
Loading preview...
CORAL: Multilingual Semantic Retrieval with Interleaved Multi-Condition Query
CORAL (Contrastive Reconstruction for Multimodal Retrieval) is a 3 billion parameter multi-modal embedding model based on Qwen2.5-3B-Instruct. Developed by Bia, CORAL is specifically designed to handle interleaved multi-condition semantic retrieval queries, making it highly effective for complex search scenarios involving both text and images.
Key Capabilities
- Multi-modal Embedding Generation: Produces embeddings from combined text and image inputs for advanced retrieval.
- Interleaved Multi-Condition Queries: Processes queries that integrate multiple conditions across different modalities (e.g., "Find a product with the same brand as <Image 1> and the same style as <Image 2> with a quilted texture.").
- Robust Training Objective: Utilizes a unique loss function comprising Contrastive Learning Loss, Vision Reconstruction Loss, and Masked Language Modeling Loss, enabling the model to reconstruct both queries and their corresponding positive samples.
- Semantic Retrieval: Optimized for finding relevant items based on semantic understanding of complex multi-modal inputs.
Good For
- E-commerce Product Search: Ideal for advanced product discovery where users combine visual and textual cues to find specific items.
- Content-Based Image Retrieval: Enhances image search by allowing detailed, multi-faceted queries.
- Research in Multi-modal AI: Provides a strong baseline for exploring interleaved multi-modal understanding and retrieval tasks.
- Applications requiring nuanced multi-modal matching: Suitable for scenarios where simple keyword or image matching is insufficient.