Emova-ollm/Qwen2.5-3B-Instruct_add_speech_token_4096_nostrip
Emova-ollm/Qwen2.5-3B-Instruct_add_speech_token_4096_nostrip is a 3.1 billion parameter instruction-tuned causal language model, based on Qwen2.5-3B-Instruct, developed by Emova-ollm. This model has been specifically modified by inserting 4096 speech tokens into its vocabulary, enabling end-to-end omni-modal alignment for the EMOVA series of models. It is designed for use as an initialization checkpoint in the Stage 2: Omni-modal text-centric alignment phase of EMOVA training, facilitating advanced speech and language integration.
Loading preview...
Overview
This model, Emova-ollm/Qwen2.5-3B-Instruct_add_speech_token_4096_nostrip, is a specialized variant of the Qwen2.5-3B-Instruct base model, developed by Emova-ollm. It features 3.1 billion parameters and a context length of 32768 tokens. Its primary distinction lies in the integration of 4096 speech tokens into its vocabulary, a crucial step for achieving end-to-end omni-modal alignment within the EMOVA framework. This modification prepares the model for advanced multimodal tasks, particularly those involving speech and text.
Key Capabilities
- Speech Token Integration: Incorporates 4096 speech tokens, enabling multimodal processing capabilities.
- EMOVA Training Component: Serves as a foundational checkpoint for the "Stage 2: Omni-modal text-centric alignment" in the EMOVA training pipeline.
- Instruction-Tuned Base: Built upon the robust Qwen2.5-3B-Instruct architecture, retaining its language understanding and generation strengths.
Good for
- Developers working on the EMOVA project requiring a speech-token-augmented LLM.
- Research into omni-modal language models that integrate speech and text.
- Customizing and fine-tuning for applications requiring advanced speech-text alignment and processing.