CloudGoat/Baphomet-1.0-4B
CloudGoat/Baphomet-1.0-4B is a 4.5 billion parameter multimodal language model based on Qwen3.5-4B, created by CloudGoat through a linear merge of eight task-vectors. This model leverages a unique merging technique, inspired by a 2025 paper on model merging, to combine capabilities from various high-quality community fine-tuned Qwen3.5-4B versions. It features a 32,768 token context length and supports vision inputs, making it suitable for multimodal applications and tasks requiring extended context, though no performance claims are made as benchmarks are in progress.
Loading preview...
Overview
Baphomet-1.0-4B is a 4.5 billion parameter multimodal model developed by CloudGoat, built upon the Qwen/Qwen3.5-4B base. It was created using a linear merge of eight distinct task-vectors, each derived from high-quality community fine-tuned versions of Qwen3.5-4B. This merging approach is based on the principles of Task Arithmetic, as outlined in a 2025 paper, which suggests it consistently improves performance in "in-the-wild" models.
Key Capabilities
- Multimodal Support: Inherits vision capabilities from its base architecture, supporting image inputs alongside text.
- Extended Context Window: Features a substantial context length of 32,768 tokens, enabling processing of longer inputs and maintaining conversational coherence over extended interactions.
- Modular Design: Created by combining specialized task-vectors, potentially integrating diverse functionalities from models like
Agents-A1-4B,DeepSeek-V4-Pro,Qwopus3.5-4B-Coder, and others. - Qwen3.5 Architecture: Utilizes the
Qwen3_5ForConditionalGenerationarchitecture, including a248,320vocabulary size and a repeating pattern of linear and full attention layers.
Good For
- Multimodal Applications: Ideal for tasks requiring both image and text understanding, such as visual question answering or image description generation.
- Long-Context Processing: Suitable for applications that benefit from a large context window, like summarizing lengthy documents, complex code analysis, or extended dialogue systems.
- Experimental Merging Research: Provides a practical example of linear model merging based on recent research, offering a foundation for further experimentation in combining model capabilities.
Note: As of release, benchmark evaluations are in progress, and no specific performance claims are made. Users should be aware that the model's characteristics are a result of a linear merge, and specific capabilities from individual donor models are not guaranteed to be preserved.