suryeon123/fusion-model-v2
suryeon123/fusion-model-v2 is a 0.5 billion parameter language model created by suryeon123, merged using the Linear method with Qwen/Qwen2.5-0.5B as its base. This model integrates Qwen/Qwen2.5-0.5B-Instruct, leveraging its instruction-tuned capabilities. With a 32768 token context length, it is designed for efficient processing and generation tasks based on its merged components.
Loading preview...
Model Overview
suryeon123/fusion-model-v2 is a 0.5 billion parameter language model developed by suryeon123. It was created using the Linear merge method from MergeKit, combining the strengths of two foundational models.
Key Characteristics
- Base Model: The merge utilized Qwen/Qwen2.5-0.5B as its primary base.
- Merged Components: It incorporates Qwen/Qwen2.5-0.5B-Instruct, bringing instruction-following capabilities to the model.
- Merge Configuration: The merge process applied a 0.5 weight to both Qwen/Qwen2.5-0.5B-Instruct and Qwen/Qwen2.5-0.5B, with normalization enabled, using
bfloat16data type. - Context Length: The model supports a context length of 32768 tokens, allowing for processing of substantial input sequences.
Potential Use Cases
This model is suitable for applications requiring a compact yet capable language model, particularly where the instruction-following abilities inherited from Qwen/Qwen2.5-0.5B-Instruct are beneficial. Its efficient parameter count makes it a candidate for resource-constrained environments or tasks where rapid inference is critical.