TuralBayev/axeron-forge-ea776bbc
DeLabs/axeron-forge-ea776bbc is a 7.6 billion parameter language model forged by DeLabs using the Task Arithmetic method. It leverages Qwen/Qwen2.5-7B as its base model and incorporates Qwen/Qwen2.5-7B-Instruct, suggesting an optimization for instruction-following tasks. This model is designed for developers seeking a specialized Qwen2.5-7B variant with enhanced instruction capabilities through model merging.
Loading preview...
Model Overview
DeLabs/axeron-forge-ea776bbc is a 7.6 billion parameter language model created by DeLabs using the forgelm framework. This model was developed with the Task Arithmetic forge method, which combines the strengths of multiple pre-trained models into a single, more specialized model.
Forge Details
The base model for this forge was Qwen/Qwen2.5-7B. The forging process specifically integrated Qwen/Qwen2.5-7B-Instruct, indicating an intent to enhance the model's instruction-following capabilities. The configuration involved merging layers from both the base Qwen2.5-7B and the instruction-tuned variant, with specific weighting applied to the instruction model's contribution.
Key Characteristics
- Architecture: Based on the Qwen2.5-7B family.
- Parameter Count: 7.6 billion parameters.
- Context Length: 32768 tokens.
- Forge Method: Utilizes Task Arithmetic for model merging.
- Source Models: Combines Qwen/Qwen2.5-7B and Qwen/Qwen2.5-7B-Instruct.
When to Use This Model
This model is particularly suitable for use cases where the robust base capabilities of Qwen2.5-7B are desired, combined with improved performance on instruction-tuned tasks. Developers looking for a merged model that benefits from the explicit instruction-following fine-tuning of Qwen2.5-7B-Instruct, without necessarily training from scratch, would find this model beneficial.