TuralBayev/axeron-forge-1c31cf27
DeLabs/axeron-forge-1c31cf27 is a 7.6 billion parameter language model created by DeLabs using the Task Arithmetic forge method. Based on Qwen/Qwen2.5-7B, this model integrates capabilities from Qwen/Qwen2.5-7B-Instruct. It is designed for general language understanding and generation tasks, leveraging its Qwen2.5 foundation for robust performance.
Loading preview...
Model Overview
DeLabs/axeron-forge-1c31cf27 is a 7.6 billion parameter language model developed by DeLabs using the forgelm framework. This model was constructed via the Task Arithmetic forge method, which combines the strengths of multiple pre-trained models.
Key Characteristics
- Base Model: Built upon the robust Qwen/Qwen2.5-7B architecture.
- Forging Method: Utilizes the Task Arithmetic technique, as detailed in the research paper "Task Arithmetic", to merge model functionalities.
- Integrated Capabilities: Incorporates features from Qwen/Qwen2.5-7B-Instruct, enhancing its instruction-following and conversational abilities.
- Configuration: The forging process involved specific layer ranges (0-28) from both the base and instruct models, with a weight of 1.0 applied to the instruct model's parameters for integration.
Use Cases
This model is well-suited for applications requiring a general-purpose language model with enhanced instruction-following capabilities, benefiting from the combined strengths of its Qwen2.5 foundations.