prithivMLmods/Qwen2.5-7B-DeepSeek-R1-1M

Qwen2.5-7B-DeepSeek-R1-1M Overview

This model is a 7.6 billion parameter merged language model, created by prithivMLmods using the TIES merge method. It leverages Qwen/Qwen2.5-7B-Instruct-1M as its foundational base model. The merge process integrates two distinct models: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B and Qwen/Qwen2.5-7B-Instruct, combining their strengths with equal weight and density.

Key Capabilities

• Optimized Performance: Utilizes bfloat16 precision, normalization, and int8 masking for efficient operation.
• Blended Intelligence: Combines the characteristics of DeepSeek-R1-Distill-Qwen-7B and Qwen2.5-7B-Instruct, potentially offering a broader range of capabilities than either individual model.
• MergeKit Integration: Developed using mergekit, a tool for creating new models by merging existing ones.

Good For

• Experimentation with Merged Models: Ideal for researchers and developers interested in exploring the performance and emergent properties of models created via the TIES merge method.
• Applications requiring diverse capabilities: Suitable for use cases that could benefit from a model that synthesizes the strengths of both DeepSeek's distilled reasoning and Qwen's instruction-following abilities.
• Resource-efficient deployment: The int8 masking and bfloat16 precision suggest an emphasis on optimized performance, making it potentially suitable for environments with computational constraints.