ForSureTesterSim/QwenR1-7B-Breadcrumbs-TIES
ForSureTesterSim/QwenR1-7B-Breadcrumbs-TIES is a 7.6 billion parameter language model based on the DeepSeek-R1-Distill-Qwen-7B architecture, created by ForSureTesterSim using the Model Breadcrumbs with TIES merge method. This model integrates components from Polaris-7B-Preview, a local 'padded-maestro' model, and AceReason-Nemotron-7B. It is designed to leverage the strengths of its merged constituents, offering a unique blend of capabilities for various language tasks.
Loading preview...
Model Overview
ForSureTesterSim/QwenR1-7B-Breadcrumbs-TIES is a 7.6 billion parameter language model developed by ForSureTesterSim. It was created using the Model Breadcrumbs with TIES merge method, a technique designed to combine the strengths of multiple pre-trained language models. The base model for this merge is deepseek-ai/DeepSeek-R1-Distill-Qwen-7B, providing a robust foundation.
Merge Details
This model is a composite of several distinct models, carefully integrated to enhance overall performance. The merged components include:
- POLARIS-Project/Polaris-7B-Preview
- A local model referred to as
./padded-maestro - nvidia/AceReason-Nemotron-7B
The merge process utilized specific parameters for the breadcrumbs_ties method:
- Density: 0.15 (retaining the top 15% of parameters, trimming bottom 85% of SFT noise)
- Gamma: 0.99 (trimming the top 1% to address extreme RL outliers)
- Weight: 1.0
This configuration aims to optimize the model by selectively incorporating and refining features from its constituent parts, resulting in a specialized language model with a 32768 token context length.