Xiaojian9992024/Qwen2.5-THREADRIPPER-Small
Xiaojian9992024/Qwen2.5-THREADRIPPER-Small is a 7.6 billion parameter language model based on the Qwen2.5-7B-Instruct architecture, created by Xiaojian9992024 through a Linear DELLA merge of multiple Qwen-based models. This model excels at Boolean expression tasks, achieving 83.6% accuracy on BBH Boolean Expressions, and supports a 131072 token context length. While demonstrating strong performance in specific logical reasoning, it shows significant limitations in mathematical reasoning and object counting.
Loading preview...
Xiaojian9992024/Qwen2.5-THREADRIPPER-Small: A Merged Qwen2.5-7B Model
This model, developed by Xiaojian9992024, is a 7.6 billion parameter language model built upon the Qwen2.5-7B-Instruct base. It was created using the Linear DELLA merge method, combining several Qwen-based models including fblgit/cybertron-v4-qw7B-MGS, huihui-ai/Qwen2.5-7B-Instruct-abliterated-v3, FreedomIntelligence/HuatuoGPT-o1-7B, and rombodawg/Rombos-LLM-V2.5-Qwen-7b. The merge aimed to integrate diverse capabilities, resulting in a model with a notable strength in specific logical tasks.
Key Capabilities and Performance
- Boolean Expression Champion: Achieves a high normalized accuracy of 83.6% on BBH Boolean Expressions, indicating strong performance in logical truth evaluations.
- Instruction Following: Demonstrates 76.89% strict accuracy on IFEval (0-Shot), suggesting reasonable instruction adherence.
- Multilingual Support: Supports text generation across numerous languages including Chinese, English, French, Spanish, German, and Japanese.
Limitations and Considerations
- Mathematical Reasoning: Exhibits significant weaknesses in mathematical tasks, scoring 0.0% exact match on MATH Lvl 5, making it unsuitable for complex calculations.
- Object Counting & Tracking: Struggles with object counting (33.6% accuracy on BBH Object Counting) and tracking shuffled objects (14.4% accuracy for 7 objects).
- General Knowledge: Achieves 37.3% accuracy on MMLU-PRO and 8.05% on GPQA, indicating limited general factual knowledge compared to its Boolean logic prowess.
Intended Use Cases
- Conversational AI: Suitable for conversational applications where logical consistency in Boolean expressions is critical, though general coherence may vary.
- Text Generation: Capable of generating text, particularly useful for tasks that align with its strengths in logical reasoning.
- Experimental Merging: Serves as an interesting case study for the Linear DELLA merge method and its impact on specialized capabilities.