ShinojiResearch/Senku-70B-Full
ShinojiResearch/Senku-70B-Full is a 69 billion parameter language model fine-tuned from a dequantized version of miqu-70b-sf, which is based on an alleged early Mistral-70B. This model is specifically fine-tuned on the Slimorca dataset and achieves an EQ-Bench score of 85.09 using the ChatML prompt format. It is notable for being the first open-weight model to surpass GPT-4 on EQ-Bench, making it suitable for complex reasoning and general conversational AI tasks.
Loading preview...
ShinojiResearch/Senku-70B-Full Overview
Senku-70B-Full is a 69 billion parameter language model developed by ShinojiResearch. It is a fine-tuned version of miqu-70b-sf, which itself is a dequantized variant of an alleged early Mistral-70B model. The model was trained using the Axolotl framework on the Slimorca dataset, with a sequence length of 8192 tokens and a learning rate of 0.0002.
Key Capabilities & Performance
- Exceptional Reasoning: Achieves an impressive 85.09 EQ-Bench score using the ChatML prompt template, notably dethroning GPT-4 on this benchmark. This indicates strong performance in complex reasoning tasks.
- Robust General Performance: On the Open LLM Leaderboard, it demonstrates solid average performance with a score of 75.44. Specific benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 71.50
- HellaSwag (10-Shot): 87.88
- MMLU (5-Shot): 75.20
- GSM8k (5-Shot): 71.34
- Optimized Prompting: The model performs best with the ChatML prompt format, which also resolves a known stop token issue present in the base Miqu dequant model.
When to Use This Model
- Advanced Reasoning Applications: Ideal for use cases requiring high-level reasoning and problem-solving, as evidenced by its leading EQ-Bench score.
- General Conversational AI: Its strong performance across various benchmarks makes it suitable for a wide range of conversational and instruction-following tasks.
- Research and Development: Offers a powerful base for further fine-tuning or experimentation, especially given its lineage from an alleged Mistral-70B variant.