Sakalti/model-3
Sakalti/model-3 is a 7.6 billion parameter language model, created by Sakalti using the TIES merge method. It is based on Qwen/Qwen2.5-7B and incorporates Qwen/Qwen2.5-7B-Instruct, offering a 32768-token context length. This model is a merge of pre-trained language models, designed for general language understanding and generation tasks, with its performance evaluated across various benchmarks including IFEval and MMLU-PRO.
Loading preview...
Overview
Sakalti/model-3 is a 7.6 billion parameter language model, developed by Sakalti through the TIES merge method. It leverages the robust base of Qwen/Qwen2.5-7B and integrates the instruction-tuned capabilities of Qwen/Qwen2.5-7B-Instruct. This model is designed for general-purpose language tasks, benefiting from a substantial 32768-token context window.
Key Capabilities
- Merged Architecture: Utilizes the TIES merge method to combine strengths of multiple pre-trained models.
- Instruction Following: Incorporates an instruction-tuned model, enhancing its ability to follow directives.
- Extended Context: Supports a 32768-token context length, suitable for processing longer inputs and generating coherent extended outputs.
Performance Highlights
Evaluated on the Open LLM Leaderboard, Sakalti/model-3 demonstrates a balanced performance across various benchmarks. Detailed results are available here.
- IFEval (0-Shot): Achieves 62.64, indicating proficiency in instruction following.
- MMLU-PRO (5-shot): Scores 38.39, reflecting its general knowledge and reasoning capabilities.
- BBH (3-Shot): Reaches 36.32, showing its ability to handle complex reasoning tasks.
Good for
- General Language Generation: Suitable for a wide range of text generation tasks, from creative writing to summarization.
- Instruction-Based Applications: Ideal for applications requiring the model to adhere to specific instructions or prompts.
- Research and Experimentation: Provides a solid base for further fine-tuning or architectural exploration due to its merged nature.