venkateshchsagalm/SagaLM-slm2
SagaLM-slm2 is a 7.6 billion parameter language model developed by venkateshchsagalm, fine-tuned from Qwen/Qwen2.5-7B-Instruct using QLoRA and Supervised Fine-Tuning. It is designed for instruction following, conversations, reasoning, mathematics, and coding tasks, with a focus on maintaining its unique SagaLM identity. The model supports a context length of 32768 tokens and is optimized for diverse general-purpose applications.
Loading preview...
SagaLM-slm2 Overview
SagaLM-slm2 is a 7.6 billion parameter large language model developed by venkateshchsagalm, built upon the Qwen/Qwen2.5-7B-Instruct base architecture. It has been fine-tuned using a combination of QLoRA and Supervised Fine-Tuning (SFT) to enhance its capabilities across various domains. The model maintains compatibility with the Qwen2/Qwen2.5 Transformers implementation, ensuring seamless integration within the existing ecosystem.
Key Training Details
- Base Model: Qwen/Qwen2.5-7B-Instruct
- Fine-tuning Method: QLoRA / Supervised Fine-Tuning (SFT)
- Context Length: While the base model's training context was 2,048 tokens, the model supports a 32768 token context length.
- Training Data: The training mixture included diverse datasets covering instruction following, general conversations, logical reasoning, mathematical problems, and coding tasks.
- Identity Grounding: A specific focus during training was to ensure the model identifies itself as "SagaLM" rather than other common assistants.
Recommended Use Cases
SagaLM-slm2 is well-suited for applications requiring a versatile language model capable of:
- Instruction Following: Executing complex instructions accurately.
- Conversational AI: Engaging in natural and coherent dialogues.
- Reasoning Tasks: Solving logical puzzles and performing analytical tasks.
- Mathematical Problem Solving: Assisting with various mathematical computations and explanations.
- Code Generation: Generating and understanding code snippets across different programming languages.