lewtun/mistral-7b-sft-ultrachat-arithmo-25
lewtun/mistral-7b-sft-ultrachat-arithmo-25 is a fine-tuned Mistral-7B-v0.1 model, developed by lewtun, optimized for general conversational tasks and arithmetic reasoning. This 7 billion parameter model was trained on a combination of the UltraChat and Arithmo (25%) datasets. It demonstrates a validation loss of 0.8943, indicating its proficiency in processing and generating human-like text, particularly in mathematical contexts.
Loading preview...
Model Overview
lewtun/mistral-7b-sft-ultrachat-arithmo-25 is a 7-billion parameter language model derived from the mistralai/Mistral-7B-v0.1 architecture. This model has undergone supervised fine-tuning (SFT) using a specialized dataset blend, combining the general conversational capabilities of UltraChat with a 25% inclusion of the Arithmo dataset, which focuses on arithmetic reasoning.
Key Capabilities
- General Conversation: Inherits strong conversational abilities from its Mistral-7B base and UltraChat fine-tuning.
- Arithmetic Reasoning: Enhanced performance in mathematical and numerical tasks due to the integration of the Arithmo dataset.
- Instruction Following: Designed to respond effectively to user prompts and system instructions, as demonstrated by its chat template usage.
Training Details
The model was trained with a learning rate of 2e-05 over 1 epoch, utilizing a total batch size of 512 across 16 devices. It achieved a validation loss of 0.8943, indicating effective learning during the fine-tuning process.
Intended Uses
This model is suitable for applications requiring a balance of general-purpose conversational AI with an improved capacity for handling arithmetic queries. It can be deployed for chatbots, virtual assistants, or educational tools where both natural language understanding and basic mathematical problem-solving are beneficial.