CorticalStack/mistral-7b-alpaca-sft
CorticalStack/mistral-7b-alpaca-sft is a 7 billion parameter Mistral-based language model, fine-tuned using the Alpaca-cleaned dataset. This model leverages Unsloth for efficient SFT training, making it optimized for instruction-following tasks. It is designed for applications requiring a compact yet capable model for general conversational AI and task execution.
Loading preview...
CorticalStack/mistral-7b-alpaca-sft Overview
This model is a 7 billion parameter instruction-tuned variant of the Mistral architecture, specifically fine-tuned from unsloth/mistral-7b-bnb-4bit. The fine-tuning process utilized the yahma/alpaca-cleaned dataset, which is known for its diverse instruction-following examples.
Key Characteristics
- Base Model: Mistral-7B, known for its strong performance in its size class.
- Fine-tuning: Supervised Fine-Tuning (SFT) on the Alpaca-cleaned dataset, enhancing its ability to follow instructions and engage in conversational tasks.
- Efficiency: Trained with Unsloth and Huggingface's TRL library, indicating an emphasis on efficient training and potentially efficient inference.
- Training Configuration: Employed LoRA with specific parameters (r: 256, alpha: 128, dropout: 0.0) and 4-bit BNB quantization during training, optimizing for resource usage.
Ideal Use Cases
- Instruction Following: Excels at responding to prompts and executing specific instructions due to its Alpaca-based fine-tuning.
- General Conversational AI: Suitable for chatbots and interactive applications where clear and coherent responses are needed.
- Resource-Constrained Environments: The 7B parameter count and 4-bit quantization make it a good candidate for deployment where computational resources are a consideration.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.