ignos/Mistral-T5-7B-v1
The ignos/Mistral-T5-7B-v1 is a 7 billion parameter Mistral-based language model developed by Ignos, fine-tuned from Toten5/Marcoroni-neural-chat-7B-v2. This model is specifically designed to improve instructional behavior, leveraging the Mistral architecture for enhanced performance. It was trained using the QLoRA approach on the tatsu-lab/alpaca dataset, making it suitable for instruction-following tasks.
Loading preview...
ignos/Mistral-T5-7B-v1: Instruction-Tuned Mistral Model
This model, developed by Ignos, is a 7 billion parameter language model built upon the Mistral architecture. It is a fine-tuned version of the Toten5/Marcoroni-neural-chat-7B-v2 base model, specifically optimized for improving instructional behavior.
Key Capabilities
- Instruction Following: Designed to enhance responses to instructional prompts.
- Mistral Architecture: Benefits from the efficient and capable Mistral model design.
- QLoRA Fine-tuning: Utilizes the QLoRA approach for efficient adaptation and merging with the base model.
- Apache-2.0 License: Offers flexible usage under an open-source license.
Training Details
The model was trained using the QLoRA method, leveraging the tatsu-lab/alpaca dataset for instruction-tuning. Training was conducted on a robust compute infrastructure featuring 3 x RTX 4090 GPUs, 48 vCPUs, and 377 GB RAM, utilizing Axolotl 0.3.0 and PEFT 0.6.0 frameworks. The model aims to provide improved performance in instructional contexts, building on its 8192 token context length.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.