arcee-ai/zilo-instruct-v2-sft-filtered
arcee-ai/zilo-instruct-v2-sft-filtered is a 7 billion parameter instruction-tuned causal language model developed by arcee-ai. It is a fine-tuned version of Mistral-7B-Instruct-v0.2, specifically optimized using the arcee-ai/Zilo-Filtered-SQL-Instruct-v2 dataset. This model is designed for tasks related to SQL instruction following, leveraging its base architecture and specialized training.
Loading preview...
Model Overview
arcee-ai/zilo-instruct-v2-sft-filtered is a 7 billion parameter instruction-tuned model built upon the mistralai/Mistral-7B-Instruct-v0.2 architecture. It has been fine-tuned by arcee-ai using the proprietary arcee-ai/Zilo-Filtered-SQL-Instruct-v2 dataset.
Key Characteristics
- Base Model: Mistral-7B-Instruct-v0.2
- Parameter Count: 7 billion
- Context Length: 4096 tokens
- Fine-tuning Dataset: arcee-ai/Zilo-Filtered-SQL-Instruct-v2
Training Details
The model was trained with a learning rate of 2e-05 over 3 epochs. Key hyperparameters included a train_batch_size of 16, eval_batch_size of 8, and an Adam optimizer. The training process resulted in a final validation loss of 0.5474.
Potential Use Cases
Given its fine-tuning on a SQL instruction dataset, this model is likely best suited for tasks involving:
- Generating SQL queries from natural language instructions.
- Interpreting and explaining SQL commands.
- Assisting with SQL-related development and data manipulation tasks.