EpistemeAI/Fireball-12B
EpistemeAI/Fireball-12B is a 12 billion parameter language model, fine-tuned from Mistral-Nemo-Base-2407, developed by EpistemeAI. It is specifically optimized for coding and general response quality, aiming to outperform models like Llama-3.1-8B and Google Gemma 2 9B. The model utilizes an Alpaca-style instruction format for best performance and has a context window of 32768 tokens.
Loading preview...
Fireball-12B Overview
Fireball-12B is a 12 billion parameter large language model developed by EpistemeAI, built upon the Mistral-Nemo-Base-2407 architecture. This model has undergone extensive fine-tuning, including an ORPO method with specific datasets, to enhance its performance in coding tasks and general response generation. It is designed to offer improved capabilities compared to models in similar size classes, such as Llama-3.1-8B and Google Gemma 2 9B.
Key Capabilities
- Enhanced Coding Performance: Super fine-tuned to provide better coding responses.
- Improved General Responses: Optimized for higher quality general text generation.
- Alpaca Instruction Format: Best performance achieved when used with an Alpaca-style instruction prompt template.
- Large Context Window: Features a 128k context window (though the model card states 32768 tokens for this specific model, the base architecture supports 128k).
- Multilingual and Code Data Training: Benefits from pre-training on a significant proportion of multilingual and code data.
Good For
- Code Generation and Assistance: Ideal for applications requiring robust code generation and understanding.
- Instruction Following Tasks: Excels in scenarios where precise instruction adherence is critical, especially with Alpaca-formatted prompts.
- General Purpose Text Generation: Suitable for a wide range of text generation tasks where high-quality, coherent responses are needed.
- Research and Development: Provides a strong base for further fine-tuning or experimentation in LLM applications.