Locutusque/llama-3-neural-chat-v1-8b is an 8 billion parameter language model fine-tuned by Locutusque, based on Meta Llama 3 architecture. It is specifically optimized for enhanced performance in coding, mathematical problem-solving, and creative writing tasks. This model leverages both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to achieve its specialized capabilities. It is intended for conversational AI applications requiring strong generative abilities in these domains.
Loading preview...
Model Overview
Locutusque/llama-3-neural-chat-v1-8b is an 8 billion parameter language model developed by Locutusque, built upon the Meta Llama 3 architecture. This model has been fine-tuned using an approach similar to Intel's Neural Chat, with modifications to its data sources to bolster its proficiency in specific areas. It incorporates both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) during its training process.
Key Capabilities
- Enhanced Coding: Demonstrates strong performance in code generation and understanding.
- Mathematical Reasoning: Improved ability to solve mathematical word problems.
- Writing Proficiency: Excels in various writing tasks, making it suitable for content generation.
- Conversational AI: Designed for direct use in conversational AI applications.
Training and Performance
The model was trained on a diverse dataset including Open-Orca/SlimOrca-Dedup, jondurbin/airoboros-3.2, microsoft/orca-math-word-problems-200k, m-a-p/Code-Feedback, MaziyarPanahi/WizardLM_evol_instruct_V2_196k, and mlabonne/orpo-dpo-mix-40k. Evaluations on the Open LLM Leaderboard show an average score of 66.50, with specific scores including 64.69 on MMLU (5-shot) and 54.81 on GSM8k (5-shot). The model has a context length of 8192 tokens and is available under the Llama 3 license.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.