Locutusque/Orca-2-13b-SFT-v4
Locutusque/Orca-2-13b-SFT-v4 is a 13 billion parameter instruction-tuned language model based on the Microsoft Orca-2-13b architecture, fine-tuned on a diverse dataset including HuggingFaceH4/no_robots and OpenAssistant/oasst_top1_2023-08-25. This model is optimized for conversational AI and instruction following, achieving a test loss of 0.18. It is notable for its strong performance among Llama2-13b models on the Open LLM Leaderboard.
Loading preview...
Model Overview
Locutusque/Orca-2-13b-SFT-v4 is a 13 billion parameter instruction-tuned model built upon the microsoft/Orca-2-13b base. It has been fine-tuned using a combination of high-quality datasets, including HuggingFaceH4/no_robots, totally-not-an-llm/EverythingLM-data-V3, mlabonne/guanaco-llama2-1k, and OpenAssistant/oasst_top1_2023-08-25.
Key Capabilities
- Instruction Following: The model is specifically fine-tuned for instruction-based tasks, utilizing the ChatML prompt template.
- Performance: Achieved a low test loss of 0.18, indicating strong generalization capabilities.
- Competitive Ranking: As of December 18, 2023, this model holds the #5 position on the Open LLM Leaderboard among Llama2-13b models, demonstrating its competitive performance in its class.
Usage Considerations
- Licensing: Users must comply with the Microsoft Research License associated with the base model.
- Inference Parameters: Recommended sampling parameters for optimal responses include
temperature = 0.1,top_p = 0.14,top_k = 41, andrepetition_penalty = 1.176.