Model Overview

Locutusque/Orca-2-13b-SFT-v4 is a 13 billion parameter instruction-tuned model built upon the microsoft/Orca-2-13b base. It has been fine-tuned using a combination of high-quality datasets, including HuggingFaceH4/no_robots, totally-not-an-llm/EverythingLM-data-V3, mlabonne/guanaco-llama2-1k, and OpenAssistant/oasst_top1_2023-08-25.

Key Capabilities

• Instruction Following: The model is specifically fine-tuned for instruction-based tasks, utilizing the ChatML prompt template.
• Performance: Achieved a low test loss of 0.18, indicating strong generalization capabilities.
• Competitive Ranking: As of December 18, 2023, this model holds the #5 position on the Open LLM Leaderboard among Llama2-13b models, demonstrating its competitive performance in its class.

Usage Considerations

• Licensing: Users must comply with the Microsoft Research License associated with the base model.
• Inference Parameters: Recommended sampling parameters for optimal responses include temperature = 0.1, top_p = 0.14, top_k = 41, and repetition_penalty = 1.176.