Locutusque/Orca-2-13b-SFT-v6

TEXT GENERATIONConcurrency Cost:1Model Size:13BQuant:FP8Ctx Length:4kPublished:Dec 22, 2023License:otherArchitecture:Transformer0.0K Cold

Locutusque/Orca-2-13b-SFT-v6 is a 13 billion parameter language model, fine-tuned from Microsoft's Orca-2-13b base model. It was trained on a diverse collection of instruction datasets including HuggingFaceH4/no_robots and LDJnr/Capybara, achieving a test loss of 0.39 on LDJnr/Verified-Camel. This model is optimized for general instruction following and reasoning tasks, demonstrating competitive performance across various benchmarks with a 4096-token context length.

Loading preview...

Locutusque/Orca-2-13b-SFT-v6: An Instruction-Tuned LLM

This model is a 13 billion parameter instruction-tuned large language model, built upon the robust microsoft/Orca-2-13b base. It has undergone extensive fine-tuning using a comprehensive suite of high-quality instruction datasets, including HuggingFaceH4/no_robots, totally-not-an-llm/EverythingLM-data-V3, LDJnr/Capybara, and OpenAssistant/oasst_top1_2023-08-25, among others. The training process resulted in a notable test loss of 0.39 on the LDJnr/Verified-Camel dataset.

Key Capabilities & Performance

  • Instruction Following: Optimized for understanding and executing a wide range of user instructions, leveraging the ChatML prompt template.
  • Reasoning: Demonstrates strong reasoning capabilities, as indicated by its performance on benchmarks like AI2 Reasoning Challenge (60.41) and MMLU (59.51).
  • General Knowledge: Achieves an average score of 56.15 across the Open LLM Leaderboard evaluations, including HellaSwag (80.46) and TruthfulQA (54.01).
  • Context Length: Supports a context window of 4096 tokens, suitable for moderately long interactions.

When to Use This Model

This model is particularly well-suited for applications requiring:

  • General-purpose chatbots and conversational agents that need to follow complex instructions.
  • Reasoning tasks and question-answering scenarios.
  • Content generation where adherence to specific prompts is crucial.

Users should review the Microsoft Research license before deployment. Detailed benchmark results are available on the Open LLM Leaderboard.

Finetunes
3 models