Orca 2: Enhanced Reasoning for Smaller Language Models

Orca 2 is a 13 billion parameter language model, a fine-tuned variant of the LLaMA-2 architecture, developed by Microsoft. Its primary distinction lies in its enhanced reasoning capabilities, achieved through training on a synthetic dataset specifically designed for this purpose. This model is intended for research purposes only, serving as a foundation for exploring and improving smaller language models.

Key Capabilities

• Reasoning: Excels in tasks that require logical deduction and inference over provided data.
• Reading Comprehension: Demonstrates proficiency in understanding and extracting information from text.
• Math Problem Solving: Capable of addressing mathematical challenges.
• Text Summarization: Can condense longer texts into concise summaries.
• Single-Turn Responses: Designed to provide direct answers to queries.

Intended Use and Limitations

Orca 2 is built exclusively for the research community to assess its abilities and contribute to the development of better frontier models. It has been evaluated across a wide range of tasks, including reasoning, grounding, and safety, with detailed evaluations available in the Orca 2 paper.

As a derivative of LLaMA 2, Orca 2 inherits many of its limitations, including potential data biases, limited real-world contextual understanding, and the risk of hallucination. Its performance can also be influenced by the distribution of its synthetic training data and system instructions. The model is not intended for downstream applications without further analysis of potential harm or bias.