uukuguy/speechless-orca-platypus-coig-lite-4k-0.6e-13b

TEXT GENERATIONConcurrency Cost:1Model Size:13BQuant:FP8Ctx Length:4kPublished:Aug 31, 2023License:cc-by-nc-4.0Architecture:Transformer Open Weights Cold

uukuguy/speechless-orca-platypus-coig-lite-4k-0.6e-13b is a 13 billion parameter Llama 2-based autoregressive language model. It is a fine-tuned variant of OpenOrca-Platypus2-13B, enhanced with 10% COIG-PC-LITE and 10% OpenOrca datasets to improve Chinese language capabilities, while maintaining a 4096-token context window. This model is a merge of Platypus2-13B and OpenOrcaxOpenChat-Preview2-13B, excelling in reasoning and logical tasks as evidenced by its strong performance on AGIEval and BigBench-Hard benchmarks.

Loading preview...

Model Overview

uukuguy/speechless-orca-platypus-coig-lite-4k-0.6e-13b is a 13 billion parameter Llama 2-based instruction-tuned language model, developed by uukuguy. It is a specialized fine-tune of the OpenOrca-Platypus2-13B model, which itself is a merge of garage-bAInd/Platypus2-13B and Open-Orca/OpenOrcaxOpenChat-Preview2-13B. This version specifically incorporates 10% COIG-PC-LITE and 10% OpenOrca datasets, alongside 100% Open-Platypus data, to enhance its Chinese language capabilities while retaining strong English performance.

Key Capabilities & Performance

  • Multilingual Enhancement: Specifically fine-tuned to improve Chinese language understanding and generation, building upon its English-centric predecessors.
  • Reasoning & Logic: Demonstrates strong performance in reasoning and logical tasks, achieving 112% of the base model's performance on AGIEval (averaging 0.463) and 105% on BigBench-Hard (averaging 0.442).
  • Benchmark Scores: On the HuggingFace Leaderboard, it achieves:
    • MMLU (5-shot): 59.5
    • ARC (25-shot): 62.88
    • HellaSwag (10-shot): 83.19
    • TruthfulQA (0-shot): 52.69
    • Average: 64.56
  • Context Window: Supports a context window of 4096 tokens.

Training Details

The model was instruction fine-tuned using LoRA. The base Platypus2-13B was trained on STEM and logic-based datasets, while OpenOrcaxOpenChat-Preview2-13B utilized a refined subset of GPT-4 data from the OpenOrca dataset.

Good For

  • Applications requiring strong reasoning and logical problem-solving in English.
  • Use cases that benefit from enhanced Chinese language understanding alongside general English capabilities.
  • Developers looking for a 13B parameter model with a balanced performance across various benchmarks, particularly in academic and complex reasoning tasks.

Finetunes
1 models