kyujinpy/Korean-OpenOrca-v3
TEXT GENERATIONConcurrency Cost:1Model Size:13BQuant:FP8Ctx Length:4kLicense:cc-by-nc-sa-4.0Architecture:Transformer0.0K Open Weights Warm
Korean-OpenOrca-v3 is a 13 billion parameter auto-regressive language model developed by Kyujin Han, based on the LLaMA2 transformer architecture. Fine-tuned on the OpenOrca-ko-v3 dataset, this model is optimized for Korean language tasks. It demonstrates competitive performance across various Korean benchmarks, making it suitable for applications requiring robust Korean language understanding and generation.
Loading preview...
Korean-OpenOrca-13B-v3 Overview
Developed by Kyujin Han (kyujinpy) as part of an LLM research consortium, Korean-OpenOrca-13B-v3 is a 13 billion parameter auto-regressive language model built upon the LLaMA2 transformer architecture. It utilizes hyunseoki/ko-en-llama2-13b as its base model.
Key Capabilities & Training
- Architecture: LLaMA2-based transformer, optimized for causal language modeling.
- Training Data: Fine-tuned using the OpenOrca-ko-v3 dataset, which is a DeepL translation of the original OpenOrca dataset.
- Performance: Achieves an average score of 48.86 across several Korean benchmarks, including Ko-ARC (43.77), Ko-HellaSwag (54.30), Ko-MMLU (41.79), Ko-TruthfulQA (43.85), and Ko-CommonGen V2 (60.57).
- Context Length: Supports a context length of 4096 tokens.
Use Cases
This model is particularly well-suited for:
- Korean Language Processing: Tasks requiring strong understanding and generation in Korean.
- Research and Development: As a base for further fine-tuning on specific Korean NLP applications.
- Benchmarking: Provides a solid baseline for evaluating other Korean LLMs.