krevas/LDCC-Instruct-Llama-2-ko-13B-v4
krevas/LDCC-Instruct-Llama-2-ko-13B-v4 is a 13 billion parameter instruction-tuned causal language model developed by Lotte Data Communication, based on Meta's Llama 2 architecture. This model is fine-tuned for instruction following, leveraging DeepSpeed and HuggingFace libraries. It is designed for general natural language generation tasks, with a focus on assistant-like chat applications, and has a context length of 4096 tokens.
Loading preview...
Model Overview
krevas/LDCC-Instruct-Llama-2-ko-13B-v4 is a 13 billion parameter instruction-tuned model developed by Lotte Data Communication. It is built upon Meta's Llama 2 architecture, which is an auto-regressive language model utilizing an optimized transformer architecture. The fine-tuning process for this specific model involved using the DeepSpeed library in conjunction with HuggingFace Trainer/Accelerate, indicating a focus on efficient and robust training methodologies.
Key Capabilities
- Instruction Following: Optimized for understanding and responding to user instructions.
- Dialogue Generation: The base Llama 2-Chat models are specifically aligned for assistant-like chat use cases through supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF).
- General Text Generation: Capable of a variety of natural language generation tasks.
- Context Length: Supports a context window of 4096 tokens.
Training and Performance
The underlying Llama 2 13B model was pretrained on 2 trillion tokens of publicly available online data, with a data cutoff of September 2022, and some tuning data up to July 2023. Benchmarks for the Llama 2 13B base model show strong performance across various academic tasks, including commonsense reasoning, world knowledge, and reading comprehension, with notable scores in MATH (28.7) and MMLU (54.8). The fine-tuned Llama-2-Chat 13B also demonstrates high scores in safety evaluations like TruthfulQA (62.18) and very low toxicity (0.00% on Toxigen).
Intended Use
This model is intended for commercial and research use, primarily in English, for assistant-like chat applications and other natural language generation tasks. Developers should perform safety testing tailored to their specific applications.