werty1248/Llama-3-Ko-8B-Instruct-AOG
werty1248/Llama-3-Ko-8B-Instruct-AOG is an 8 billion parameter instruction-tuned Llama 3 model, fine-tuned from beomi/Llama-3-Open-Ko-8B. This model specializes in Korean language instruction following, leveraging datasets like KoAlpaca-v1.1a, OpenOrca-KO, and openassistant-guanaco-ko. It is optimized for detailed and friendly conversational responses in Korean, showing improved accuracy on the kobest_boolq benchmark.
Loading preview...
Overview
werty1248/Llama-3-Ko-8B-Instruct-AOG is an 8 billion parameter instruction-tuned model based on the Llama 3 architecture, specifically fine-tuned from beomi/Llama-3-Open-Ko-8B. This model is designed for enhanced instruction following in the Korean language.
Key Capabilities
- Korean Instruction Following: Optimized for generating detailed and friendly responses to user instructions in Korean.
- Dataset Integration: Trained on a combination of commercially available Korean datasets, including beomi/KoAlpaca-v1.1a, kyujinpy/OpenOrca-KO, and nlpai-lab/openassistant-guanaco-ko.
- Multi-turn Conversation: Incorporates multi-turn conversation data, though its reliability in this area is noted as lower.
- Performance: Shows an improvement in accuracy on the
kobest_boolqbenchmark (0.8312) compared to its base model (0.7963).
Training Details
The model was trained using Axolotl with LoRA for 3 epochs, utilizing a sequence length of 4096 and bf16 precision. Training took approximately 8 hours on a single A100 GPU.
Recommended Usage
It is recommended to use an "Instruction - Input - Response" or "Question - Instruction - Response" format for optimal performance, with the latter often yielding better results. The model aims to provide comprehensive answers without requiring external searches.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.