Llama-KoEmpathy: Empathetic Korean Chatbot

Llama-KoEmpathy is an 8 billion parameter language model developed by byeolki, built upon the Llama 3.1 architecture. Its core purpose is to generate empathetic and emotionally understanding responses in Korean conversations. This model was fine-tuned using the AIHub Empathy Dialogue dataset with LoRA (Low-Rank Adaptation) for efficient training.

Key Capabilities

• Emotion Recognition and Empathy: Designed to understand user emotions and respond empathetically.
• Korean Chatbot: Optimized for conversational AI in the Korean language.
• Efficient Fine-tuning: Utilizes LoRA (r=16, alpha=16) on the unsloth/Meta-Llama-3.1-8B base model.

Training Details

The model was trained with a maximum sequence length of 2048, a batch size of 128, and 3 epochs. It uses an AdamW 8bit optimizer with a learning rate of 2e-4 and is available in GGUF q8_0 quantization. The model operates under the Llama 3.1 Community License, requiring adherence to Meta's Acceptable Use Policy.

Good For

• Applications requiring AI to engage in emotionally intelligent and empathetic dialogue in Korean.
• Chatbots focused on user support, mental wellness, or any scenario where understanding and responding to user sentiment is crucial.