UEC-InabaLab/Llama-3.1-KokoroChat-High
UEC-InabaLab/Llama-3.1-KokoroChat-High is an 8 billion parameter Japanese language model, based on the Llama-3.1-Swallow architecture, fine-tuned specifically for psychological counseling dialogues. It was trained on over 2,600 high-quality counseling conversations from the KokoroChat dataset, enabling it to generate empathetic and context-aware responses for mental health-related conversational tasks. This model excels in simulating counselor-like interactions in Japanese, covering diverse topics such as depression, family, and relationships.
Loading preview...
Llama-3.1-KokoroChat-High: Japanese Counseling Dialogue Model
This model is a specialized 8 billion parameter Japanese language model, fine-tuned by UEC-InabaLab on the Llama-3.1-Swallow-8B-Instruct-v0.3 base. Its primary distinction lies in its training on a curated subset of the KokoroChat dataset, which comprises over 6,000 psychological counseling dialogues. Specifically, this "High" variant was fine-tuned on 2,601 dialogues that received client feedback scores between 70 and 98, indicating high quality and effectiveness.
Key Capabilities
- Empathetic Response Generation: Designed to produce empathetic and context-aware responses in Japanese counseling scenarios.
- Mental Health Dialogue: Optimized for conversational tasks related to mental health, covering topics like depression, family issues, school, career, and relationships.
- Role-Play Data: Benefits from training on data collected through text-based role-play by trained counselors, ensuring realistic and appropriate dialogue patterns.
Fine-Tuning Details
The model was fine-tuned using QLoRA with 4-bit NF4 quantization. The training involved 5 epochs with an adamw_8bit optimizer and a learning rate of 1e-3.
Good For
- Developing AI assistants for mental health support in Japanese.
- Simulating counseling conversations for training or research purposes.
- Applications requiring nuanced, empathetic dialogue generation in a Japanese context, particularly within the mental health domain.