Hugging Face Overview
Overview
SkkuDS-DPO-72B-v4 is a 72.3 billion parameter language model developed by logicker, built upon the Qwen1.5 architecture. It is a transformer-based, decoder-only model that has undergone DPO (Direct Preference Optimization) tuning using the Intel/orca_dpo_pairs dataset. This model is designed to offer improved performance in conversational and instruction-following tasks.
Key Capabilities
- Multilingual Support: Features enhanced multilingual capabilities for both base and chat models.
- Extended Context Length: Provides stable support for a 32K token context length, allowing for processing longer inputs and generating more coherent extended responses.
- DPO Fine-tuning: Optimized through Direct Preference Optimization, which typically leads to better alignment with human preferences and improved instruction following.
- Robust Architecture: Based on the Qwen1.5 architecture, which includes features like SwiGLU activation, attention QKV bias, and group query attention, contributing to its strong performance.
Good For
- General-purpose conversational AI applications.
- Tasks requiring understanding and generation of long-form text.
- Multilingual text generation and comprehension.