Overview

SkkuDS-DPO-72B-v4 is a 72.3 billion parameter language model developed by logicker, built upon the Qwen1.5 architecture. It is a transformer-based, decoder-only model that has undergone DPO (Direct Preference Optimization) tuning using the Intel/orca_dpo_pairs dataset. This model is designed to offer improved performance in conversational and instruction-following tasks.

Key Capabilities

• Multilingual Support: Features enhanced multilingual capabilities for both base and chat models.
• Extended Context Length: Provides stable support for a 32K token context length, allowing for processing longer inputs and generating more coherent extended responses.
• DPO Fine-tuning: Optimized through Direct Preference Optimization, which typically leads to better alignment with human preferences and improved instruction following.
• Robust Architecture: Based on the Qwen1.5 architecture, which includes features like SwiGLU activation, attention QKV bias, and group query attention, contributing to its strong performance.

Good For

• General-purpose conversational AI applications.
• Tasks requiring understanding and generation of long-form text.
• Multilingual text generation and comprehension.