HoangCuongNguyen/qwen3-8b-safetyorpo
HoangCuongNguyen/qwen3-8b-safetyorpo is an 8 billion parameter language model fine-tuned from Qwen/Qwen3-8B-Base. This model was trained using the ORPO (Monolithic Preference Optimization without Reference Model) method, which optimizes for safety and preference alignment. It is designed for general text generation tasks where safety and alignment are critical considerations, leveraging its 32768 token context length.
Loading preview...
Model Overview
HoangCuongNguyen/qwen3-8b-safetyorpo is an 8 billion parameter language model, fine-tuned from the robust Qwen/Qwen3-8B-Base architecture. This model distinguishes itself through its training methodology, employing ORPO (Monolithic Preference Optimization without Reference Model), a technique designed to enhance safety and align model outputs with desired preferences without requiring a separate reference model.
Key Capabilities
- Preference Optimization: Utilizes the ORPO method for monolithic preference optimization, aiming for improved safety and alignment in generated text.
- Base Model Foundation: Built upon the Qwen3-8B-Base, inheriting its strong general language understanding and generation capabilities.
- Context Length: Supports a substantial context window of 32768 tokens, allowing for processing and generating longer, more coherent texts.
Training Details
The model was trained using the TRL library, specifically implementing the ORPO algorithm. This approach integrates preference optimization directly into the fine-tuning process, making it a single-stage optimization method. The ORPO method is detailed in the paper "ORPO: Monolithic Preference Optimization without Reference Model" (arXiv:2403.07691).
Good For
- Applications requiring a balance of strong language generation and enhanced safety/alignment.
- Scenarios where preference optimization is crucial for model behavior.
- General text generation tasks benefiting from a large context window.