Model Overview
DCAgent/a1-wizardlm_orca is an 8 billion parameter language model, developed by DCAgent, based on the robust Qwen/Qwen3-8B architecture. This model has undergone supervised fine-tuning (SFT) using a specialized dataset derived from wizardlm-orca-sandboxes_glm_4.7_traces_jupiter.
Key Characteristics
- Base Model: Qwen3-8B, providing a strong foundation for language understanding and generation.
- Fine-tuning Data: Optimized on a unique dataset focusing on conversational traces, suggesting enhanced performance in dialogue-centric applications.
- Context Length: Benefits from the Qwen3-8B's substantial 32768 token context window, enabling processing of extensive inputs and maintaining coherence over long conversations.
Training Details
The model was trained with a learning rate of 4e-05, a total batch size of 16 across 16 devices, and utilized the AdamW_Torch_Fused optimizer. Training spanned 7 epochs with a cosine learning rate scheduler and a warmup ratio of 0.1. This configuration aims to achieve stable and effective learning from the fine-tuning dataset.
Intended Use Cases
Given its fine-tuning on conversational trace data, this model is likely well-suited for:
- Complex dialogue systems.
- Advanced conversational AI agents.
- Tasks requiring deep contextual understanding from extended interactions.