Model Overview

This model, olympiad-curated-qwen3-4b-thinking-distill-30b-5ep-ablation, is a 4 billion parameter language model developed by hmdmahdavi. It is a fine-tuned variant of the Qwen3-4B-Instruct-2507 base model, utilizing a substantial 32768 token context window. The fine-tuning process was conducted using the TRL (Transformer Reinforcement Learning) framework, indicating a focus on enhancing its conversational and instruction-following capabilities.

Key Capabilities

• Instruction Following: As a fine-tuned instruction model, it is designed to respond effectively to user prompts and questions.
• Text Generation: Capable of generating coherent and contextually relevant text based on input.
• Qwen3 Architecture: Benefits from the robust architecture of the Qwen3 series, known for its general language understanding.

Training Details

The model underwent Supervised Fine-Tuning (SFT), a common method for adapting pre-trained language models to specific tasks or instruction sets. The training process utilized TRL version 0.12.0, Transformers 4.57.6, Pytorch 2.5.1, Datasets 4.5.0, and Tokenizers 0.22.2. Further details on the training run can be visualized via its Weights & Biases project.