FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef0.5-mb4-eta100-bs256x5-n2
FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef0.5-mb4-eta100-bs256x5-n2 is a 1.7 billion parameter language model based on the Qwen3 architecture, developed by FRPO. This model is an RL fine-tuned checkpoint from the KL-in-LLM-RL / FRPO experiments, specifically trained with the verl framework. It is optimized through a reinforcement learning process using a KL reward mechanism, making it suitable for tasks benefiting from advanced fine-tuning strategies.
Loading preview...
Overview
FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef0.5-mb4-eta100-bs256x5-n2 is a 1.7 billion parameter language model derived from the Qwen/Qwen3-1.7B base model. It represents a checkpoint from the KL-in-LLM-RL / FRPO experimental series, which focuses on applying Reinforcement Learning (RL) to large language models. The model was fine-tuned using the verl framework, incorporating a KL reward mechanism (krew-k1-coef0.5) during its training process. The specific run configuration is encoded within its repository name, indicating parameters such as batch size (bs256x5) and learning rate (eta100).
Key Capabilities
- Reinforcement Learning Fine-tuned: Benefits from advanced RL optimization, specifically using a KL reward for improved performance in targeted scenarios.
- Qwen3 Architecture: Built upon the robust Qwen3 base model, inheriting its foundational language understanding and generation capabilities.
- FP32 Safetensors: The model weights are provided in fp32 safetensors format, ensuring direct usability as saved by the trainer without post-processing.
Good For
- Researchers and developers exploring the impact of Reinforcement Learning with KL rewards on LLM performance.
- Applications requiring a Qwen3-based model that has undergone specific RL fine-tuning.
- Experiments and deployments where the detailed run configuration (as indicated in the model name) aligns with desired training methodologies.