FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef1e-4-mb4-eta100-bs256x5-n2
FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef1e-4-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model based on the Qwen3-1.7B architecture, developed by FRPO. This model is a checkpoint from KL-in-LLM-RL / FRPO experiments, specifically fine-tuned using Reinforcement Learning (RL) with the verl framework. It is designed for tasks benefiting from RL-based optimization, offering a 32768 token context length.
Loading preview...
Model Overview
FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef1e-4-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model derived from the Qwen3-1.7B base architecture. It represents a specific checkpoint from the KL-in-LLM-RL / FRPO experimental series, where it underwent Reinforcement Learning (RL) fine-tuning using the verl framework.
Key Characteristics
- Base Model: Qwen/Qwen3-1.7B, indicating its foundational architecture.
- Fine-tuning Method: Utilizes Reinforcement Learning (RL) with KL-reward, suggesting optimization for specific reward signals.
- Checkpoint: The repository contains the
global_step_200checkpoint, representing a specific stage of the RL training process. - Weights: Provided in
fp32 safetensorsformat, directly as saved by the trainer without further post-processing. - Context Length: Supports a substantial context window of 32768 tokens.
Intended Use Cases
This model is particularly suited for research and development in:
- RL-based Language Model Applications: Exploring the effects and benefits of KL-reward-based RL fine-tuning.
- Experimental Setups: Serving as a foundation for further experimentation with RL techniques on Qwen3-1.7B.
- Performance Analysis: Evaluating the impact of specific RL configurations (encoded in the repo name) on model behavior and output quality.