FRPO/qwen3-1.7b-a6_k3_center-k3-cGroupBoth-clip0.2-mb4-eta100-bs256x5-n2
FRPO/qwen3-1.7b-a6_k3_center-k3-cGroupBoth-clip0.2-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model based on the Qwen3-1.7B architecture, fine-tuned using Reinforcement Learning (RL) with the KL-in-LLM-RL / FRPO experimental framework. This model, with a 32768 token context length, is a specific checkpoint from RL fine-tuning experiments, focusing on the effects of the FRPO method. It is suitable for research into RL-based language model optimization and understanding the impact of specific training configurations.
Loading preview...
Model Overview
FRPO/qwen3-1.7b-a6_k3_center-k3-cGroupBoth-clip0.2-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model derived from the Qwen/Qwen3-1.7B base model. It has been fine-tuned using Reinforcement Learning (RL) as part of the KL-in-LLM-RL / FRPO experimental series, utilizing the verl training framework. This specific checkpoint, global_step_200, represents a particular configuration within these experiments, with its run parameters encoded directly in the repository name.
Key Characteristics
- Base Model: Qwen3-1.7B architecture.
- Parameter Count: Approximately 2 billion parameters.
- Context Length: Supports a context window of 32768 tokens.
- Training Method: Fine-tuned using Reinforcement Learning (RL) with the FRPO method.
- Weights: Provided in fp32 safetensors format, directly as saved from the trainer without post-processing.
Intended Use Cases
This model is primarily intended for:
- Research and Development: Exploring the effects of specific RL fine-tuning configurations, particularly the FRPO method, on language model performance.
- Experimental Analysis: Studying the impact of parameters like
a6_k3_center-k3-cGroupBoth-clip0.2-mb4-eta100-bs256x5-n2on model behavior and capabilities. - Comparative Studies: Benchmarking against other RL-tuned or base models to understand the advantages or disadvantages of this particular training approach.