FRPO/qwen3-1.7b-a10_lengthnorm-k1-cNone-lnorm-clip0.2-mb4-eta100-bs256x5-n2
FRPO/qwen3-1.7b-a10_lengthnorm-k1-cNone-lnorm-clip0.2-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model based on Qwen3-1.7B, developed by FRPO as part of the KL-in-LLM-RL experiments. This checkpoint has undergone RL fine-tuning using the verl framework, focusing on specific reinforcement learning objectives. It is designed for research and development in RL-tuned language models, offering insights into the effects of length normalization and other training configurations.
Loading preview...
Model Overview
FRPO/qwen3-1.7b-a10_lengthnorm-k1-cNone-lnorm-clip0.2-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model derived from the Qwen/Qwen3-1.7B base model. It is a product of the KL-in-LLM-RL / FRPO experimental series, specifically fine-tuned using Reinforcement Learning (RL) with the verl framework.
Key Characteristics
- Base Model: Qwen3-1.7B, a 2 billion parameter model.
- Fine-tuning: Utilizes Reinforcement Learning (RL) as part of the KL-in-LLM-RL experiments.
- Framework: Trained with the
verllibrary. - Configuration: The model's specific run configuration, including parameters like
lengthnorm-k1-cNone-lnorm-clip0.2-mb4-eta100-bs256x5-n2, is encoded within its repository name, indicating various experimental settings applied during training. - Weights: Provided in fp32 safetensors format, directly as saved by the trainer without additional post-processing.
Intended Use Cases
This model is primarily suited for:
- RL Research: Investigating the impact of different RL fine-tuning strategies on language models.
- Experimental Analysis: Studying the effects of specific training configurations, such as length normalization and clipping parameters, on model performance and behavior.
- Comparative Studies: Benchmarking RL-tuned models against their base counterparts or other RL-tuned variants.