FRPO/qwen3-1.7b-a18_onpolicy_seqmean_center-k1-cGroupBoth-clip0.2-mb1-eta100-bs64x5-n2
FRPO/qwen3-1.7b-a18_onpolicy_seqmean_center-k1-cGroupBoth-clip0.2-mb1-eta100-bs64x5-n2 is a 1.7 billion parameter language model based on the Qwen3 architecture, fine-tuned using Reinforcement Learning (RL) with the FRPO algorithm. This checkpoint is derived from the KL-in-LLM-RL experiments, specifically optimized for on-policy sequence mean centering. It is intended for research and development in RL fine-tuning methodologies for large language models, offering insights into specific training configurations.
Loading preview...
Model Overview
This model, FRPO/qwen3-1.7b-a18_onpolicy_seqmean_center-k1-cGroupBoth-clip0.2-mb1-eta100-bs64x5-n2, is a 1.7 billion parameter language model built upon the Qwen3-1.7B base architecture. It represents a specific checkpoint from the KL-in-LLM-RL / FRPO experimental series, fine-tuned using Reinforcement Learning (RL) with the verl framework.
Key Characteristics
- Base Model: Qwen/Qwen3-1.7B, a 1.7 billion parameter model.
- Fine-tuning Method: Reinforcement Learning (RL) using the FRPO algorithm.
- Experimental Context: Part of the KL-in-LLM-RL research, focusing on specific RL fine-tuning configurations.
- Checkpoint: The provided checkpoint is
global_step_800. - Weights: Stored in fp32 safetensors format, directly as saved by the trainer without post-processing.
- Configuration: The specific run configuration, including parameters like
onpolicy_seqmean_center-k1-cGroupBoth-clip0.2-mb1-eta100-bs64x5-n2, is encoded within the repository name.
Intended Use
This model is primarily intended for researchers and developers interested in:
- Exploring the effects of specific RL fine-tuning strategies, particularly the FRPO algorithm.
- Analyzing the performance and characteristics of models fine-tuned with on-policy sequence mean centering.
- Contributing to or understanding the KL-in-LLM-RL experiments.