FRPO/qwen3-1.7b-a19_seqmean_center-k1-cGroupBoth-clip0.2-mb4-eta100-bs256x5-n2
FRPO/qwen3-1.7b-a19_seqmean_center-k1-cGroupBoth-clip0.2-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model based on the Qwen3-1.7B architecture, fine-tuned using Reinforcement Learning (RL) with the FRPO (KL-in-LLM-RL) experimental framework. This model is specifically a checkpoint from these RL experiments, focusing on optimizing performance through reinforcement learning techniques. It is designed for research and development in RL-tuned language models, offering insights into the effects of specific RL configurations.
Loading preview...
Model Overview
FRPO/qwen3-1.7b-a19_seqmean_center-k1-cGroupBoth-clip0.2-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model derived from the Qwen/Qwen3-1.7B base model. It represents a specific checkpoint (global_step_200) from the KL-in-LLM-RL / FRPO experimental series, which utilizes Reinforcement Learning (RL) for fine-tuning. The model was trained using the verl framework.
Key Characteristics
- Base Architecture: Qwen3-1.7B.
- Fine-tuning Method: Reinforcement Learning (RL) using the FRPO (KL-in-LLM-RL) experimental setup.
- Checkpoint: This repository contains the
global_step_200checkpoint. - Weights: Provided in fp32 safetensors format, exactly as saved by the trainer without post-processing.
- Configuration: The specific run configuration is encoded within the model's repository name, detailing parameters like
seqmean_center,k1,cGroupBoth,clip0.2,mb4,eta100,bs256x5, andn2.
Intended Use Cases
This model is primarily suited for:
- RL Research: Investigating the impact of specific RL fine-tuning strategies on language models.
- Experimental Analysis: Studying the performance and characteristics of models trained with the FRPO framework.
- Comparative Studies: Benchmarking against other RL-tuned or base models to understand the effects of different training configurations.