FRPO/qwen3-1.7b-a17_seqmean-k1-cNone-clip0.2-mb4-eta100-bs256x5-n2
FRPO/qwen3-1.7b-a17_seqmean-k1-cNone-clip0.2-mb4-eta100-bs256x5-n2 is a 1.7 billion parameter language model based on Qwen3-1.7B, developed by FRPO. This model is an RL fine-tuned checkpoint from the KL-in-LLM-RL / FRPO experiments, specifically optimized using the verl training framework. It provides a specialized variant of the Qwen3 architecture, focusing on reinforcement learning applications.
Loading preview...
Model Overview
FRPO/qwen3-1.7b-a17_seqmean-k1-cNone-clip0.2-mb4-eta100-bs256x5-n2 is a 1.7 billion parameter language model derived from the Qwen/Qwen3-1.7B base architecture. This particular checkpoint is a result of Reinforcement Learning (RL) fine-tuning conducted as part of the KL-in-LLM-RL / FRPO experimental series.
Key Characteristics
- Base Model: Utilizes
Qwen/Qwen3-1.7Bas its foundation. - Fine-tuning Method: Underwent RL fine-tuning using the verl framework, indicating a focus on optimizing for specific reward signals or behaviors.
- Parameter Count: Features 1.7 billion parameters, offering a balance between performance and computational efficiency.
- Weights: Provided in fp32 safetensors format, directly as saved by the trainer without further post-processing.
- Configuration: The specific run configuration for its training is encoded within the repository name itself.
Intended Use Cases
This model is primarily suited for research and development in areas involving:
- Reinforcement Learning Applications: Ideal for exploring the effects of RL fine-tuning on large language models.
- Experimental Setups: Useful for researchers replicating or extending the KL-in-LLM-RL / FRPO experiments.
- Specialized Task Optimization: Potentially applicable where a Qwen3-based model benefits from RL-driven behavioral alignment.