FRPO/qwen3-1.7b-a3_onpolicy-k1-cNone-clip0.2-mb1-eta100-bs256x5-n2
FRPO/qwen3-1.7b-a3_onpolicy-k1-cNone-clip0.2-mb1-eta100-bs256x5-n2 is a 2 billion parameter language model based on the Qwen3-1.7B architecture, developed by FRPO. This model has been fine-tuned using Reinforcement Learning (RL) with the FRPO algorithm, specifically for on-policy optimization. It features a 32768-token context length and is designed for tasks benefiting from RL-based performance enhancements.
Loading preview...
Overview
This model, FRPO/qwen3-1.7b-a3_onpolicy-k1-cNone-clip0.2-mb1-eta100-bs256x5-n2, is a 2 billion parameter language model derived from the Qwen3-1.7B base architecture. It has undergone Reinforcement Learning (RL) fine-tuning using the FRPO algorithm, specifically configured for on-policy learning experiments.
Key Characteristics
- Base Model: Qwen/Qwen3-1.7B, providing a robust foundation.
- Fine-tuning Method: Utilizes the FRPO algorithm for RL-based optimization, trained with the verl framework.
- Parameter Count: Approximately 2 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Weights: Provided in fp32 safetensors format, directly from the trainer without post-processing.
- Configuration: The specific run configuration for its RL training is encoded within the repository name itself.
When to Consider This Model
This model is particularly suited for research and applications where:
- You are exploring the effects of Reinforcement Learning from Human Feedback (RLHF) or similar RL-based fine-tuning approaches.
- Your use case benefits from a model optimized through on-policy RL algorithms.
- You require a model with a large context window (32768 tokens) for processing extensive inputs.
- You are working with the Qwen3-1.7B family and need an RL-tuned variant.