FRPO/qwen3-1.7b-a3_onpolicy-k1-cNone-clip0.2-mb1-eta100-bs256x5-n2-s800

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 15, 2026Architecture:Transformer Featherless Exclusive Cold

FRPO/qwen3-1.7b-a3_onpolicy-k1-cNone-clip0.2-mb1-eta100-bs256x5-n2-s800 is a 2 billion parameter language model based on the Qwen3-1.7B architecture, fine-tuned using Reinforcement Learning (RL) with the FRPO method. This checkpoint, specifically global_step_800, is part of the KL-in-LLM-RL experiments. It is provided in fp32 safetensors format, directly from the trainer, and is suitable for research and development in RL-tuned language models.

Loading preview...

Model Overview

FRPO/qwen3-1.7b-a3_onpolicy-k1-cNone-clip0.2-mb1-eta100-bs256x5-n2-s800 is a 2 billion parameter language model derived from the Qwen3-1.7B base model. This specific checkpoint, global_step_800, represents an outcome of the KL-in-LLM-RL / FRPO experimental series, where it was fine-tuned using Reinforcement Learning (RL) with the verl framework.

Key Characteristics

  • Base Model: Qwen/Qwen3-1.7B, indicating its foundational architecture.
  • Fine-tuning Method: Utilizes Reinforcement Learning (RL) with the FRPO (Fictitious Reinforcement Policy Optimization) approach, as part of the KL-in-LLM-RL experiments.
  • Checkpoint: The model provided is global_step_800, reflecting a specific stage in its RL training.
  • Weights: Distributed in fp32 safetensors format, preserving the exact output from the training process without further post-processing.
  • Configuration: The specific run configuration parameters are encoded within the repository name itself.

Intended Use Cases

This model is primarily intended for:

  • Research and Development: Exploring the effects and performance of RL fine-tuning, particularly with the FRPO method, on Qwen3-1.7B.
  • Experimental Evaluation: Analyzing the behavior and capabilities of models trained with specific RL configurations (e.g., k1-cNone-clip0.2-mb1-eta100-bs256x5-n2-s800).
  • Comparative Studies: Benchmarking against other RL-tuned or instruction-tuned models to understand the impact of different RL algorithms and hyperparameters.

Finetunes
1 models