FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef0.1-mb4-eta100-bs256x5-n2

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 15, 2026Architecture:Transformer Featherless Exclusive Cold

FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef0.1-mb4-eta100-bs256x5-n2 is a 2 billion parameter language model based on the Qwen3-1.7B architecture, developed by FRPO. This model is an RL fine-tuned checkpoint from KL-in-LLM-RL experiments, specifically optimized using the FRPO method. It features a 32768 token context length and is distinguished by its reinforcement learning fine-tuning approach, making it suitable for tasks benefiting from advanced reward-based optimization.

Loading preview...

Model Overview

This model, FRPO/qwen3-1.7b-a8_klreward-krew-k1-coef0.1-mb4-eta100-bs256x5-n2, is a 2 billion parameter language model built upon the Qwen/Qwen3-1.7B base architecture. It represents a specific checkpoint from the KL-in-LLM-RL / FRPO experimental series, fine-tuned using reinforcement learning (RL) with the verl framework.

Key Characteristics

  • Base Model: Qwen3-1.7B, a 2 billion parameter model.
  • Fine-tuning Method: Utilizes Reinforcement Learning (RL) with a KL-reward mechanism, specifically the FRPO algorithm.
  • Context Length: Supports a substantial context window of 32768 tokens.
  • Weights: Provided in fp32 safetensors format, directly as saved during training.
  • Configuration: The specific run configuration for its training is encoded within the model's repository name.

Differentiator

This model stands out due to its reinforcement learning fine-tuning using the FRPO method, which aims to optimize performance through reward signals. This approach differentiates it from models relying solely on supervised fine-tuning, potentially leading to improved alignment or task-specific performance in areas where reward modeling is effective.

Potential Use Cases

  • Research in RLHF: Ideal for researchers exploring the impact of KL-reward based reinforcement learning on language models.
  • Specific Task Optimization: Could be beneficial for tasks where a custom reward function can guide the model towards desired behaviors more effectively than standard instruction tuning.

Finetunes
1 models