FRPO/qwen3-1.7b-a0_klloss-lossKL-coef0.001-mb4-eta100-bs256x5-n2
FRPO/qwen3-1.7b-a0_klloss-lossKL-coef0.001-mb4-eta100-bs256x5-n2 is a 1.7 billion parameter language model based on the Qwen3 architecture, developed by FRPO. This model is an RL fine-tuned checkpoint from the KL-in-LLM-RL / FRPO experiments, specifically trained using the verl framework. It is distinguished by its application of KL-loss during reinforcement learning, making it suitable for research into RL-based language model optimization.
Loading preview...
Model Overview
This model, FRPO/qwen3-1.7b-a0_klloss-lossKL-coef0.001-mb4-eta100-bs256x5-n2, is a 1.7 billion parameter language model derived from the Qwen/Qwen3-1.7B base model. It represents a checkpoint from the KL-in-LLM-RL / FRPO experimental series, indicating its focus on reinforcement learning (RL) fine-tuning.
Key Characteristics
- Base Model: Built upon the
Qwen3-1.7Barchitecture. - Fine-tuning Method: Utilizes Reinforcement Learning (RL) with a specific configuration involving KL-loss (KLLOSS-lossKL-coef0.001), a minibatch size of 4 (mb4), an eta value of 100 (eta100), and a batch size of 256x5 (bs256x5-n2).
- Training Framework: Fine-tuned using the verl framework.
- Weights: Provided in fp32 safetensors format, directly as saved from the trainer without post-processing.
- Configuration: The specific run configuration parameters are encoded within the repository name itself.
Good For
- RL Research: Ideal for researchers exploring the effects of KL-loss coefficients and specific RL configurations in language model fine-tuning.
- Comparative Studies: Useful for comparing the performance and characteristics of RL-tuned models against their base counterparts or other RL methodologies.
- Experimental Development: Suitable for developers and researchers working on advanced fine-tuning techniques for Qwen3-based models.