FRPO/qwen3-4b-a0_klloss-lossKL-coef0.001-mb4-eta100-bs256x5-n4
FRPO/qwen3-4b-a0_klloss-lossKL-coef0.001-mb4-eta100-bs256x5-n4 is a 4 billion parameter language model based on the Qwen3-4B architecture, developed through RL fine-tuning experiments using the KL-in-LLM-RL / FRPO method. This checkpoint is specifically trained with the verl framework, focusing on reinforcement learning optimizations. It provides fp32 safetensors weights directly from the trainer, making it suitable for researchers and developers exploring advanced RL fine-tuning techniques on a Qwen3 base.
Loading preview...
Model Overview
This repository hosts FRPO/qwen3-4b-a0_klloss-lossKL-coef0.001-mb4-eta100-bs256x5-n4, a 4 billion parameter language model derived from the Qwen/Qwen3-4B base. It represents a specific checkpoint from the KL-in-LLM-RL / FRPO experimental series, which focuses on applying reinforcement learning (RL) techniques to large language models.
Key Characteristics
- Base Model: Built upon the robust
Qwen/Qwen3-4Barchitecture. - Fine-tuning Method: Utilizes Reinforcement Learning (RL) as part of the KL-in-LLM-RL / FRPO experiments.
- Training Framework: The RL fine-tuning process was conducted using the verl library.
- Checkpoint: The repository contains the
global_step_200checkpoint. - Weights: Provided in
fp32 safetensorsformat, directly as saved by the trainer without any post-processing. - Configuration: The specific run configuration details are embedded within the repository name itself.
Intended Use Cases
This model is primarily intended for:
- RL Research: Researchers exploring advanced RL fine-tuning methodologies, particularly those involving KL-divergence in LLM training.
- Experimental Development: Developers interested in evaluating the impact of specific RL configurations (like
lossKL-coef0.001,mb4,eta100,bs256x5,n4) on the Qwen3-4B base model. - Reproducibility: Providing a specific, unadulterated checkpoint for reproducing results from the KL-in-LLM-RL / FRPO experiments.