Model Overview

The xiaolesu/OsmosisProofling-SFT-NT-GRPO-NT-Overlap is an 8 billion parameter language model, fine-tuned by xiaolesu from the Qwen/Qwen3-8B base model. It was developed using the Axolotl framework, specifically leveraging its Liger plugin for enhanced training configurations including Liger Rope, RMS Norm, and GLU activation.

Key Training Details

• Base Model: Qwen/Qwen3-8B
• Fine-tuning Dataset: xiaolesu/OsmosisProofling-v3-SFT (Alpaca format)
• Context Length: 4096 tokens during training (though the model itself supports 32768 tokens)
• Training Hyperparameters:
  • Learning Rate: 1e-05
  • Optimizer: AdamW_Torch_Fused
  • LR Scheduler: Cosine
  • Epochs: 2
• Validation Performance: Achieved a final validation loss of 0.3543 and a perplexity (PPL) of 1.4252.

Intended Use Cases

This model is primarily intended for applications that align with the data it was fine-tuned on, xiaolesu/OsmosisProofling-v3-SFT. Users should evaluate its performance on their specific tasks to determine suitability, especially for instruction-following scenarios where the fine-tuning data is relevant.