xiaolesu/OsmosisProofling-v2-SFT
xiaolesu/OsmosisProofling-v2-SFT is an 8 billion parameter language model fine-tuned from Qwen/Qwen3-8B. This model was trained using the Axolotl framework on the xiaolesu/OsmosisProofling-v2-SFT dataset, achieving a validation loss of 0.3550 and a perplexity of 1.4261. It is optimized for tasks related to its specific fine-tuning dataset, demonstrating improved performance on the evaluation set compared to its base model.
Loading preview...
Model Overview
xiaolesu/OsmosisProofling-v2-SFT is an 8 billion parameter language model derived from the Qwen/Qwen3-8B architecture. It has been fine-tuned using the Axolotl framework, incorporating specific Liger plugin features such as liger_rope, liger_rms_norm, and liger_glu_activation.
Key Training Details
- Base Model: Qwen/Qwen3-8B
- Fine-tuning Dataset:
xiaolesu/OsmosisProofling-v2-SFT(alpaca format) - Context Length: 4096 tokens (with sample packing and flex attention enabled)
- Optimizer: AdamW_torch_fused with a learning rate of 1e-05
- Training Steps: 210 steps over 2 epochs
- Distributed Training: Multi-GPU setup with 7 devices
Performance Metrics
During evaluation, the model achieved a final validation loss of 0.3550 and a perplexity (PPL) of 1.4261. Memory usage during training peaked at 20.98 GiB active and allocated memory, with 36.0 GiB reserved memory.
Potential Use Cases
Given its fine-tuning on the xiaolesu/OsmosisProofling-v2-SFT dataset, this model is likely best suited for tasks aligned with the content and style of that specific dataset. Developers should evaluate its performance on their particular use cases to determine suitability.