Model Overview

xiaolesu/OsmosisProofling-v2-SFT is an 8 billion parameter language model derived from the Qwen/Qwen3-8B architecture. It has been fine-tuned using the Axolotl framework, incorporating specific Liger plugin features such as liger_rope, liger_rms_norm, and liger_glu_activation.

Key Training Details

• Base Model: Qwen/Qwen3-8B
• Fine-tuning Dataset: xiaolesu/OsmosisProofling-v2-SFT (alpaca format)
• Context Length: 4096 tokens (with sample packing and flex attention enabled)
• Optimizer: AdamW_torch_fused with a learning rate of 1e-05
• Training Steps: 210 steps over 2 epochs
• Distributed Training: Multi-GPU setup with 7 devices

Performance Metrics

During evaluation, the model achieved a final validation loss of 0.3550 and a perplexity (PPL) of 1.4261. Memory usage during training peaked at 20.98 GiB active and allocated memory, with 36.0 GiB reserved memory.

Potential Use Cases

Given its fine-tuning on the xiaolesu/OsmosisProofling-v2-SFT dataset, this model is likely best suited for tasks aligned with the content and style of that specific dataset. Developers should evaluate its performance on their particular use cases to determine suitability.