Genie2k/qwen3-0.6b-sft
Genie2k/qwen3-0.6b-sft is an 0.8 billion parameter Supervised Fine-Tuned (SFT) LoRA adapter for the Qwen3-0.6B base model, developed by Agha Salik Ali and Uzair Nadeem. This adapter enhances instruction-following capabilities and stylistic formatting for conversational interactions and natural language generation. It was fine-tuned on the vicgalle/alpaca-gpt4 dataset, focusing on aligning the model with human instructional prompts.
Loading preview...
Model Overview
Genie2k/qwen3-0.6b-sft is a Supervised Fine-Tuned (SFT) LoRA adapter designed to enhance the instruction-following and stylistic formatting of the Qwen/Qwen3-0.6B base model. Developed by Agha Salik Ali and Uzair Nadeem, this adapter applies Low-Rank Adaptation (LoRA) to align the base model with human instructional prompts.
Key Capabilities
- Improved Instruction Following: The adapter is specifically trained to better understand and respond to user instructions.
- Enhanced Stylistic Formatting: Aims to produce more coherent and appropriately formatted text.
- Conversational Interactions: Suitable for dialogue-based applications.
- Natural Language Generation: Can be used for generating various forms of text.
Training Details
The model was fine-tuned on the vicgalle/alpaca-gpt4 dataset, using the standard Alpaca chat template with a 90/10 train-evaluation split. Training utilized SFTTrainer from the trl library, employing fp16 mixed precision and specific LoRA hyperparameters (Rank: 64, Alpha: 128, Dropout: 0.1). Evaluation metrics included BLEU (5.3696) and BERTScore F1 (0.8529).
Limitations
Due to the small 0.6B parameter count of the base model, this fine-tuned version is prone to persistent hallucinations and format collapse (e.g., repetitive loops) when handling complex creative tasks. It is not recommended for applications requiring high factual accuracy without external retrieval mechanisms.