ermiaazarkhalili/Ornith-1.5-9B-SFT-Fable5-Glint
The ermiaazarkhalili/Ornith-1.5-9B-SFT-Fable5-Glint is a 9.4 billion parameter language model, fine-tuned from ornith-ai/Ornith-1.5-9B using LoRA. This model is specifically supervised fine-tuned on the private ermiaazarkhalili/Fable-5-Glint-Clean dataset, leveraging the Qwen3_5ForConditionalGeneration architecture. It is optimized for instruction-following tasks, inheriting a 32768 token context length, and is suitable for applications requiring specialized instruction adherence.
Loading preview...
Model Overview
ermiaazarkhalili/Ornith-1.5-9B-SFT-Fable5-Glint is a 9.4 billion parameter language model, developed by ermiaazarkhalili. It is a LoRA (Low-Rank Adaptation) fine-tune of the ornith-ai/Ornith-1.5-9B base model, utilizing the Qwen3_5ForConditionalGeneration architecture.
Key Characteristics
- Base Model: Built upon
ornith-ai/Ornith-1.5-9B. - Fine-tuning: Supervised fine-tuned using the private
ermiaazarkhalili/Fable-5-Glint-Cleandataset. - Methodology: Employs LoRA fine-tuning via Unsloth and TRL, with specific settings including a LoRA rank of 16, a learning rate of 0.0002, and training over 3 epochs.
- Context Length: Supports a maximum sequence length of 4096 tokens during training.
- Precision: Trained using 4-bit QLoRA for efficiency.
Intended Use Cases
- Instruction Following: Primarily designed for tasks requiring adherence to specific instructions, based on its fine-tuning dataset.
- Research and Development: Suitable for researchers and developers exploring models fine-tuned on specialized, private datasets.
Limitations
- No Benchmark Evaluation: This checkpoint has not undergone downstream benchmark evaluation; performance metrics are based solely on training loss observations.
- Inherited Biases: Inherits biases, knowledge cutoff, and potential failure modes from its base model.
- Specialized Fine-tuning: Its behavior outside the distribution of the single instruction-following dataset it was fine-tuned on is untested.
- Merged Adapters: LoRA adapters are merged into the base weights, meaning the fine-tune cannot be detached from the base model.