ermiaazarkhalili/LFM2.5-350M-SFT-Fable5-Glint
The ermiaazarkhalili/LFM2.5-350M-SFT-Fable5-Glint is a 354 million parameter causal language model, fine-tuned by ermiaazarkhalili using LoRA on the LiquidAI/LFM2.5-350M base model. It was supervised fine-tuned on a private dataset, ermiaazarkhalili/Fable-5-Glint-Clean, to specialize in instruction-following tasks. This model offers a 32768 token context length and is optimized for specific instruction-following behaviors derived from its training data.
Loading preview...
Model Overview
ermiaazarkhalili/LFM2.5-350M-SFT-Fable5-Glint is a 354 million parameter language model developed by ermiaazarkhalili. It is a LoRA (Low-Rank Adaptation) supervised fine-tune of the LiquidAI/LFM2.5-350M base model, utilizing the Lfm2ForCausalLM architecture. The fine-tuning was performed using Unsloth and TRL on a private dataset, ermiaazarkhalili/Fable-5-Glint-Clean, with a maximum sequence length of 4096 tokens.
Key Characteristics
- Base Model:
LiquidAI/LFM2.5-350M - Parameters: 354M
- Fine-tuning Method: LoRA supervised fine-tuning (r=16, alpha=16) in 4-bit QLoRA precision.
- Training Data:
ermiaazarkhalili/Fable-5-Glint-Clean(private instruction-following dataset). - Context Length: Inherits the base model's context capabilities, with training configured for 4096 tokens.
Limitations
- No Benchmarks: This checkpoint has not undergone any downstream benchmark evaluation; only training-loss observations are reported.
- Inherited Biases: It inherits biases, knowledge cutoff, and failure modes from its base model.
- Specialized Behavior: Fine-tuned on a single instruction-following dataset, its behavior outside this distribution is untested.
- Merged Adapters: LoRA adapters are merged into the base weights, preventing detachment from this fine-tune.