AyoubChLin/LFM2.5-1.2B-hermes-agent

Overview

AyoubChLin/LFM2.5-1.2B-hermes-agent is a 1.2 billion parameter model developed by AyoubChLin, built upon the LiquidAI/LFM2.5-1.2B-Instruct base model. It has been specifically fine-tuned using Supervised Fine-Tuning (SFT) with LoRA on agentic reasoning traces from the lambda/hermes-agent-reasoning-traces dataset, configured for Kimi. This specialization makes it highly effective for tasks requiring structured interaction and tool utilization.

Key Capabilities

• Agentic Behavior: Designed to understand and execute complex multi-turn reasoning, particularly in agent-based workflows.
• Function Calling: Optimized for interpreting and generating function calls, facilitating interaction with external tools.
• ChatML Format: Utilizes the LFM2.5 ChatML structure with special tokens for robust assistant-style chat.
• Extended Context: Supports a maximum sequence length of 16,384 tokens during training, enabling processing of longer interactions.

Training Details

The model was trained for one epoch on a preprocessed dataset of 4,987 samples, using an adamw_8bit optimizer and a learning rate of 2e-4. LoRA was applied with r=32 and alpha=64 targeting key attention and feed-forward modules. Training was conducted on an NVIDIA B200 GPU, achieving an evaluation token accuracy of 91.41%.

Intended Use Cases

• Assistant-style chat: Engaging in conversational AI that requires structured responses.
• Agent/tool-use workflows: Implementing AI agents that interact with tools or external systems.
• Function-calling style prompting: Generating and interpreting function calls within prompts for dynamic interactions.

Limitations

• Strongly optimized for the specific trace style of the kimi dataset, which may limit generalization to unrelated domains.
• Tool-calling quality is dependent on the quality of the provided prompt and tool schema.