reaperdoesntknow/LFM2.5-1.2B-Distilled-SFT
LFM2.5-1.2B-Distilled-SFT by Convergent Intelligence LLC is a 1.2 billion parameter hybrid SSM + attention model, distilled from a 24B MoE teacher. It is specifically fine-tuned for structured STEM reasoning and formal logical inference, offering efficient performance at 239 tok/s on AMD CPU and fitting under 1GB RAM. This model excels at on-device logical inference and STEM problem-solving, making it suitable for edge and mobile deployments.
Loading preview...
Overview
LFM2.5-1.2B-Distilled-SFT is a 1.2 billion parameter hybrid model, combining State Space Model (SSM) and attention mechanisms. Developed by Convergent Intelligence LLC, it is designed for efficient, on-device structured STEM reasoning and formal logical inference. The model was created through a two-stage pipeline: knowledge distillation from a 24B MoE hybrid teacher (LFM2-24B-A2B) on STEM chain-of-thought data, followed by supervised fine-tuning (SFT) on a logical inference dataset.
Key Capabilities
- Hybrid Architecture: Utilizes a unique SSM + attention architecture, optimized for sequential state propagation crucial for logical inference.
- STEM Reasoning: Distilled from a larger teacher model using proof-weighted cross-entropy on 2,802 STEM CoT samples across linear algebra, differential equations, electromagnetism, mathematics, and classical mechanics.
- Logical Inference: Supervised fine-tuned on the LogicInference dataset, comprising approximately 54,607 instruction-response pairs, to activate its natural alignment with propositional logic chains.
- Efficiency: Achieves inference speeds of 239 tokens/second on AMD CPUs and 82 tokens/second on mobile NPUs, operating within sub-1GB RAM, making it suitable for edge and mobile deployment.
- Context Length: Supports a context length of 1024 tokens.
Good for
- On-device logical inference and STEM reasoning.
- Mobile, edge, and IoT deployment scenarios.
- Formal reasoning tasks and educational tutoring applications.
- Embedded inference pipelines requiring structured reasoning within strict memory constraints.