reaperdoesntknow/LFM2.5-1.2B-Distilled-SFT

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.2BQuant:BF16Context Size:32kPublished:Mar 25, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

LFM2.5-1.2B-Distilled-SFT is a 1.2 billion parameter hybrid (SSM + attention) model developed by Convergent Intelligence LLC: Research Division. It was created through knowledge distillation from a 24B MoE hybrid teacher on STEM chain-of-thought data, followed by supervised fine-tuning on logical inference. This model excels at on-device logical inference and STEM reasoning, offering high efficiency with 239 tok/s on AMD CPU and fitting under 1GB of RAM.

Loading preview...

Overview

LFM2.5-1.2B-Distilled-SFT is a 1.2 billion parameter hybrid model, combining State Space Model (SSM) and attention mechanisms. Developed by Convergent Intelligence LLC: Research Division, it's notable for being the first proof-weighted distillation and SFT pipeline on a non-transformer architecture. The model was trained in two stages: first, knowledge distillation from a 24B MoE hybrid teacher (LFM2-24B-A2B) on STEM chain-of-thought data, and then supervised fine-tuning on logical inference using the LogicInference dataset. This unique approach leverages the LFM hybrid architecture's inductive bias, particularly the SSM components' strength in sequential state propagation, which aligns well with formal logical inference.

Key Capabilities

  • Efficient STEM Reasoning: Distilled from a 24B teacher on 2,802 STEM CoT samples (Linear Algebra, Differential Equations, Electromagnetism, Mathematics, Classical Mechanics).
  • Formal Logical Inference: Fine-tuned on ~54,607 instruction-response pairs from the LogicInference dataset, making it proficient in propositional logic.
  • High Performance & Low Resource Usage: Achieves 239 tokens/second on AMD CPU and operates with less than 1GB of RAM, making it suitable for edge and mobile deployments.
  • Hybrid Architecture: Utilizes a LFM2.5 hybrid (SSM + attention) architecture, offering a balance of speed and reasoning capabilities.

Good For

  • On-device logical inference and STEM reasoning.
  • Mobile, edge, and IoT deployment scenarios.
  • Formal reasoning tasks and educational tutoring applications.
  • Embedded inference pipelines requiring structured reasoning within strict memory constraints.

Finetunes
1 models