Dingdust/LFM2.5-350M-heretic

Model Overview

Dingdust/LFM2.5-350M-heretic is a 350 million parameter model derived from Liquid AI's LFM2.5-350M, specifically modified using the Heretic v1.4.0 tool to be a "decensored" version. It maintains the LFM2.5 architecture, which is designed for on-device deployment and fast edge inference, capable of 313 tok/s on AMD CPU and 188 tok/s on Snapdragon Gen4, running under 1GB of memory.

Key Differentiators

• Decensored Nature: Achieves a refusal rate of 7/100 compared to the original model's 89/100, indicating a significant reduction in content filtering.
• Optimized for Edge: Supports day-one deployment with llama.cpp, MLX, and vLLM, making it suitable for local and mobile applications.
• Robust Training: Built on LFM2.5's extended pre-training of 28T tokens and large-scale multi-stage reinforcement learning.
• Multilingual Support: Handles English, Arabic, Chinese, French, German, Japanese, Korean, Portuguese, and Spanish.
• Tool Use Capabilities: Supports function calling with a ChatML-like format, allowing for structured interactions and integration with external tools.

Performance Highlights

Benchmarks show LFM2.5-350M outperforming its predecessor, LFM2-350M, and other models in its class across various metrics like GPQA Diamond, IFEval, and specific domain benchmarks such as CaseReportBench and BFCLv3. It features a 32,768-token context length and a 65,536-token vocabulary size.

Good For

• Data Extraction: Efficiently pulls specific information from text.
• Structured Outputs: Generates responses in predefined formats.
• Tool Use: Integrates seamlessly with external functions and APIs for complex tasks.
• On-device Applications: Ideal for scenarios requiring local, low-latency inference on edge devices.