LiquidAI/LFM2.5-1.2B-Base
LiquidAI's LFM2.5-1.2B-Base is a 1.17 billion parameter pre-trained base model from the LFM2.5 family, designed for on-device deployment. It features a hybrid architecture with 16 layers (10 double-gated LIV convolution blocks + 6 GQA blocks) and was trained on 28 trillion tokens with a 32,768 token context length. This multilingual model supports English, Arabic, Chinese, French, German, Japanese, Korean, and Spanish, making it ideal for heavy fine-tuning for language-specific or domain-specific applications.
Loading preview...
LFM2.5-1.2B-Base: A Hybrid Model for On-Device AI
LFM2.5-1.2B-Base, developed by LiquidAI, is a 1.17 billion parameter pre-trained text-only checkpoint within the LFM2.5 family. This model is specifically engineered for on-device deployment, building upon the LFM2 architecture with extensive pre-training (28 trillion tokens) and reinforcement learning. It features a substantial context length of 32,768 tokens and a vocabulary size of 65,536, with a knowledge cutoff of mid-2024.
Key Capabilities & Features
- Hybrid Architecture: Utilizes 16 layers, combining 10 double-gated LIV convolution blocks and 6 GQA blocks.
- Multilingual Support: Trained to handle English, Arabic, Chinese, French, German, Japanese, Korean, and Spanish.
- Deployment Flexibility: Available in native format for Transformers and vLLM, as well as quantized GGUF for CPU inference (llama.cpp) and ONNX for cross-platform hardware-accelerated deployment.
Recommended Use Cases
LFM2.5-1.2B-Base is primarily recommended as a foundation for heavy fine-tuning. It is well-suited for:
- Developing language-specific assistants (e.g., Japanese).
- Creating domain-specific assistants (e.g., medical).
- Training on proprietary datasets.
- Experimenting with novel post-training approaches.