LiquidAI/LFM2.5-1.2B-Base
LFM2.5-1.2B-Base by LiquidAI is a 1.17 billion parameter pre-trained base model from the LFM2.5 family, designed for on-device deployment. It features a hybrid architecture with 16 layers and was trained on a substantial 28 trillion tokens, supporting a 32,768-token context length. This model is optimized for heavy fine-tuning across various applications, including language-specific or domain-specific assistants, and training on proprietary datasets.
Loading preview...
LFM2.5-1.2B-Base: A Hybrid Model for On-Device AI
LFM2.5-1.2B-Base is a 1.17 billion parameter pre-trained base model developed by LiquidAI, part of the LFM2.5 family of hybrid models. Building upon the LFM2 architecture, it incorporates extended pre-training and reinforcement learning, specifically designed for on-device deployment.
Key Technical Specifications
- Parameters: 1.17 billion
- Architecture: 16 layers (10 double-gated LIV convolution blocks + 6 GQA blocks)
- Training Budget: 28 trillion tokens
- Context Length: 32,768 tokens
- Vocabulary Size: 65,536
- Knowledge Cutoff: Mid-2024
- Multilingual Support: English, Arabic, Chinese, French, German, Japanese, Korean, Spanish
Deployment and Fine-Tuning
This base model serves as the foundation for other LFM2.5-1.2B variants, including instruction-tuned, reasoning, and Japanese-optimized models. It is provided in multiple formats for flexible deployment:
- Native format: Ideal for fine-tuning or inference with Transformers and vLLM.
- GGUF: Quantized for
llama.cpp, optimized for CPU inference and local deployment. - ONNX: For cross-platform deployment and hardware-accelerated inference across diverse environments.
Recommended Use Cases
LFM2.5-1.2B-Base is primarily recommended for tasks requiring heavy fine-tuning. This includes:
- Developing language-specific assistants (e.g., Japanese).
- Creating domain-specific assistants (e.g., medical).
- Training on proprietary datasets.
- Experimenting with novel post-training approaches.
LiquidAI provides extensive documentation and Colab notebooks for inference and various fine-tuning methods (CPT, SFT, DPO, GRPO) using frameworks like Unsloth and TRL.