LiquidAI/LFM2.5-350M-Base
LFM2.5-350M-Base by LiquidAI is a 350 million parameter pre-trained base model from the LFM2.5 family, designed for on-device deployment. It features a hybrid architecture with 16 layers and a 32,768 token context length, trained on 28 trillion tokens. This model is specifically recommended for heavy fine-tuning in language-specific or domain-specific applications, or for experimenting with novel post-training methods.
Loading preview...
LFM2.5-350M-Base: A Hybrid Model for On-Device Deployment
LiquidAI's LFM2.5-350M-Base is a 350 million parameter pre-trained model, part of the LFM2.5 family, engineered for efficient on-device deployment. Building upon the LFM2 architecture, it incorporates extended pre-training and reinforcement learning, distinguishing it as a hybrid model.
Key Characteristics
- Architecture: Features 16 layers, combining 10 double-gated LIV convolution blocks and 6 GQA blocks.
- Training: Pre-trained on an extensive 28 trillion tokens, ensuring a broad knowledge base.
- Context Length: Supports a substantial context window of 32,768 tokens.
- Multilingual Support: Capable of processing English, Arabic, Chinese, French, German, Japanese, Korean, Portuguese, and Spanish.
- Knowledge Cutoff: Information up to mid-2024.
Recommended Use Cases
This base model is primarily intended for scenarios requiring significant customization and adaptation:
- Heavy Fine-Tuning: Ideal for developing highly specialized models.
- Domain-Specific Assistants: Creating assistants tailored for particular fields, such as medical or legal.
- Language-Specific Applications: Adapting the model for specific languages, like Japanese.
- Proprietary Data Training: Leveraging private datasets for unique applications.
- Experimental Post-Training: Exploring new approaches in model refinement and optimization.