itsZyn/ZynDwarf-1.0
ZynDwarf-1.0 is a 350 million parameter base model from the LFM2.5 family by Liquid AI, designed for on-device deployment. This hybrid model features 16 layers, a 32,768 token context length, and was trained on 28 trillion tokens. It is primarily intended for heavy fine-tuning for language-specific or domain-specific applications, supporting English and eight other languages.
Loading preview...
LFM2.5-350M-Base Overview
LFM2.5-350M-Base is a 350 million parameter pre-trained base model developed by Liquid AI, part of their LFM2.5 family of hybrid models. It is specifically engineered for on-device deployment, building upon the LFM2 architecture with extensive pre-training and reinforcement learning.
Key Capabilities & Features
- Compact Size: With 350 million parameters, it is optimized for efficient deployment on edge devices.
- Extended Context Window: Supports a substantial context length of 32,768 tokens.
- Multilingual Support: Trained to understand and generate text in English, Arabic, Chinese, French, German, Japanese, Korean, Portuguese, and Spanish.
- Robust Training: Pre-trained on a massive 28 trillion tokens, ensuring broad linguistic understanding.
- Hybrid Architecture: Incorporates 16 layers, including 10 double-gated LIV convolution blocks and 6 GQA blocks.
Recommended Use Cases
This base model is particularly suited for scenarios requiring significant customization and specialization:
- Heavy Fine-Tuning: Ideal for continued pre-training or supervised fine-tuning for specific tasks.
- Domain-Specific Assistants: Training on proprietary data to create medical, legal, or other specialized AI assistants.
- Language-Specific Applications: Adapting the model for optimal performance in particular languages, such as Japanese.
- Novel Post-Training Experimentation: A strong foundation for exploring new fine-tuning and post-training methodologies.
LFM2.5-350M-Base is supported by various inference frameworks, including Transformers, vLLM, llama.cpp, and MLX, facilitating flexible deployment and integration.