Q1ngMang/LFM2.5-350M-PT-zh_CN
Q1ngMang/LFM2.5-350M-PT-zh_CN is a pre-trained language model based on LiquidAI's LFM2.5-350M architecture, specifically adapted for Chinese. This 350 million parameter model was pre-trained using the 0xDing/wikipedia-cn-20230720-filtered dataset, focusing on Chinese language understanding. It is optimized for foundational Chinese text processing tasks, serving as a base for further fine-tuning in Chinese NLP applications.
Loading preview...
LFM2.5-350M-PT-zh_CN Overview
This model, developed by Q1ngMang, is a pre-trained variant of LiquidAI's LFM2.5-350M, specifically adapted for the Chinese language. It leverages the 0xDing/wikipedia-cn-20230720-filtered dataset for its pre-training phase.
Key Training Details
- Base Model: LiquidAI/LFM2.5-350M
- Training Framework: LLaMA Factory
- Hardware: NVIDIA Tesla V100 SXM2 16GB
- Training Method: Pre-Training
- Learning Rate: 4.5e-4
- Training Epochs: 1.0
- Truncation Length: 1968 tokens
- Input Tokens Processed: 158,319,896
- Total Compute: 254,539,784 GF
- Training Loss: 3.8676
- Validation Loss: 3.5849
- Validation Perplexity: 36.0495
Licensing Considerations
The model's licensing involves a potential conflict between LFM 1.0, which requires a commercial license from LiquidAI for companies with annual revenues exceeding $10 million, and CC BY-SA 3.0, which permits commercial use. While the model prioritizes CC BY-SA 3.0, it's important to note that the requirement to purchase a commercial license from LiquidAI for high-revenue companies is not negated.