MTSAIR/Cotype-Nano: A Lightweight, Efficient LLM
MTSAIR/Cotype-Nano is a 1.5 billion parameter large language model specifically engineered for resource-constrained environments. Its primary design goal is to deliver fast and efficient interaction while maintaining high performance, making it ideal for applications where computational resources are limited.
Key Capabilities & Training
- Optimized for Efficiency: Designed to operate effectively with minimal resources, ensuring quick response times.
- Two-Stage Training: The model underwent a two-stage training process:
- Stage 1: MLP layers were trained on a combination of mathematics and code datasets.
- Stage 2: The entire model was further trained on internal and open synthetic instructional datasets, enhancing its ability to follow instructions.
- Multilingual Support: The README provides examples and system prompts in Russian, indicating potential multilingual capabilities, particularly for Russian language tasks.
Performance Highlights
- ru-llm-arena Score: Achieved a score of 30.2 (local measurement) on the ru-llm-arena benchmark, outperforming several other models in its class, including
vikhr-it-5.3-fp16-32k and Qwen-Qwen2.5-1.5B-Instruct. - 4-bit Quantization: A 4-bit quantized version,
Cotype-Nano-4bit, also performed competitively with a score of 22.5.
Good For
- Edge Devices & Low-Resource Deployments: Its lightweight nature makes it suitable for deployment on devices with limited memory and processing power.
- Applications Requiring Fast Inference: Ideal for scenarios where quick, real-time responses are crucial.
- Instruction Following: Benefits from training on synthetic instructional datasets, making it capable of generating detailed and expanded answers to user queries.
- Code Generation & Mathematical Tasks: Initial training on mathematics and code suggests proficiency in these domains.