TinyDolphin-2.8.2-1.1b-laser by QuixiAI is a 1.1 billion parameter language model based on the TinyLlama architecture, fine-tuned on the Dolphin 2.8 dataset. This model utilizes a 'laser' denoising technique to enhance performance and is optimized for applications requiring a compact footprint. It maintains the Llama 2 architecture and tokenizer for broad compatibility.
Loading preview...
QuixiAI/TinyDolphin-2.8.2-1.1b-laser Overview
TinyDolphin-2.8.2-1.1b-laser is a 1.1 billion parameter model developed by QuixiAI, building upon the TinyLlama base model. It has been fine-tuned using the Dolphin 2.8 dataset by Eric Hartford, with increased epochs and refined datasets. A key differentiator is the application of QuixiAI's proprietary 'laser' denoising technique (from cognitivecomputations/laserRMT) to improve model quality.
Key Characteristics
- Base Architecture: Derived from TinyLlama, which adopts the exact architecture and tokenizer of Llama 2, ensuring broad compatibility with existing Llama-based projects.
- Parameter Count: 1.1 billion parameters, making it compact and suitable for applications with restricted computational and memory resources.
- Training: Fine-tuned on the Dolphin 2.8 dataset, known for its instruction-following capabilities.
- Denoising: Incorporates a 'laser' technique for enhanced model quality.
Use Cases
This model is particularly well-suited for scenarios where a compact yet capable language model is required. Its Llama 2 compatibility allows for seamless integration into various open-source projects. The Dolphin 2.8 dataset training suggests proficiency in instruction-following tasks, while its small size makes it ideal for edge devices or applications with limited resources.