TinyDolphin-2.8.1-1.1b is a 1.1 billion parameter Llama-architecture model developed by Kearm, fine-tuned on the Dolphin 2.8 dataset by Eric Hartford. This compact model is designed for applications requiring restricted computation and memory footprints, building upon the pretraining efforts of the TinyLlama project. It leverages the same architecture and tokenizer as Llama 2, making it highly compatible with existing open-source projects.
Loading preview...
TinyDolphin-2.8.1-1.1b Overview
TinyDolphin-2.8.1-1.1b is a 1.1 billion parameter language model developed by Kearm, based on the Llama architecture. It is a fine-tuned version of the TinyLlama base model, specifically trained on the Dolphin 2.8 dataset by Eric Hartford. This iteration, version 2, involved increasing training epochs and refining the datasets used, building upon the initial training on two 3090 GPUs.
Key Characteristics
- Compact Size: With 1.1 billion parameters, it is designed for efficiency in applications with limited computational and memory resources.
- Llama 2 Compatibility: It utilizes the same architecture and tokenizer as Llama 2, ensuring broad compatibility with open-source projects built around the Llama ecosystem.
- Dolphin 2.8 Dataset: Fine-tuned on the Dolphin 2.8 dataset, suggesting a focus on instruction-following and conversational capabilities.
Good For
- Resource-Constrained Environments: Ideal for deployment where computational power or memory is limited, such as edge devices or specific local applications.
- Llama 2 Ecosystem Integration: Developers already working with Llama 2-based projects can easily integrate this model due to its architectural and tokenizer consistency.
- Instruction-Following Tasks: The fine-tuning on the Dolphin 2.8 dataset indicates suitability for tasks requiring adherence to instructions and generating coherent responses.