KaeriJenti/kaori-70b-v1
KaeriJenti/kaori-70b-v1 is a 69 billion parameter language model fine-tuned by Kaeri and Jenti using QLoRA on a dataset comprising Open-Platypus, dolphin, and OpenOrca. This model, with a 32768 token context length, is optimized for general language understanding and generation tasks. Its training methodology focuses on efficient tuning, making it suitable for applications requiring robust performance from a large language model.
Loading preview...
Model Overview
KaeriJenti/kaori-70b-v1 is a 69 billion parameter language model developed by Kaeri and Jenti. It was fine-tuned using the QLoRA method, leveraging the LLaMA-Efficient-Tuning framework. The training was conducted over 1 epoch with a batch size of 8, utilizing four A100 GPUs (80GB each).
Key Training Details
- Fine-tuning Type: QLoRA
- GPUs Used: 4 x A100 (80GB)
- Epochs: 1
- Batch Size: 8
Datasets
The model's fine-tuning incorporated a diverse set of instruction-following and conversational datasets, including:
- Open-Platypus
- dolphin
- OpenOrca
This selection of datasets aims to enhance the model's ability to understand and generate human-like text across various prompts and tasks. The 32768 token context length allows for processing and generating longer sequences of text, making it versatile for complex applications.