KaeriJenti/kaori-34b-v4
KaeriJenti/kaori-34b-v4 is a 34 billion parameter language model fine-tuned by Kaeri and Jenti, built upon an unspecified base architecture. It was trained using a Supervised Fine-Tuning (SFT) strategy on a curated mix of Open-Platypus, Dolphin, and OpenOrca datasets. This model is specifically designed with careful data contamination filtering to avoid common benchmark datasets, making it suitable for general-purpose conversational AI and instruction-following tasks where benchmark integrity is crucial.
Loading preview...
KaeriJenti/kaori-34b-v4 Overview
KaeriJenti/kaori-34b-v4 is a 34 billion parameter language model developed through supervised fine-tuning (SFT) by Kaeri and Jenti. The model was trained using a LoRA finetuning type over 3 epochs on A100 GPUs.
Training Details
The model's training regimen focused on a specific blend of datasets:
- Open-Platypus: 100% of this dataset was utilized.
- Dolphin: 5% of this dataset was incorporated.
- OpenOrca: 10% of this dataset was used.
Data Contamination Filtering
A key aspect of kaori-34b-v4's development involved rigorous data contamination filtering. The creators explicitly excluded GSM8k samples and applied similarity filtering against a list of common benchmark tasks, including cot_gsm8k, drop, winogrande, ai2_arc, and hellaswag. This careful approach aims to ensure the model's performance is not artificially inflated by exposure to benchmark data during training.
Use Cases
This model is well-suited for applications requiring a general-purpose instruction-following language model, particularly where the integrity of evaluation against standard benchmarks is a concern due to the deliberate filtering of common benchmark datasets from its training data.