nbeerbower/Dumpling-Qwen2.5-7B-1k-r64-2e-5
nbeerbower/Dumpling-Qwen2.5-7B-1k-r64-2e-5 is a 7.6 billion parameter language model based on the Qwen2.5 architecture, fine-tuned using a QLoRA ORPO method. This model is an experimental variant investigating the effects of increased LoRA rank (r=64) and learning rate (2e-5) on the base nbeerbower/EVA-abliterated-Qwen2.5-7B model. It was trained on a diverse collection of DPO datasets, focusing on specific response characteristics, making it suitable for research into fine-tuning methodologies and their impact on model behavior.
Loading preview...
Dumpling-Qwen2.5-7B-1k-r64-2e-5 Overview
This model, developed by nbeerbower, is an experimental 7.6 billion parameter language model built upon the Qwen2.5 architecture. It serves as a research vehicle to explore the impact of specific fine-tuning parameters on model performance and behavior. The base model, nbeerbower/EVA-abliterated-Qwen2.5-7B, was subjected to a QLoRA ORPO tune, with a particular focus on adjusting the LoRA rank and learning rate.
Key Capabilities & Experimental Focus
- Parameter Investigation: This model is specifically designed to investigate the effects of changing LoRA rank (set to
r=64) and an increased learning rate (2e-5) compared to previous iterations. - Diverse DPO Training: Fine-tuned on a curated selection of Direct Preference Optimization (DPO) datasets, including nbeerbower/GreatFirewall-DPO, nbeerbower/Schule-DPO, nbeerbower/Purpura-DPO, nbeerbower/Arkhaios-DPO, and others, to shape specific response characteristics.
- QLoRA ORPO Method: Utilizes a QLoRA ORPO tuning method for efficient fine-tuning.
Good for
- Research into Fine-tuning: Ideal for researchers and developers interested in understanding how LoRA rank and learning rate adjustments influence the outcomes of DPO fine-tuning.
- Comparative Analysis: Useful for comparing against other experimental versions, such as v1 here, to observe the effects of parameter changes.
- Exploring DPO Dataset Impact: Provides a platform to study the collective influence of various DPO datasets on a Qwen2.5 base model.