nbeerbower/Dumpling-Qwen2.5-7B-1k-r16
nbeerbower/Dumpling-Qwen2.5-7B-1k-r16 is a 7.6 billion parameter Qwen2.5-based language model developed by nbeerbower. It is a QLoRA ORPO fine-tune, part of an experiment investigating the effects of changing LoRA rank on model performance. This model is specifically trained on a diverse collection of DPO datasets, making it suitable for tasks requiring nuanced instruction following and preference alignment.
Loading preview...
Overview
nbeerbower/Dumpling-Qwen2.5-7B-1k-r16 is a 7.6 billion parameter language model built on the Qwen2.5 architecture. This model is an experimental QLoRA ORPO fine-tune, specifically designed to investigate how varying the LoRA rank impacts the tuning process and resulting model capabilities. It was trained using 2x RTX 3090 GPUs over two epochs.
Key Capabilities
- Preference Alignment: Fine-tuned using a diverse set of Direct Preference Optimization (DPO) datasets, including
GreatFirewall-DPO,Schule-DPO,Purpura-DPO,Arkhaios-DPO,truthy-dpo-v0.1,physical-reasoning-dpo,Date-DPO-NoAsterisks,Prude-Phi3-DPO,HelpSteer2-DPO(1,000 samples),gutenberg-dpo-v0.1,gutenberg2-dpo, andgutenberg-moderne-dpo. - Efficient Fine-tuning: Utilizes QLoRA with a rank of 16 (
r=16) andlora_alpha=32, enabling efficient fine-tuning of the basenbeerbower/EVA-abliterated-Qwen2.5-7Bmodel. - Experimental Focus: Serves as a testbed for understanding the impact of LoRA rank on model performance and characteristics.
Good for
- Researchers and developers interested in the effects of LoRA rank on fine-tuned models.
- Applications requiring a model with strong preference alignment and instruction following capabilities derived from DPO training.
- Tasks benefiting from a Qwen2.5-based model fine-tuned on a broad range of DPO datasets.