nbeerbower/Kartoffel-Deepfry-12B
Kartoffel-Deepfry-12B by nbeerbower is a 12 billion parameter language model, a fine-tuned version of mistral-nemo-kartoffel-12B. It was fine-tuned using QLoRA ORPO on the Schule-DPO dataset for 5 epochs, featuring a 32768 token context length. This model is optimized for tasks aligned with its specific DPO fine-tuning, making it suitable for applications requiring nuanced response generation based on preference learning.
Loading preview...
Overview
nbeerbower/Kartoffel-Deepfry-12B is a 12 billion parameter language model, building upon the mistral-nemo-kartoffel-12B architecture. This model has been specifically fine-tuned using the QLoRA ORPO (Quantized Low-Rank Adaptation with Odds Ratio Preference Optimization) method, a technique known for efficiently aligning models with human preferences. The training was conducted over 5 epochs on the Schule-DPO dataset, utilizing a single RTX A6000 GPU.
Key Capabilities
- Preference-aligned responses: Benefits from Direct Preference Optimization (DPO) fine-tuning, which helps in generating outputs that are preferred over others.
- Efficient fine-tuning: Leverages QLoRA for memory-efficient adaptation, making it accessible for training with limited hardware resources.
- Contextual understanding: Supports a substantial context length of 32768 tokens, enabling it to process and generate longer, more coherent texts.
Good for
- Applications requiring models to adhere to specific stylistic or content preferences.
- Research and development in preference-based fine-tuning techniques.
- Tasks where efficient fine-tuning on custom datasets is crucial, given its QLoRA implementation.