juanjucm/gemma-3-270m-dpo-capybara
The juanjucm/gemma-3-270m-dpo-capybara is a 270 million parameter Gemma-3 model fine-tuned by juanjucm using Direct Preference Optimization (DPO). This model is designed for text generation tasks, leveraging its DPO training to produce high-quality, preference-aligned outputs. It offers a substantial 32,768 token context length, making it suitable for applications requiring extensive contextual understanding.
Loading preview...
Model Overview
The juanjucm/gemma-3-270m-dpo-capybara is a 270 million parameter language model based on Google's Gemma-3 architecture. It has been specifically fine-tuned by juanjucm using the Direct Preference Optimization (DPO) method, which is detailed in the paper "Direct Preference Optimization: Your Language Model is Secretly a Reward Model". This training approach aims to align the model's outputs more closely with human preferences, enhancing its performance in generating desirable text.
Key Capabilities
- Preference-Aligned Text Generation: Utilizes DPO training to produce outputs that are optimized based on preferences, potentially leading to more helpful or engaging responses.
- Extended Context Window: Features a significant context length of 32,768 tokens, allowing it to process and generate text based on very long inputs.
- Gemma-3 Base: Built upon the efficient and capable Gemma-3 architecture, providing a solid foundation for its language understanding and generation abilities.
Training Details
The model was trained using the TRL (Transformers Reinforcement Learning) library, specifically leveraging its DPO implementation. This method directly optimizes a policy to satisfy preferences without needing an explicit reward model. The training framework versions include TRL 1.8.0, Transformers 5.13.1, Pytorch 2.8.0+cu128, Datasets 5.0.0, and Tokenizers 0.22.2.
Use Cases
This model is well-suited for applications where generating high-quality, preference-aligned text is crucial, especially when dealing with longer conversational histories or documents due to its extended context window. Developers can integrate it using the Hugging Face transformers library for various text generation tasks.