manu02/gemma-3-1b-it-4bit-lora-dpo-aligned

The manu02/gemma-3-1b-it-4bit-lora-dpo-aligned model is a 1 billion parameter instruction-tuned variant of Google's Gemma 3 architecture, fine-tuned using Direct Preference Optimization (DPO) on the ultrafeedback_binarized dataset. This model is optimized for text generation tasks, offering improved alignment with human preferences compared to its base model. It leverages 4-bit NF4 quantization and LoRA for efficient deployment and maintains a context length of 32768 tokens.