AbdallahNasir/gender-mt-llama-3.2-3b-cpo-fewshot
The AbdallahNasir/gender-mt-llama-3.2-3b-cpo-fewshot model is a 3.2 billion parameter language model, based on the Llama 3.2 architecture, fine-tuned using CPO (Constitutional Preference Optimization) on TED gender pairs. It is specifically optimized for gender-aware machine translation tasks, utilizing a 3-shot prompting strategy. This model is designed for applications requiring nuanced handling of gender in text generation and translation, with a context length of 32768 tokens.
Loading preview...
Overview
The AbdallahNasir/gender-mt-llama-3.2-3b-cpo-fewshot model is a specialized language model built upon the Llama 3.2 architecture, featuring 3.2 billion parameters and a substantial context length of 32768 tokens. Its core distinction lies in its fine-tuning methodology: it leverages Constitutional Preference Optimization (CPO) specifically on TED gender pairs. This training approach, combined with a 3-shot prompting strategy, aims to enhance its performance in gender-aware machine translation and text generation.
Key Capabilities
- Gender-Aware Translation: Optimized to handle gender nuances in text, particularly within the context of machine translation.
- CPO Fine-tuning: Utilizes Constitutional Preference Optimization for alignment, trained on specific gender-paired data from TED talks.
- Few-shot Learning: Designed to operate effectively with a 3-shot prompting mechanism, demonstrating one male speaker, one female speaker, and one neutral prompt.
Good for
- Machine Translation Research: Ideal for researchers exploring gender bias mitigation and gender-aware translation techniques.
- Content Generation with Gender Sensitivity: Suitable for applications where generated text needs to reflect or respect gender distinctions accurately.
- Experiments with CPO: Provides a practical example of CPO applied to a specific linguistic challenge.