Overview
Xiangxin-2XL-Chat-1048k is a 70 billion parameter conversational model developed by Xiangxin AI. It is built upon the Meta Llama-3-70B-Instruct architecture and integrates expanded context capabilities from Gradient AI, supporting a substantial context length of up to 1 million words. The model was specifically trained using ORPO (Odds Ratio Preference Optimization) with a proprietary Chinese value-aligned dataset, which is not publicly disclosed due to its sensitive nature. This training methodology aims to enhance the model's Chinese language proficiency and align its responses with Chinese cultural values.
Key Capabilities & Performance
- Extended Context Window: Supports an impressive context length of 1,048,000 tokens (approximately 1 million words), enabling processing of very long inputs and generating coherent, contextually relevant responses over extended dialogues.
- Enhanced Chinese Proficiency: Fine-tuned with a specialized Chinese value-aligned dataset, leading to improved performance in Chinese language understanding and generation.
- Benchmark Performance: Achieved an average score of 70.22 across eight diverse benchmarks, including ARC, HellaSwag, MMLU, TruthfulQA_mc2, Winogrande, GSM8K_flex, CMMLU, and C-EVAL. This score surpasses the Gradientai-Llama-3-70B-Instruct-Gradient-1048k model, indicating strong general and Chinese-specific reasoning abilities.
- Proprietary Training: Utilizes ORPO training with an in-house developed dataset, ensuring unique alignment and performance characteristics.
Use Cases
- Long-form Chinese Content Generation: Ideal for applications requiring the processing and generation of extensive Chinese texts, such as document summarization, detailed report writing, or complex conversational agents.
- Culturally Aligned Chatbots: Suitable for developing chatbots and virtual assistants that need to interact in Chinese with a strong understanding of cultural nuances and values.
- Research and Development: Can serve as a robust base model for further fine-tuning on specific industry or domain-specific tasks within the Chinese language context.