Duxiaoman-DI/Llama3-XuanYuan3-70B-Chat

XuanYuan3-70B-Chat: Financial Domain LLM

Duxiaoman-DI's XuanYuan3-70B-Chat is a 70 billion parameter language model, fine-tuned from Llama3-70B, with a strong focus on financial applications. It leverages extensive Chinese and English corpora for incremental pre-training and high-quality instruction data for SFT and reinforcement learning alignment.

Key Capabilities

• Financial Expertise: Excels in financial event interpretation, business analysis, investment research, and compliance, often surpassing other open-source models and competing with closed-source models like GPT-4o in financial benchmarks.
• Long Context Understanding: Supports a 16k context length, ideal for analyzing lengthy financial reports and building sophisticated financial agents.
• Technical Innovations: Features refined data organization and dynamic control strategies during pre-training and SFT, an "omnipotent financial reward model" (UFRM) trained with contrastive learning and inverse reinforcement learning, and an iterative reinforcement training method (PEI-RLHF) to enhance financial performance and alignment.

Ideal Use Cases

• Financial Research & Analysis: Generating insightful research reports, analyzing financial events, and performing in-depth business analysis.
• Compliance & Risk Management: Identifying and analyzing financial risks, providing compliant advice.
• Financial Agent Development: Building applications requiring long-context understanding in the financial sector.