Hugging Face Anthracite-org's Magnum-v2-72b is a 72.7 billion parameter instruction-tuned causal language model, fine-tuned on Qwen-2 72B Instruct. This model is specifically designed to replicate the prose quality of Claude 3 models, excelling in generating high-quality, nuanced text. It features a substantial 131072 token context length, making it suitable for complex conversational and long-form content generation tasks.
Magnum-v2-72b: Claude 3 Prose Quality
Magnum-v2-72b is a 72.7 billion parameter language model developed by Anthracite-org, fine-tuned on the Qwen-2 72B Instruct architecture. Its primary objective is to emulate the sophisticated prose quality found in Claude 3 models (Sonnet and Opus).
Key Capabilities & Features
- Prose Generation: Optimized for generating high-quality, nuanced, and human-like text, mirroring the style of advanced commercial LLMs.
- Large Context Window: Features a 131072 token context length, enabling the processing and generation of extensive and complex interactions.
- Instruction-Tuned: Utilizes ChatML formatting for instruction-tuned interactions, ensuring responsive and coherent dialogue.
- Robust Training: Fine-tuned over two epochs using 8x AMD Instinct™ MI300X Accelerators, incorporating techniques like weight decay (0.01) and a peak learning rate of 4e-6 to prevent overfitting.
- Sample Packing: Employs 16k token sample packing during training, an increase from previous runs, to enhance learning efficiency.
Performance Highlights
Evaluations on the Open LLM Leaderboard show an average score of 41.15. Notable scores include 75.60 on IFEval (0-Shot) and 57.85 on BBH (3-Shot), indicating strong instruction following and reasoning capabilities.
Ideal Use Cases
- Creative Writing: Generating stories, articles, or other long-form content requiring high prose quality.
- Advanced Chatbots: Developing conversational agents that produce sophisticated and natural-sounding responses.
- Content Creation: Assisting with tasks that demand nuanced language and stylistic consistency.