Magnum-v2-32b: Claude 3 Prose Replication

Magnum-v2-32b is a 32.5 billion parameter language model developed by natezahedi, built upon the Qwen1.5 32B architecture. Its primary objective is to emulate the high prose quality characteristic of Anthropic's Claude 3 Sonnet and Opus models, offering a sophisticated text generation capability.

Key Capabilities

• Advanced Prose Generation: Designed to produce text with a quality comparable to Claude 3 models, focusing on nuance and sophistication.
• Large Context Window: Supports a context length of 32768 tokens, enabling the processing and generation of extensive and complex narratives or conversations.
• Instruction-Tuned: Utilizes ChatML formatting for instruction tuning, ensuring responsive and coherent interactions.

Good For

• Creative Writing: Ideal for generating high-quality stories, articles, and other creative content that requires refined language.
• Sophisticated Conversational AI: Suitable for chatbots and virtual assistants where nuanced and human-like dialogue is crucial.
• Content Generation: Excellent for tasks requiring detailed and well-structured textual output across various domains.

Training Details

The model underwent full-parameter fine-tuning for 2 epochs using 8 NVIDIA H100 Tensor Core GPUs. The training leveraged a diverse set of filtered datasets, including Stheno, NobodyExistsOnTheInternet/claude_3.5s_single_turn_unslop_filtered, NobodyExistsOnTheInternet/PhiloGlanSharegpt, NobodyExistsOnTheInternet/Magpie-Reasoning-Medium-Subset, kalomaze/Opus_Instruct_25k, Nopm/Opus_WritingStruct, and a subset of Gryphe/Sonnet3.5-SlimOrcaDedupCleaned.