ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jan 30, 2025License:mitArchitecture:Transformer0.0K Open Weights Featherless Exclusive Cold

ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M is a 7.61 billion parameter merged language model based on the Qwen2.5-7B series, developed by ZeroXClem. It is specifically designed for ultra-long context processing, supporting up to 1 million tokens, and excels in enhanced reasoning, immersive roleplaying, and deep contextual awareness. This model leverages an optimized layer-wise weight averaging (MODEL_STOCK) merge method to combine capabilities from multiple fine-tuned Qwen2.5-7B variants, making it suitable for complex logical thinking and character-based storytelling.

Loading preview...

Overview

ZeroXClem/Qwen2.5-7B-CelestialHarmony-1M is a custom merged language model built upon the Qwen2.5-7B series, specifically designed to enhance reasoning, roleplaying, and long-context capabilities. This 7.61 billion parameter model stands out by supporting an impressive 1 million token context length, making it highly effective for processing extensive texts and engaging in deep, sustained interactions.

Key Capabilities

  • Ultra-Long Context Handling: Processes sequences up to 1,010,000 tokens, ideal for applications requiring extensive memory and contextual understanding.
  • Enhanced Roleplaying: Offers immersive character-based storytelling with deep contextual awareness, making it suitable for interactive narrative generation.
  • Structured Reasoning: Capable of logical thinking, particularly when guided by <think> tags, which can be used to prompt structured thought processes.
  • Optimized Merge Strategy: Utilizes the MODEL_STOCK method, averaging weights from multiple fine-tuned models (including bunnycore/Qwen2.5-7B-RRP-1M, Triangle104/Q2.5-Instruct-1M_Harmony, and Sakalti/SJT-7B-1M) to achieve a balanced and performant model.

Good for

  • Applications requiring extensive context: Such as summarizing long documents, analyzing large codebases, or maintaining complex conversational states over extended periods.
  • Immersive storytelling and character-driven interactions: Its fine-tuning for roleplaying makes it excellent for generating dynamic and consistent character responses.
  • Tasks benefiting from structured thought: When logical reasoning is critical, the model's ability to process <think> tags can guide it towards more coherent and structured outputs.
  • Developers seeking a specialized 7B model: Offers a unique blend of capabilities not commonly found in models of its size, particularly its exceptional context window.

Finetunes
3 models