Predonia-24B-V2.1 Overview

Ateron/Predonia-24B-V2.1 is a 24 billion parameter language model featuring a substantial 32768 token context length. This iteration, developed by Ateron, focuses on performance enhancements achieved through a distinct model merging strategy.

Key Capabilities

• Advanced Merge Method: Employs a 'ties' merge method, combining components from 'Precog' and 'Cydonia 4.3' models. This technique involves density and weight gradients for fine-tuned integration.
• Improved Performance: The V2.1 release specifically highlights a different merge approach intended to yield better overall performance compared to previous versions.
• Configuration Transparency: The model's merging configuration, including parameters like density, weight, normalize, and int8_mask, is explicitly detailed, offering insight into its construction.

Good For

• Developers interested in models built with advanced merging techniques.
• Use cases requiring a 24B parameter model with a large context window and potentially optimized performance from model fusion.