CultriX/Qwen2.5-14B-Ultimav2

TEXT GENERATIONConcurrency Cost:1Model Size:14.8BQuant:FP8Ctx Length:32kPublished:Feb 4, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

CultriX/Qwen2.5-14B-Ultimav2 is a 14.8 billion parameter language model created by CultriX using the SLERP merge method. This model combines several specialized Qwen2.5-14B variants and other 14B models, with a focus on improving reasoning benchmarks, general performance, and instruction following (IFEval). It leverages a 32768 token context length, making it suitable for tasks requiring extensive context processing.

Loading preview...

CultriX/Qwen2.5-14B-Ultimav2: A Merged Language Model

CultriX/Qwen2.5-14B-Ultimav2 is a 14.8 billion parameter language model developed by CultriX. It was created using the SLERP (Spherical Linear Interpolation) merge method, combining multiple pre-trained models to enhance specific capabilities. This approach allows for the integration of strengths from various specialized models into a single, more versatile model.

Key Merge Components and Focus Areas

This model is a sophisticated merge of several 14B parameter models, each contributing to different aspects of performance:

  • CultriX/Qwen2.5-14B-Hyperionv5 & v3: Primarily aimed at improving reasoning benchmarks and overall general performance.
  • arcee-ai/Virtuoso-Small-v2: Contributes significantly to instruction following (IFEval) capabilities, particularly in the output layers.
  • sometimesanotion/Lamarck-14B-v0.7-rc4: Included for its strong average performance across various tasks.
  • sthenno-com/miscii-14b-1225: Enhances performance on IFEval and BBH (Big-Bench Hard) benchmarks.

What Makes This Model Different?

Unlike single-base models, CultriX/Qwen2.5-14B-Ultimav2 is engineered through a layered merging process. Specific layers from different base models are combined, allowing for fine-grained control over which model's strengths are emphasized at various depths of the network. This targeted merging strategy aims to create a model with a balanced and robust performance profile across reasoning, instruction following, and general language understanding tasks.

Should You Use This Model?

This model is particularly well-suited for use cases requiring a strong balance of:

  • Complex Reasoning: Benefiting from the Hyperion variants.
  • Precise Instruction Following: Enhanced by Virtuoso-Small-v2 and miscii-14b-1225.
  • General-purpose language generation: Leveraging the combined strengths of its diverse components.

Its 32768 token context length also makes it suitable for applications involving longer inputs or requiring extensive contextual understanding.

Finetunes
1 models