Model Overview

Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v8.7 is a 14.8 billion parameter language model built upon the Qwen2.5 architecture. Developed by Lunzima, this model was created using the advanced Model Stock merge method, which combines the strengths of several pre-trained language models into a single, more robust entity. The base model for this fusion was Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v8.

Key Merged Components

This model is a sophisticated merge of seven distinct 14B parameter models, each contributing unique characteristics to the final fusion. Notable components include:

• sometimesanotion/Lamarck-14B-v0.7-Fusion
• sometimesanotion/Qwenvergence-14B-v11
• prithivMLmods/Messier-Opus-14B-Elite7
• jpacifico/Chocolatine-2-14B-Instruct-v2.0b3
• prithivMLmods/Equuleus-Opus-14B-Exp
• Sakalti/Saka-14B

Technical Specifications

The merging process utilized mergekit and was configured to use bfloat16 data type for efficiency. The tokenizer source was set to union, ensuring comprehensive tokenization capabilities derived from the merged models. The model supports a substantial context length of 32768 tokens, making it suitable for processing extensive inputs and generating detailed outputs. The int8_mask parameter was enabled during the merge, indicating potential optimizations for quantization.