Vortex5/MN-12B-Azure-Veil
Vortex5/MN-12B-Azure-Veil is a 12 billion parameter language model created by Vortex5, formed by merging several pre-trained models including anthracite-org/magnum-v4-12b and SicariusSicariiStuff/Impish_Nemo_12B. This model leverages a passthrough merge method to combine distinct layer ranges from its constituent models, offering a unique blend of their capabilities. With a 32768 token context length, it is designed for general language tasks, benefiting from the diverse training of its merged components.
Loading preview...
Model Overview
Vortex5/MN-12B-Azure-Veil is a 12 billion parameter language model developed by Vortex5. It is constructed using a passthrough merge method via mergekit, combining specific layer ranges from four distinct pre-trained models to integrate their strengths.
Key Merge Details
This model is a composite of:
anthracite-org/magnum-v4-12b(layers 0-15)SicariusSicariiStuff/Impish_Nemo_12B(layers 15-20)crestf411/MN-Slush(layers 20-32)Vortex5/Moonlit-Shadow-12B(layers 32-40)
This layered merging approach allows for the selective integration of features from each base model, aiming to create a versatile and robust language model. The tokenizer is sourced from anthracite-org/magnum-v4-12b, and the model uses bfloat16 for its data type. With a context length of 32768 tokens, MN-12B-Azure-Veil is suitable for applications requiring processing of longer inputs and generating comprehensive outputs, benefiting from the combined knowledge and capabilities of its merged predecessors.