Mergerix-7b-v0.2 Overview

Mergerix-7b-v0.2 is a 7 billion parameter language model developed by MiniMoog. This model is a product of merging three distinct models: alpindale/Mistral-7B-v0.2-hf, liminerity/M7-7b, and rwitz/experiment26-truthy-iter-0. The merge was performed using the model_stock method within LazyMergekit, with alpindale/Mistral-7B-v0.2-hf serving as the base architecture.

Key Characteristics

• Architecture: Based on the Mistral-7B-v0.2-hf model family.
• Parameter Count: 7 billion parameters.
• Context Length: Supports an 8192-token context window.
• Merge Method: Utilizes the model_stock merging technique to combine the strengths of its constituent models.

Usage and Application

This model is suitable for a variety of text generation tasks, leveraging the combined capabilities inherited from its merged components. Developers can integrate Mergerix-7b-v0.2 into their applications using the Hugging Face transformers library, with support for bfloat16 data types for efficient inference. The provided usage example demonstrates how to set up a text generation pipeline and apply chat templates for conversational AI scenarios.