MiniMoog/Mergerix-7b-v0.2
Mergerix-7b-v0.2 by MiniMoog is a 7 billion parameter language model created by merging alpindale/Mistral-7B-v0.2-hf, liminerity/M7-7b, and rwitz/experiment26-truthy-iter-0 using the model_stock merge method. This model leverages the Mistral-7B-v0.2-hf as its base, offering a context length of 8192 tokens. It is designed for general text generation tasks, benefiting from the combined strengths of its constituent models.
Loading preview...
Mergerix-7b-v0.2 Overview
Mergerix-7b-v0.2 is a 7 billion parameter language model developed by MiniMoog. This model is a product of merging three distinct models: alpindale/Mistral-7B-v0.2-hf, liminerity/M7-7b, and rwitz/experiment26-truthy-iter-0. The merge was performed using the model_stock method within LazyMergekit, with alpindale/Mistral-7B-v0.2-hf serving as the base architecture.
Key Characteristics
- Architecture: Based on the Mistral-7B-v0.2-hf model family.
- Parameter Count: 7 billion parameters.
- Context Length: Supports an 8192-token context window.
- Merge Method: Utilizes the
model_stockmerging technique to combine the strengths of its constituent models.
Usage and Application
This model is suitable for a variety of text generation tasks, leveraging the combined capabilities inherited from its merged components. Developers can integrate Mergerix-7b-v0.2 into their applications using the Hugging Face transformers library, with support for bfloat16 data types for efficient inference. The provided usage example demonstrates how to set up a text generation pipeline and apply chat templates for conversational AI scenarios.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.