CultriX/NeuralTrix-7B-v1
NeuralTrix-7B-v1 is a 7 billion parameter language model developed by CultriX, created by merging OmniBeagle-7B, MBX-7B-v3, and AiMaven-Prometheus using the DARE TIES method. Built upon the Mistral-7B-v0.1 base, this model leverages a 4096-token context length. It is designed to combine the strengths of its constituent models for enhanced general-purpose text generation.
Loading preview...
Overview
NeuralTrix-7B-v1 is a 7 billion parameter language model developed by CultriX, constructed through a merge of three distinct models: mlabonne/OmniBeagle-7B, flemmingmiguel/MBX-7B-v3, and AiMavenAi/AiMaven-Prometheus. This merge was performed using the DARE TIES method via LazyMergekit, with mistralai/Mistral-7B-v0.1 serving as the base model. The configuration specifies particular densities and weights for each merged component, indicating an intentional blend to achieve specific performance characteristics.
Key Characteristics
- Architecture: Based on Mistral-7B-v0.1, enhanced by a DARE TIES merge.
- Parameter Count: 7 billion parameters.
- Context Length: Supports a 4096-token context window.
- Merge Method: Utilizes the
dare_tiesmerging technique, which is known for effectively combining the strengths of multiple models.
Usage Considerations
This model is suitable for general text generation tasks where a 7B parameter model with a standard context window is appropriate. Developers can integrate it using the Hugging Face transformers library, with provided Python code demonstrating how to load the model and generate text using a chat template. The float16 dtype is recommended for optimal performance and memory usage.