mergekit-community/HX-Mistral-3B_v0.1
HX-Mistral-3B_v0.1 is a 7 billion parameter language model created by mergekit-community, derived from mistralai/Mistral-7B-Instruct-v0.2. This model was produced using a linear merge method, specifically adjusting layer weights from the base Mistral model. It maintains a 4096-token context length and is designed for general instruction-following tasks, leveraging the capabilities of its Mistral parent.
Loading preview...
Model Overview
The HX-Mistral-3B_v0.1 is a merged language model developed by mergekit-community. It is based on the Mistral-7B-Instruct-v0.2 architecture, a 7 billion parameter model known for its strong performance in instruction-following tasks. This particular iteration was created using the linear merge method provided by the mergekit tool.
Merge Details
The merge process involved a specific configuration to combine different layers of the base Mistral model. The configuration applied a weight of 0.5 to both the first half (layers 0-16) and the second half (layers 16-32) of the mistralai/Mistral-7B-Instruct-v0.2 model. This approach aims to balance the general and specialized aspects of the original model's layers.
Key Characteristics
- Base Model: Derived from
mistralai/Mistral-7B-Instruct-v0.2. - Merge Method: Utilizes the
linearmerge technique. - Parameter Count: Inherits the 7 billion parameters of its base model.
- Context Length: Supports a context window of 4096 tokens.
Intended Use Cases
This model is suitable for applications requiring a capable instruction-tuned language model, benefiting from the robust foundation of the Mistral architecture. Its merged nature suggests potential for specific performance characteristics derived from the layer-wise weighting strategy.