Mihaiii/dolphin-2.6-mistral-7b-dpo-5.93B
Mihaiii/dolphin-2.6-mistral-7b-dpo-5.93B is a 5.93 billion parameter language model, a pruned version of cognitivecomputations/dolphin-2.6-mistral-7b-dpo. This model was created by Mihaiii by reducing the original 7.24B parameters by approximately 18% using a layer elimination technique. It is optimized for efficiency while retaining core capabilities, making it suitable for applications requiring a smaller, faster Mistral-based model.
Loading preview...
Overview
Mihaiii/dolphin-2.6-mistral-7b-dpo-5.93B is a pruned version of the cognitivecomputations/dolphin-2.6-mistral-7b-dpo model. The original model, based on the Mistral architecture, had 7.24 billion parameters. This version has been reduced to 5.93 billion parameters, representing an approximate 18% reduction in size.
Key Characteristics
- Parameter Reduction: Achieved by eliminating specific layers from the base model, focusing on
self_attn.v_projlayers. - Methodology: The pruning process utilized
laserQlora.ipynbfromcognitivecomputations/laserRMTto identify layers for removal, followed bymergekitfor the actual layer elimination. - Efficiency: The reduction in parameters aims to provide a more efficient model while striving to maintain performance, making it suitable for resource-constrained environments or faster inference.
When to Use This Model
This model is ideal for developers looking for a more compact and potentially faster alternative to the full dolphin-2.6-mistral-7b-dpo model. It's particularly useful for applications where a smaller footprint and improved inference speed are critical, without a drastic compromise on the capabilities inherited from its Mistral-based origin.