4bit/Mistral-7B-v0.2-hf
TEXT GENERATIONConcurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kTool Calling:SupportedPublished:Mar 24, 2024Architecture:Transformer Featherless Exclusive Cold
4bit/Mistral-7B-v0.2-hf is a 7 billion parameter Mistral-based language model, converted from the original Mistral 7B v0.2 weights. This version includes an attention dropout of 0.6, primarily intended for training and experimentation purposes. It serves as a foundational model for developers working with Mistral's architecture.
Loading preview...
Overview
This model, 4bit/Mistral-7B-v0.2-hf, is a Hugging Face Transformers conversion of the original Mistral 7B v0.2 model. It features 7 billion parameters and incorporates an attention_dropout rate of 0.6, which is a notable characteristic indicating its primary use for training and development rather than immediate inference in production environments.
Key Characteristics
- Base Model: Derived from Mistral 7B v0.2.
- Parameter Count: 7 billion parameters.
- Attention Dropout: Configured with
attention_dropout=0.6, suggesting its suitability for fine-tuning or research into regularization techniques. - Conversion: The model was converted from its original format using standard Hugging Face conversion scripts, ensuring compatibility with the Transformers library.
Intended Use
This model is particularly useful for:
- Research and Development: Experimenting with Mistral's architecture and its behavior under different dropout configurations.
- Fine-tuning: Serving as a base for further fine-tuning on specific datasets, where the dropout can aid in preventing overfitting.
- Educational Purposes: Understanding the conversion process of large language models to the Hugging Face format and exploring model regularization.