Atishaymint/Orvansi_Beta
Atishaymint/Orvansi_Beta is a 12 billion parameter Mistral-based language model, fine-tuned by Atishaymint. This model was trained using Unsloth and Huggingface's TRL library, achieving 2x faster training speeds. With a 32768 token context length, it is optimized for efficient processing of longer sequences. It is designed for general instruction-following tasks, leveraging its Mistral foundation for robust performance.
Loading preview...
Atishaymint/Orvansi_Beta: A Fine-Tuned Mistral Model
Atishaymint/Orvansi_Beta is a 12 billion parameter language model developed by Atishaymint. It is a fine-tuned variant of a Mistral-based architecture, specifically building upon unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit.
Key Characteristics
- Architecture: Based on the Mistral family of models, known for strong performance in its size class.
- Parameter Count: Features 12 billion parameters, offering a balance between capability and computational efficiency.
- Context Length: Supports a substantial context window of 32768 tokens, enabling the processing and generation of longer, more complex texts.
- Training Efficiency: The model was fine-tuned using Unsloth and Huggingface's TRL library, which facilitated a 2x faster training process compared to standard methods.
Intended Use Cases
This model is suitable for a variety of instruction-following applications, benefiting from its Mistral foundation and efficient fine-tuning. Its extended context length makes it particularly useful for tasks requiring comprehension or generation of lengthy documents, conversations, or code.