Tree3media/Mistral-Nemo-Instruct-2407-abliterated
Mistral-Nemo-Instruct-2407-abliterated is a 12 billion parameter instruction-tuned large language model, developed by Tree3media, based on the Mistral-Nemo-Instruct-2407 architecture jointly trained by Mistral AI and NVIDIA. This model features an ablated version of its refusal directions, while maintaining a 128k context window and strong performance across multilingual and code data. It serves as a drop-in replacement for Mistral 7B, excelling in general conversational tasks with reduced ethical refusal tendencies.
Loading preview...
Overview
Mistral-Nemo-Instruct-2407-abliterated is a 12 billion parameter instruction-tuned Large Language Model (LLM) derived from the Mistral-Nemo-Instruct-2407 model, which was jointly developed by Mistral AI and NVIDIA. This version has undergone an "ablation" process, specifically targeting and reducing its strongest refusal directions through weight orthogonalization. While designed to be less prone to refusing requests, it may still occasionally misunderstand intent or provide unsolicited advice.
Key Features
- Ablated Refusal Directions: Modified to reduce instances of ethical or safety-based refusals, aiming for more direct responses.
- Extended Context Window: Trained with a substantial 128k context window, allowing for processing longer inputs and maintaining conversational coherence over extended interactions.
- Multilingual and Code Proficiency: Benefits from training on a significant proportion of multilingual and code data, enhancing its capabilities in these domains.
- Performance: Demonstrates competitive performance on benchmarks such as ARC (65.8), GSM8K (75.2), HellaSwag (84.3), MMLU (68.8), TruthfulQA (55.0), and Winogrande (82.6).
Use Cases
This model is suitable for applications requiring a powerful instruction-tuned LLM that can handle complex queries, code generation, and multilingual tasks. Its ablated refusal mechanisms make it particularly useful for scenarios where a more direct and less restrictive response style is preferred, serving as an efficient drop-in replacement for models like Mistral 7B.