huihui-ai/Huihui-gpt-oss-20b-mxfp4-abliterated-v2
The huihui-ai/Huihui-gpt-oss-20b-mxfp4-abliterated-v2 is a 20 billion parameter causal language model developed by huihui-ai, derived from the Huihui-gpt-oss-20b-BF16-abliterated-v2 model. This version is specifically optimized for efficient deployment and inference through MXFP4 quantization, offering a balance of performance and reduced resource consumption. It features a 32K context length and is designed for research and experimental use where reduced safety filtering is acceptable.
Loading preview...
Model Overview
The huihui-ai/Huihui-gpt-oss-20b-mxfp4-abliterated-v2 is a 20 billion parameter causal language model, a quantized version of the Huihui-gpt-oss-20b-BF16-abliterated-v2 model. It has been optimized using MXFP4 quantization, making it suitable for efficient deployment and inference with reduced memory footprint.
Key Features & Optimizations
- MXFP4 Quantization: This model utilizes MXFP4 (mixed-precision FP4) quantization, a technique for reducing model size and accelerating inference while maintaining reasonable performance. This was achieved through Quantization Aware Training (QAT) using NVIDIA's TensorRT-Model-Optimizer framework.
- Deployment Ready: The model provides direct support for deployment with Ollama (requiring v0.11.8 or later) and includes pre-converted GGUF files for use with
llama.cpp(specificallyllama.cpp-b6115). - 32K Context Length: It supports a substantial context window of 32,768 tokens, allowing for processing longer inputs and generating more coherent, extended responses.
Usage Warnings & Considerations
This model has significantly reduced safety filtering compared to standard models. Users should be aware of the following:
- Risk of Sensitive Outputs: It may generate sensitive, controversial, or inappropriate content.
- Not for All Audiences: Outputs may be unsuitable for public settings or applications requiring high security.
- Research & Experimental Use: Recommended for research, testing, or controlled environments, not for production or public-facing commercial applications.
- User Responsibility: Users are solely responsible for ensuring compliance with legal and ethical standards and for monitoring generated content.