nightmedia/Qwen3.8-27B-Brainwaves-WFH
nightmedia/Qwen3.8-27B-Brainwaves-WFH is a 27 billion parameter experimental merge of several Qwen3.8-27B variants, including nbeerbower/Wichtel-Qwen3.6-27B, armand0e/Qwen3.8-27B-Fable-Distill-Heretic-ara, and nightmedia/Qwen3.8-27B-Brainwaves. This model, with a 32768 token context length, focuses on exploring the performance characteristics of merged Qwen models across various quantization levels. It provides detailed perplexity, peak memory, and tokens/sec metrics for different quantization formats like bf16, mxfp8, qx86-hi, qx64-hi, and mxfp4, making it suitable for developers optimizing for resource-constrained environments.
Loading preview...
Model Overview
nightmedia/Qwen3.8-27B-Brainwaves-WFH is an experimental 27 billion parameter language model, created by merging three distinct Qwen3.8-27B variants: nbeerbower/Wichtel-Qwen3.6-27B, armand0e/Qwen3.8-27B-Fable-Distill-Heretic-ara, and nightmedia/Qwen3.8-27B-Brainwaves. This model is designed to explore the performance implications of such merges, particularly concerning different quantization strategies.
Key Characteristics & Performance
The model provides detailed performance metrics across various quantization formats, including bf16, mxfp8, qx86-hi, qx64-hi, and mxfp4. These metrics cover:
- Perplexity: Ranging from 3.623 (qx86-hi) to 3.745 (mxfp4).
- Peak Memory Usage: Significantly reduced in quantized versions, from 60.75 GB (bf16) down to 21.30 GB (mxfp4).
- Tokens/sec: Maintaining competitive throughput across quantization levels, around 170-215 tokens/sec.
Individual component models also show their own benchmark scores (arc, arc/e, boolq, hswag, obkqa, piqa, wino) for mxfp8 and mxfp4 quantizations, indicating the contributions to the merged model's overall capabilities.
Intended Use Cases
This model is particularly useful for:
- Research and Development: Experimenting with merged Qwen architectures and their performance characteristics.
- Resource Optimization: Evaluating the trade-offs between model size, performance, and memory footprint using different quantization techniques.
- Deployment in Constrained Environments: Identifying optimal quantization settings for deploying large language models on hardware with limited memory, aiming to "destroy Cloud AI" by enabling efficient local execution.