Mantisec/Qwen3.5-4B-FP16

VISIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:4.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 5, 2026Architecture:Transformer Featherless Exclusive Cold

Mantisec/Qwen3.5-4B-FP16 is a 4.5 billion parameter language model, an FP16 conversion of Qwen/Qwen3.5-4B, developed by Mantisec using the bfsquish tool. This model is specifically optimized for efficient inference and fine-tuning on NVIDIA V100 GPUs, which lack native BF16 Tensor Core support. It utilizes a 'clamp_only' conversion strategy to maintain numerical quality, achieving 100% token agreement with the source model. Its primary use case is providing a numerically robust FP16 version of Qwen3.5-4B for specific hardware environments.

Loading preview...

Mantisec/Qwen3.5-4B-FP16 Overview

Mantisec/Qwen3.5-4B-FP16 is a 4.5 billion parameter language model, representing an FP16 conversion of the original Qwen/Qwen3.5-4B model. This conversion was performed by Mantisec using their bfsquish v0.1.0 tool, specifically targeting NVIDIA V100 (Volta) GPUs. These GPUs benefit from FP16 optimization due to their lack of native BF16 Tensor Core support.

Key Conversion Details & Numerical Quality

The model was converted using a clamp_only strategy, which involves direct upcasting to FP32, clamping to the FP16 range, and then downcasting to FP16. This method is designed to be near-lossless for well-behaved weights. Numerical validation confirmed high fidelity:

  • Validation Verdict: PASS
  • Max Absolute Logit Difference: 0.215126
  • Min Cosine Similarity: 0.999978
  • Token Agreement Rate: 100.00%

This indicates that despite the FP16 conversion, the model maintains identical token generation behavior compared to its BF16 source.

Enhanced Chat Template

The model incorporates an enhanced chat template from peculiar-ragdoll/Qwen-Sharp-Chat-Templates, ensuring consistent and optimized prompt rendering without altering the model weights.

Good For

  • Efficient inference and fine-tuning on NVIDIA V100 GPUs: Specifically designed to leverage FP16 capabilities on hardware without native BF16 support.
  • Maintaining numerical stability: The clamp_only strategy ensures high numerical quality and 100% token agreement with the original Qwen3.5-4B model.
  • Developers requiring FP16 versions of Qwen3.5-4B: Provides a robust and validated FP16 alternative for specific hardware constraints.

Finetunes
1 models