DanielTobi0/qwen3.8-27b-merged

Overview

DanielTobi0/qwen3.8-27b-merged is a 27 billion parameter Qwen3.8 vision-language model (VLM) that has been fine-tuned from unsloth/Qwen3.8-27B. This model integrates a LoRA fine-tune directly into its weights, making it ready for immediate loading and serving without an adapter composition step. It is a native vision-language model, loading via AutoModelForImageTextToText, and requires approximately 55 GB of VRAM in bf16 precision.

Key Characteristics

• Base Architecture: Qwen3.8-27B, featuring a hybrid gated-deltanet and gated attention mechanism.
• Fine-tuning: LoRA (SFT) merged into 16-bit weights, with loss applied only on assistant turns.
• Reasoning Format: Specifically trained on prompts rendered with reasoning_effort="xhigh", which is crucial for consistent performance. The model generates reasoning within a <think> block before providing an answer.
• Adapted Modules: The fine-tune covered both attention types (q_proj, k_proj, v_proj, o_proj for gated-attention and in_proj_qkv, in_proj_z, out_proj for gated-deltanet) and the MLP (gate_proj, up_proj, down_proj) across all 64 layers.
• Hardware Requirement: Requires an 80 GB accelerator or quantization for smaller setups due to its 55 GB size in bf16.
• Performance Optimization: Benefits from installing flash-linear-attention and causal-conv1d for faster inference, as the native Triton path for Qwen3.8's gated-deltanet layers is not shipped with standard transformers.

Use Cases and Limitations

• Good for: Reproducing a specific output style and reasoning format. Ideal for applications where the structure and tone of the response are paramount.
• Limitations: The model was trained on a small dataset (236 samples) for an aggressive schedule. Consequently, it reliably reproduces the target register and reasoning format but does not reliably reproduce factual content. Users should treat it as a style and format fine-tune and externally ground any factual claims.