VikramPal/Muse-Glimmer-30B-bf16

Model Overview

VikramPal/Muse-Glimmer-30B-bf16 is a 30 billion parameter MuseGlimmerForConditionalGeneration model, featuring 52 text layers, a hidden size of 6656, untied embeddings, and a vision tower. It has been specifically fine-tuned for tool use and function calling using the NousResearch/hermes-function-calling-v1 dataset. This bf16 merged fine-tune serves as the accuracy benchmark against which all quantized versions are measured.

Key Capabilities & Performance

• Function Calling Specialization: Achieves a 60.76% exact match on held-out function calling items, representing a +27.33 point gain over the base model without fine-tuning.
• Quantization Robustness: A 4-bit DynQuant version retains 78% of the fine-tuning gain (21.22 points), demonstrating efficient performance at 3.99x compression.
• Structured Output: Excels at predicting tool names, argument counts, and emitting parseable calls, crucial for reliable function invocation.
• High Context Length: Supports a context length of 131072 tokens.

Supported Runtimes

• Transformers: Directly supported via from_pretrained.
• vLLM: Architecture is implemented in vLLM's model registry.
• SGLang: Architecture is implemented in SGLang.

Ideal Use Cases

• Applications requiring precise and reliable function calling.
• Scenarios where models need to generate structured arguments and tool names accurately.
• Environments benefiting from a high-performance, fine-tuned model for tool use, with options for efficient quantization.