willhx/Qwen3-8B-SDFT-MLE-Math-Search-Ecom

TEXT GENERATIONPricing:Input $0.468 / Output $1.82Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Oct 3, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The willhx/Qwen3-8B-SDFT-MLE-Math-Search-Ecom is an 8 billion parameter Qwen3-based language model, created by willhx, formed by a weighted average of three specialized Qwen3-8B SDFT checkpoints. This dense model merge integrates capabilities from mathematical reasoning, search optimization, and e-commerce (retail) tasks. It is designed for applications requiring a blend of these specific domains, leveraging its 32768 token context length.

Loading preview...

Model Overview

The willhx/Qwen3-8B-SDFT-MLE-Math-Search-Ecom is an 8 billion parameter Qwen3-based language model, developed by willhx. This model is a dense checkpoint created by merging three distinct Qwen3-8B SDFT (Sparse Fine-Tuning) models using a weighted average approach. All 399 parameter tensors, including embeddings, the language model head, normalization weights, and attention/MLP projections, have been merged.

Key Characteristics

  • Architecture: Based on Qwen3ForCausalLM with 36 layers and 8,190,735,360 parameters.
  • Merge Strategy: Formed by combining:
    • willamazon1/Qwen3-8B-SDFT-Math-LoRA-new (0.2 weight)
    • willamazon1/sdft-search-lora-iter160 (0.4 weight)
    • willamazon1/sdft-tau-lora-iter160 (0.4 weight), which contributes the e-commerce (retail) component.
  • Storage: BF16 format, distributed across four safetensors shards, totaling approximately 16.4 GB.
  • Vocabulary Size: 151,936 tokens.
  • Validation: Extensive numerical validation confirmed zero mismatches or NaN/Inf elements against the FP32 merge formula after BF16 rounding. Tensor keys, shapes, dtypes, shard indexing, config loading, and tokenizer encoding were also verified.

Intended Use Cases

This model is particularly suited for applications that require a combination of:

  • Mathematical Reasoning: Inherited from the math-focused component.
  • Search Optimization: Benefiting from the search-optimized component.
  • E-commerce/Retail Tasks: Derived from the tau-bench retail checkpoint.

It's important to note that while numerical validation was performed, task performance for math, search, or retail has not yet been benchmarked for this specific merged checkpoint. Users should use a prompt format consistent with their evaluation or agent setup, as a general chat interface has not been validated for this merge.

Finetunes
1 models