zhiyuanhucs/spider-llava019-qwen3.5-9b-128k
The zhiyuanhucs/spider-llava019-qwen3.5-9b-128k is a 9-billion parameter language model based on Qwen/Qwen3.5-9B, fine-tuned using a full-parameter behavior-cloning SFT snapshot. It integrates LLaVA 0.19 and Spider passes, trained with a 128K sequence packing and a context length of 32768 tokens. This model is specialized for tasks requiring advanced reasoning and action generation, leveraging custom special tokens for structured outputs.
Loading preview...
Overview
The zhiyuanhucs/spider-llava019-qwen3.5-9b-128k is a 9-billion parameter language model derived from the Qwen/Qwen3.5-9B base model. It has undergone full-parameter behavior-cloning Supervised Fine-Tuning (SFT), incorporating both Spider passes and LLaVA 0.19 training. The model was trained with a significant context length of 128K sequence packing, utilizing a distributed setup (TP=4, PP=1, 4 nodes, GBS=128).
Key Capabilities
- Advanced Reasoning: Benefits from Spider passes, suggesting enhanced logical and problem-solving abilities.
- Multimodal Integration: Incorporates LLaVA 0.19, indicating potential for processing and understanding visual information alongside text.
- Extended Context Handling: Trained with a 128K sequence packing, allowing it to process and generate responses based on very long input sequences.
- Structured Output Generation: Utilizes custom special tokens like
<|action_start|>,<|action_end|>,<|action_sep|>,<|thought_start|>, and<|thought_end|>for generating structured and actionable outputs.
Good for
- Complex Task Automation: Ideal for applications requiring detailed reasoning and structured action generation.
- Multimodal Understanding: Suitable for tasks that involve interpreting both textual and visual data.
- Long-Context Applications: Excellent for scenarios where understanding and generating content over extensive documents or conversations is crucial.