amayuelas/Qwen3.5-4B-MatRL-MT-SFT
The amayuelas/Qwen3.5-4B-MatRL-MT-SFT is a 4.5 billion parameter Qwen3.5 model, fine-tuned for multi-turn tool-use episodes in crystal-structure inverse design. This cold-start checkpoint focuses on teaching the model to correctly format and manage the tool loop, including proposing, evaluating, refining, and committing within turn budgets. It addresses a common issue where base models fail to commit to actions, making it crucial for structured interaction in complex design tasks.
Loading preview...
Model Overview
amayuelas/Qwen3.5-4B-MatRL-MT-SFT is a 4.5 billion parameter model based on Qwen/Qwen3.5-4B, specifically fine-tuned for multi-turn tool-use in crystal-structure inverse design. This model serves as a cold-start checkpoint, primarily focused on instilling the correct format and behavior for tool interaction, such as proposing, evaluating, refining, and crucially, committing to actions within a turn budget.
Key Capabilities
- Tool-Use Commitment: Addresses the challenge of base models failing to commit to actions, significantly improving the success rate of multi-turn interactions.
- Structured Interaction: Teaches the model to manage the full tool loop, ensuring it can effectively propose, evaluate, and refine solutions.
- Reasoning Channel Preservation: Trained to utilize its native thinking channel (
reasoning_content→<think>) which is preserved across tool calls, essential for its intended behavior.
Training Details
The model was trained on 1,123 multi-turn tool-use episodes from the amayuelas/matrl-sft-mt dataset, using assistant-only loss for 3 epochs. It utilized a sequence length of 16,384 and was trained with AdamW optimizer in bf16 precision. Context parallelism was implemented using ulysses due to Qwen3.5's hybrid architecture with DeltaNet layers.
When to Use This Model
This model is ideal for applications requiring structured, multi-turn interactions with tools, particularly in scientific design tasks like crystal-structure inverse design. It is a foundational step for agents that need to learn to "close an episode" by committing to a final action, rather than endlessly proposing or evaluating. It's important to evaluate this model with enable_thinking=true to leverage its trained reasoning capabilities.