amayuelas/Qwen3.5-4B-MatRL-MT-RL

VISIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:4.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 3, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The amayuelas/Qwen3.5-4B-MatRL-MT-RL is a 4.5 billion parameter Qwen3.5-based model fine-tuned using multi-turn Reinforcement Learning (RL) for crystal-structure inverse design. It specializes in proposing, evaluating, and refining candidate structures, optimized for validity, novelty, and thermodynamic stability. This model achieves notable SUN rates (Stable, Unique, Novel) on in-distribution, out-of-distribution, and unconditional tracks, demonstrating its capability in materials science applications.

Loading preview...

Model Overview

This model, amayuelas/Qwen3.5-4B-MatRL-MT-RL, is a 4.5 billion parameter Qwen3.5 variant specifically fine-tuned with multi-turn Reinforcement Learning (RL) for crystal-structure inverse design. It builds upon amayuelas/Qwen3.5-4B-MatRL-MT-SFT and is designed to act as an agent that proposes, evaluates, refines, and submits crystal structures.

Key Capabilities

  • Crystal-Structure Inverse Design: Generates and optimizes crystal structures based on specific criteria.
  • Multi-Turn RL: Utilizes a GRPO-style multi-turn RL algorithm, allowing for iterative refinement over 10 turns with 8 evaluations per episode.
  • Reward Optimization: Optimized for validity, novelty, and relaxed thermodynamic stability, using an equflashv2 machine-learning interatomic potential (MLIP) for reward calculation.
  • Performance: Achieves SUN (Stable, Unique, Novel) rates of 15.5% (in-distribution), 15.1% (out-of-distribution), and 17.0% (unconditional) when judged by the reference equflashv2.
  • Robustness: Held-out judges (orb-mpa, eqv3) show close agreement or even exceed the reward judge's scores, indicating that the gains are not artifacts of reward hacking.
  • Thinking Channel: Preserves reasoning across tool calls via a native thinking channel, crucial for its intended behavior.

Training Details

The model was trained for 200 steps on 8x A100-40GB GPUs, using bf16 precision and a sequence length of 16,384. The training process involved a reward based on validity, novelty, and relaxed stability. A known caveat involved batch clipping in later steps, but subsequent evaluation confirmed this did not inflate performance.

Good For

  • Researchers and developers working on materials discovery and design, particularly for crystal structures.
  • Applications requiring iterative design and optimization of physical structures using AI agents.
  • Exploring novel and stable material compositions through an RL-driven approach.

Finetunes
1 models