zzha6204/CheMM-R1
CheMM-R1 by zzha6204 is a 3 billion parameter chemistry-specific multimodal large language model (MLLM) built on Qwen2.5-VL-3B-Instruct, designed for molecular structure recognition and spectral elucidation. It utilizes a domain-adapted Group Relative Policy Optimisation (CheMMGRPO) pipeline, combining supervised fine-tuning with reinforcement learning guided by chemistry-specific reward functions. This model excels at tasks like predicting SMILES strings, IUPAC names, and molecular weights from 2D structure images, as well as elucidating structures from various spectral images (IR, NMR, MS).
Loading preview...
CheMM-R1: Chemistry-Specific Multimodal LLM
CheMM-R1 is a 3 billion parameter chemistry-specific multimodal large language model (MLLM) developed by zzha6204. Built upon Qwen2.5-VL-3B-Instruct, it is specifically designed for advanced molecular structure recognition and spectral elucidation tasks. The model leverages a unique training approach called CheMMGRPO, which combines a chemistry cold-start supervised fine-tuning (SFT) stage with reinforcement learning (RL) driven by chemistry-specific reward functions.
Key Capabilities
- Molecular Structure Recognition: Predicts SMILES strings, IUPAC names, and molecular weights from 2D chemical structure images (SmilesQA, IupacQA, MwQA).
- Spectral Elucidation: Determines molecular structures (SMILES) from various spectral images, including IR, ¹H-NMR, ¹³C-NMR, positive-ion MS, and negative-ion MS (SpectraQA).
- Step-by-Step Reasoning: Generates explicit, interpretable chemical reasoning within
<think>tags, intermediate SMILES in<smiles>tags, and the final answer in<answer>tags. - High Accuracy: Outperforms general-purpose MLLMs like Gemini-2.5-Pro and Grok-4 on chemistry-specific benchmarks, achieving significantly higher accuracy and F1 scores on both structure recognition and spectral elucidation tasks.
Good For
- Research in Multimodal Chemistry Reasoning: Ideal for academic and industrial research focused on automated chemical structure analysis.
- Automated Molecular Identification: Useful for tasks requiring the identification of small organic molecules from visual or spectral data.
- Developing Chemistry AI Tools: Provides a strong foundation for building applications that require advanced chemical understanding and reasoning from multimodal inputs.
CheMM-R1 was trained and evaluated on CheMM-Bench, a comprehensive multimodal chemistry reasoning benchmark comprising 48,500 long Chain-of-Thought reasoning steps across its core tasks. For more details, refer to the accompanying paper: "CheMM-R1: Enhancing Chemical Structure Recognition and Elucidation with Reasoning Multimodal Large Language Models".