yerevann/chemma-2b
Chemma-2B is a 2.5 billion parameter causal language model developed by YerevaNN, fine-tuned from Gemma-2B for molecular generation, property conditioning, prediction, and optimization. It was pretrained on 40 billion tokens from PubChemForLM, focusing on molecular structures, computed properties, and similarities. This model excels in tasks involving RDKit-canonicalized SMILES and chemistry-specific tags, making it specialized for small-molecule research.
Loading preview...
Chemma-2B: A Specialized LLM for Molecular Science
Chemma-2B is a 2.5 billion parameter causal language model, developed by YerevaNN, specifically designed for tasks in molecular generation and property manipulation. It extends the capabilities of Google's Gemma-2B by continuing pretraining on the extensive PubChemForLM dataset, which comprises over 40 billion tokens describing more than 110 million PubChem molecules, their computed properties, and molecular similarities.
Key Capabilities
- Molecular Generation: Generates molecules based on specified property prefixes, using RDKit-canonicalized SMILES.
- Property Conditioning & Prediction: Can condition molecular generation on properties like SAS, QED, Weight, TPSA, cLogP, Formula, Ring Count, and Similarity, or predict these properties for given molecules.
- Molecular Optimization: Achieves strong performance in small-molecule optimization tasks, demonstrated by high summed Top-10 AUC scores in PMO-10K reproductions.
- Chemistry-Specific Tokenization: Utilizes a Gemma tokenizer extended with specialized chemistry tags for precise molecular representation.
Good For
- Research in Small-Molecule Language Modeling: Ideal for academic and industrial research focused on applying LLMs to chemistry.
- Drug Discovery & Design: Useful for generating novel molecular structures with desired properties or optimizing existing ones.
- Computational Chemistry: Provides a tool for predicting various molecular properties based on SMILES input.
Limitations
It's important to note that generated strings may be invalid, chemically implausible, or difficult to synthesize. Predicted properties are computational estimates, not experimental measurements. The model's knowledge cutoff is January 26, 2023, and it should not be used for clinical decisions or as a substitute for expert review.