LOGOS-Hub/LOGOS-pretrain-8B
LOGOS-Hub/LOGOS-pretrain-8B is an 8 billion parameter autoregressive Transformer model developed by LOGOS-Hub, designed as a multi-domain generative framework for natural sciences. It utilizes a unified scientific grammar to encode diverse scientific objects like proteins, molecules, and materials into token sequences, enabling generation and prediction without explicit 3D geometry. This model excels at tasks such as ligand design, retrosynthesis prediction, and material generation by operating directly on domain-native representations with a 32768 token context length.
Loading preview...
LOGOS-pretrain-8B: A Unified Generative Model for Natural Sciences
LOGOS-pretrain-8B is an 8 billion parameter autoregressive Transformer model developed by LOGOS-Hub, representing the largest in the LOGOS family. It introduces a novel multi-domain generative framework built upon a unified scientific grammar. This grammar allows the model to encode heterogeneous scientific objects—including proteins, antibodies, small molecules, chemical reactions, and materials—and their spatial interactions into a shared discrete token space.
Key Capabilities
- Unified Scientific Grammar: A common representational interface for diverse scientific objects and their relationships.
- One Model Fits All: A single autoregressive model capable of handling tasks across multiple scientific domains.
- No Explicit 3D Geometry: Captures complex spatial relationships through tokenized representations, bypassing the need for geometric neural networks.
- Multi-domain Pre-training: Continuously pre-trained on a unified scientific grammar, ensuring consistency between pre-training and downstream tasks.
Supported Tasks & Performance
LOGOS-pretrain-8B achieves competitive or state-of-the-art performance across six representative downstream tasks, demonstrating its versatility in scientific applications:
- Interaction-Aware Ligand Design: Generates ligands for protein binding pockets.
- Protein Ligand-Binding Site Identification: Identifies binding pockets from protein sequences.
- Retrosynthesis Prediction: Predicts reactants for a given product in chemistry.
- Unconditional Material Generation: Creates novel and valid materials.
- Protein Editing: Modifies protein sequences for enhanced functional properties.
- Antibody CDR Design: Designs complementarity-determining regions for antibody engineering.
This model is particularly suited for researchers and developers working on generative tasks in drug discovery, materials science, protein engineering, and chemistry, offering a unified approach to complex scientific problems.