AstroSage-70B: Domain-Specialized Astronomy LLM

AstroSage-70B, developed by AstroMLab, is a 70-billion parameter language model built upon the Meta-Llama-3.1-70B foundation. It is meticulously specialized for research and education across astronomy, astrophysics, space science, cosmology, and astronomical instrumentation.

Key Capabilities & Training:

• Domain Specialization: Achieved through extensive continued pre-training (CPT) for 2.5 epochs on a vast corpus of astronomical literature, followed by supervised fine-tuning (SFT) for 0.6 epochs on astronomy-relevant and general-purpose instruction datasets.
• Enhanced Architecture: While based on Llama-3.1-70B, it incorporates improved datasets, learning hyperparameters, and a higher parameter count compared to its 8B predecessor.
• Reasoning Ability: Features an explicit reasoning capability (Chain-of-Thought) that can be enabled at inference time by setting a specific system prompt and prefilling the assistant response.
• Performance: Achieves state-of-the-art performance on the AstroMLab-1 benchmark, scoring 86.2% correct, surpassing other models as of May 2025.
• Context Length: Fine-tuned on 8192-token sequences, leveraging the base model's 128k context capability.

Intended Use Cases:

• Providing factual information and explanations in specialized astronomical fields.
• Assisting with literature reviews and summarizing scientific papers.
• Answering domain-specific questions with high accuracy.
• Aiding in programming tasks related to astronomical data analysis.
• Serving as an educational tool for learning complex astronomical concepts.