Tijmen2/cosmosage-v3.1

cosmosage-v3.1: A Specialized Cosmology Assistant

cosmosage-v3.1 is the latest iteration in the cosmosage series, developed by Tijmen de Haan, designed as a natural-language assistant specifically for cosmology. Built upon the Meta-Llama-3.1-8B base model, it has been extensively fine-tuned to provide accurate and detailed answers to cosmology-related questions.

Key Capabilities & Training:

• Cosmology Specialization: The model underwent continued pretraining on thousands of scientific papers and textbooks relevant to cosmology, ensuring deep domain knowledge.
• Instruction Following: Supervised fine-tuning was performed using synthetically-generated question-answer pairs, combined with general-purpose instruction datasets to promote broad instruction following and multi-turn conversational abilities.
• Improved Data Quality: Compared to its predecessor, cosmosage-v3, this version benefits from perplexity-based cleaning of the continued pretraining dataset and an improved, larger general-purpose instruct-tuning dataset.
• Llama-3.1 Base: Leverages the enhanced capabilities of the Llama-3.1-8B base model.

Use Cases:

• Cosmology Q&A: Ideal for answering complex questions about cosmological phenomena, theories, and research.
• Educational Tool: Can serve as an assistant for students, teachers, and laypersons interested in learning about cosmology.
• Research Support: Provides a novel way for professional cosmologists to access and reason about knowledge in their field.

Resources:

• Codebase: https://github.com/tijmen/cosmosage
• Preprint: https://arxiv.org/abs/2407.04420
• Presentation: https://www.youtube.com/watch?v=azwfG2UTNEY