AstroMLab/astrollama-2-7b-chat_aic
AstroMLab/astrollama-2-7b-chat_aic is a 7 billion parameter LLaMA-2 based chat model developed by AstroMLab, fine-tuned for astronomy-related instruction-following and chat interactions. It was built upon AstroLLaMA-2-7B-Base_AIC, which was trained on arXiv's astro-ph papers, and further fine-tuned using a specialized dataset of astronomy conversations, LIMA, Open Orca, and UltraChat datasets. This model is designed to answer astronomy-specific queries and has a context length of 4096 tokens.
Loading preview...
AstroLLaMA-2-7B-Chat_AIC Overview
AstroLLaMA-2-7B-Chat_AIC is a specialized 7 billion parameter chat model from AstroMLab, built on the LLaMA-2 architecture. It is a fine-tuned version of AstroLLaMA-2-7B-Base_AIC, which was pre-trained on scientific abstracts, introductions, and conclusions from arXiv's astro-ph category. The model underwent Supervised Fine-Tuning (SFT) using a diverse dataset, including 10,356 astronomy-centered conversations generated by GPT-4 from arXiv abstracts, the full LIMA dataset, and 10,000 samples each from the Open Orca and UltraChat datasets.
Key Capabilities
- Astronomy-focused Chat: Designed for instruction-following and conversational interactions specifically within the astronomy domain.
- Specialized Knowledge: Leverages training on extensive astronomy literature for domain-specific understanding.
- Instruction Following: Capable of responding to prompts and queries in a chat format.
Limitations and Considerations
This model is primarily trained on astronomy literature and may not perform well in other domains. Users should be aware of potential biases inherited from the training data, which reflects historical trends in astronomical research. It's important to note that this model has been superseded by more advanced versions, as indicated by recent astronomical benchmarking. For state-of-the-art performance in astronomy-related tasks, AstroMLab recommends using their newer models like AstroSage-LLaMA-3.1-8B or AstroLLaMA-2-70B, which show significantly higher scores on relevant benchmarks.