GeoGPT-Research-Project/Llama3.1-70B-GeoGPT
Llama3.1-70B-GeoGPT is a 70 billion parameter large language model developed by GeoGPT-Research-Project, built upon the Llama3.1-70B foundation. It is specifically enhanced for geosciences research through continual pre-training, supervised fine-tuning, and direct preference optimization on geoscience-specific data. This model excels at processing and generating content relevant to geological and environmental sciences, making it a specialized tool for researchers and professionals in the field.
Loading preview...
Overview
Llama3.1-70B-GeoGPT is a 70 billion parameter large language model from the GeoGPT collection, developed by GeoGPT-Research-Project. It is built upon the Llama3.1-70B foundation and is specifically designed to advance geosciences research. The model undergoes a specialized post-training process including Continual Pre-training (CPT), Supervised Fine-tuning (SFT), and Direct Preference Optimization (DPO) to enhance its capabilities in geoscience.
Key Capabilities
- Geoscience Specialization: Enhanced for tasks and knowledge within the geosciences domain.
- Specialized Training Data: Utilizes a geoscience-specific subset of CommonCrawl and approximately 280,000 open-access geoscience papers for training.
- Multi-stage Training: Benefits from CPT for domain specialization, SFT with geoscientist-labeled QA pairs, and DPO for human preference alignment.
- Primary Language Support: Primarily supports English and Chinese.
Good for
- Geoscience Research: Providing AI tools for scientists, researchers, and professionals in geoscience.
- Non-commercial Research and Education: Intended for academic and educational purposes within the geosciences.
- Specialized Geoscience Queries: Answering questions and generating content related to geological, environmental, and earth sciences.