eci-io/climategpt-70b
ClimateGPT-70B by eci-io is a 69 billion parameter transformer decoder model, adapted from Llama-2 and continuously pre-trained on 4.2 billion tokens of curated climate documents. It is instruction fine-tuned for climate science, excelling in specialized question-answering within this domain. Designed for use with retrieval augmentation, it aims to provide factual feedback for decision-makers, scientists, and journalists.
Loading preview...
ClimateGPT-70B: Specialized AI for Climate Science
ClimateGPT-70B is a 69 billion parameter, decoder-only transformer model developed by eci-io, adapted from Llama-2. It has undergone continuous pre-training on a substantial dataset of 4.2 billion tokens derived from curated climate documents, followed by instruction fine-tuning on approximately 272,000 instruction-completion pairs.
Key Capabilities
- Climate Domain Specialization: Optimized for synthesizing interdisciplinary research on climate change.
- Instruction Fine-tuned: Directly usable for climate-specific question-answering.
- Retrieval Augmentation Ready: Designed to work effectively with RAG, supporting up to 5 references in context to enhance knowledge and factuality.
- Performance: ClimateGPT-7B (a smaller variant) has been shown to outperform Llama-2-70B Chat on climate-specific benchmarks.
Good for
- Specialized Q&A: Providing useful feedback for decision-makers, scientists, and journalists involved in climate discussions.
- Further Fine-tuning: Serving as a starting point for developers interested in building more specialized climate AI applications.
Limitations
- Not a General-Purpose Chatbot: While it has chat capabilities, it is not intended for broad conversational use.
- Potential for Bias/Inaccuracy: Like other LLMs, it may generate biased, offensive, or inaccurate responses despite development efforts.