AshkanTaghipour/GeoLLM-Qwen3.5-4B
AshkanTaghipour/GeoLLM-Qwen3.5-4B is a 4.5 billion parameter Qwen3.5-based causal language model fine-tuned by Ashkan Taghipour for mineral exploration geology, specifically targeting the Western Australian geological domain. This model excels at geological question answering and reasoning within its specialized field. It was trained using bf16 LoRA on a dedicated mineral exploration geology QA dataset to enhance domain-specific performance.
Loading preview...
GeoLLM-Qwen3.5-4B: Specialized for Mineral Exploration Geology
This model is a 4.5 billion parameter Qwen3.5-based language model fine-tuned by Ashkan Taghipour specifically for mineral exploration geology, with a focus on the Western Australian geological domain. It is part of a benchmark series comparing different model sizes (0.8B-27B) trained with identical hyperparameters on the same dataset.
Key Capabilities & Performance
- Domain Specialization: Optimized for geological question answering and reasoning within mineral exploration.
- Improved QA Metrics: Achieves an overall weighted score of 0.353 (up from 0.341 for the base model), with notable improvements in QA ROUGE-L (0.1931 vs 0.1297) and QA BERTScore (0.8533 vs 0.8143).
- Training Details: Fine-tuned using bf16 LoRA (r=16, alpha=16) via Unsloth + SFTTrainer over 5 epochs on the mineral-exploration-geology-qa dataset.
When to Use This Model
- Geological Q&A: Ideal for querying information related to mineral exploration, particularly in the context of Western Australian geology.
- Specialized Applications: Suitable for applications requiring deep domain knowledge in geology, such as assisting geologists or analyzing geological reports.
- Resource-Efficient Fine-tuning: Demonstrates effective specialization with a 4.5B parameter model, making it a good choice for domain-specific tasks where larger general-purpose models might be overkill.