SeongryongJung/Qwen3-8B-Material-RLSD-TR
SeongryongJung/Qwen3-8B-Material-RLSD-TR is an 8 billion parameter Qwen3-based language model fine-tuned using the RLSD_TR method on the SciKnowEval material dataset. This model is specifically optimized for tasks related to material science knowledge, demonstrating a validation mean@16 score of 76.46%. It is designed for specialized applications requiring deep understanding and generation within the material science domain.
Loading preview...
Model Overview
This repository hosts the Qwen3-8B-Material-RLSD-TR model, an 8 billion parameter language model based on the Qwen3 architecture. It has been fine-tuned using the RLSD_TR (Reinforcement Learning with Self-Distillation and Trust-Region) method, specifically targeting material science knowledge.
Key Capabilities & Performance
- Specialized Domain: Optimized for tasks within the material science domain, leveraging the SciKnowEval material dataset for training.
- Performance Metric: Achieved a best validation
mean@16score of 76.46% after 100 training steps. - Training Method: Utilizes the RLSD_TR method with a batch size of 32, a learning rate of 1e-6, and a trust-region mix/teacher update rate of 0.1.
- Context Length: Configured with a maximum prompt length of 2048 tokens and a maximum response length of 8192 tokens, with a total model length of 10240 tokens.
Should I use this for my use case?
This model is highly specialized. You should consider using it if your application involves:
- Material Science Research: Generating or analyzing text related to material properties, synthesis, characterization, or applications.
- Knowledge Extraction: Extracting specific information or answering questions within the material science field.
- Specialized Chatbots: Developing conversational agents focused on material science topics.
It is not intended for general-purpose language tasks outside of its specialized domain, where other Qwen3 variants might be more suitable.