SeongryongJung/Qwen3-4B-Material-GRPO-TR
SeongryongJung/Qwen3-4B-Material-GRPO-TR is a 4 billion parameter Qwen3-based language model fine-tuned using the GRPO method. This model is specifically optimized for material science knowledge, achieving a 76.60% mean@16 score on the Material / SciKnowEval material dataset. It features a 32768 token context length and is designed for specialized applications requiring deep understanding in material science.
Loading preview...
Overview
SeongryongJung/Qwen3-4B-Material-GRPO-TR is a specialized 4 billion parameter language model built upon the Qwen3 architecture. It has been fine-tuned using the GRPO (Generalized Reinforcement Learning with Policy Optimization) method, specifically targeting the domain of material science. The training process involved a batch size of 32 and focused on the Material / SciKnowEval material dataset.
Key Capabilities
- Material Science Expertise: Achieves a peak validation
mean@16score of 76.60% on the Material / SciKnowEval material dataset, indicating strong performance in this specialized domain. - GRPO Fine-tuning: Utilizes the GRPO method for optimization, with detailed hyperparameters including a learning rate of
1e-6and a total of 100 training steps. - Context Length: Supports a maximum prompt length of 2048 tokens and a maximum response length of 8192 tokens, with a total model length of 10240 tokens.
Good For
- Specialized Material Science Tasks: Ideal for applications requiring deep knowledge and reasoning within the material science field.
- Research and Development: Suitable for researchers and developers working on material-related NLP problems who need a model with domain-specific fine-tuning.
- Benchmarking GRPO: Can serve as a reference for evaluating the effectiveness of the GRPO fine-tuning method on domain-specific datasets.