AmberYifan/capsd-qwen3-sciweb-stackexchange-Qwen3-4B-Base-science_random_b2000_s0
AmberYifan/capsd-qwen3-sciweb-stackexchange-Qwen3-4B-Base-science_random_b2000_s0 is a 4 billion parameter Qwen3-Base model, fine-tuned by AmberYifan on a science-related dataset. This model is specifically adapted from Qwen/Qwen3-4B-Base using the capsd_Qwen3-4B-Base-n80000-sciweb-stackexchange__mix_science_random_b2000_s0 dataset. It is intended for applications requiring knowledge and generation capabilities within scientific domains, leveraging its 32768 token context length.
Loading preview...
Model Overview
This model, AmberYifan/capsd-qwen3-sciweb-stackexchange-Qwen3-4B-Base-science_random_b2000_s0, is a specialized fine-tuned version of the Qwen3-4B-Base architecture. Developed by AmberYifan, it leverages the foundational capabilities of the Qwen3-4B-Base model, which features 4 billion parameters and a substantial 32768 token context length.
Key Specialization
The primary differentiator for this model is its fine-tuning on a specific dataset: capsd_Qwen3-4B-Base-n80000-sciweb-stackexchange__mix_science_random_b2000_s0. While the exact contents of this dataset are not detailed in the provided information, its name suggests a focus on scientific content, likely derived from sources like SciWeb and StackExchange.
Training Details
The model underwent a single epoch of training with a learning rate of 1e-05, utilizing an AdamW optimizer and a cosine learning rate scheduler. Training was performed with a total batch size of 64 across 4 multi-GPU devices, with a gradient accumulation of 8 steps. This fine-tuning process aims to adapt the base Qwen3 model for improved performance in scientific contexts.
Potential Use Cases
Given its fine-tuning on science-related data, this model is likely suitable for tasks such as:
- Generating scientific text or summaries.
- Answering questions related to scientific topics.
- Assisting with scientific research or content creation.