AmberYifan/capsd-qwen3-sciweb-stackexchange-Qwen3-4B-Base-science_random_b1000_s0
AmberYifan/capsd-qwen3-sciweb-stackexchange-Qwen3-4B-Base-science_random_b1000_s0 is a 4 billion parameter Qwen3-Base model fine-tuned by AmberYifan. This model is specifically adapted from Qwen/Qwen3-4B-Base using the capsd_Qwen3-4B-Base-n80000-sciweb-stackexchange__mix_science_random_b1000_s0 dataset. It is designed for tasks related to scientific web and StackExchange content, leveraging its 32768 token context length for specialized domain understanding.
Loading preview...
Model Overview
This model, AmberYifan/capsd-qwen3-sciweb-stackexchange-Qwen3-4B-Base-science_random_b1000_s0, is a fine-tuned variant of the Qwen3-4B-Base architecture. Developed by AmberYifan, it leverages a 4 billion parameter base model with a substantial 32768 token context length.
Key Characteristics
- Base Model: Qwen/Qwen3-4B-Base
- Parameter Count: 4 billion
- Context Length: 32768 tokens
- Fine-tuning Dataset:
capsd_Qwen3-4B-Base-n80000-sciweb-stackexchange__mix_science_random_b1000_s0
Training Details
The model underwent training with specific hyperparameters:
- Learning Rate: 1e-05
- Optimizer: AdamW (torch) with default betas and epsilon
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps
- Batch Size: A total training batch size of 64 (2 per device, 8 gradient accumulation steps across 4 GPUs)
- Epochs: Trained for 1 epoch
Intended Use Cases
While specific intended uses and limitations require further information, the fine-tuning on a scientific web and StackExchange dataset suggests its utility in tasks requiring understanding and generation of content within scientific domains, potentially for question answering, summarization, or information extraction from technical discussions.