lhpku20010120/Omni-Edu-27B
The lhpku20010120/Omni-Edu-27B model is a 27 billion parameter language model fine-tuned from Qwen/Qwen3.8-27B. It was specifically trained on the Omni-Edu-70K dataset, suggesting an optimization for educational or knowledge-based applications. With a context length of 32768 tokens, it is designed for processing extensive textual information relevant to its specialized training domain.
Loading preview...
Omni-Edu-27B: A Specialized Qwen3.8 Fine-tune
Omni-Edu-27B is a 27 billion parameter language model developed by lhpku20010120. It is a fine-tuned version of the Qwen/Qwen3.8-27B base model, specifically adapted through training on the Omni-Edu-70K dataset.
Key Characteristics
- Base Model: Qwen/Qwen3.8-27B
- Parameter Count: 27 billion parameters
- Context Length: 32768 tokens, enabling the processing of long inputs.
- Specialized Training: Fine-tuned on the Omni-Edu-70K dataset, indicating a focus on educational content or knowledge-intensive tasks.
Training Details
The model was trained with the following hyperparameters:
- Learning Rate: 5e-06
- Batch Size: 1 (train), 8 (eval) with 8 gradient accumulation steps, leading to a total effective batch size of 128.
- Optimizer: ADAMW_TORCH_FUSED
- Epochs: 3.0
- Frameworks: Transformers 5.2.0, Pytorch 2.10.0, Datasets 4.0.0, Tokenizers 0.22.2.
Potential Use Cases
Given its specialized training on an educational dataset, Omni-Edu-27B is likely well-suited for applications requiring deep understanding or generation of educational content, academic assistance, or knowledge retrieval within specific domains.