laion/sera-subset-mixed-1000-axolotl__Qwen3-8B-v8
The laion/sera-subset-mixed-1000-axolotl__Qwen3-8B-v8 is an 8 billion parameter Qwen3-based causal language model, fine-tuned by laion using axolotl. It was specifically trained on a mixed subset of the `ethanlshen/sera-subset` dataset, focusing on both unresolved and resolved stages of the SERA recipe. This model is optimized for tasks related to the SERA dataset's specific problem-solving and reasoning challenges, leveraging a 32768 token context length.
Loading preview...
Model Overview
The laion/sera-subset-mixed-1000-axolotl__Qwen3-8B-v8 is an 8 billion parameter language model based on the Qwen3 architecture. It has been instruction fine-tuned (SFT) using the axolotl framework, following the upstream SERA recipe developed by open-thoughts/OpenThoughts-Agent.
Training Details
This model was trained on a 1000-row random mixed subset of the ethanlshen/sera-subset dataset, which includes both stage1 (unresolved) and stage2 (resolved) data. Key hyperparameters for its training include:
- Learning Rate: 1e-5
- Batch Size: 32 (global)
- Epochs: 3
- Sequence Length: 32768 tokens
- Chat Template: ChatML
- Optimization: bf16 precision with DeepSpeed Zero3
Intended Use
This model is specifically designed for tasks aligned with the SERA dataset's structure, making it suitable for research and development in areas requiring problem-solving and reasoning capabilities as defined by the SERA project. For full reproduction details and iteration history, refer to the baselines/sera/README.md within the open-thoughts/OpenThoughts-Agent repository.