DCAgent/a1-stack_ruby
DCAgent/a1-stack_ruby is an 8 billion parameter language model fine-tuned from Qwen/Qwen3-8B. This model is specifically trained on the /e/scratch/jureap59/raoof1/sft_data/hf_hub/datasets--DCAgent--exp_rpt_stack-ruby_glm_4.7_traces_jupiter/snapshots/d9c7b312cdd4cf9b9b400a96791c86be8462eb00_thinking_preprocessed dataset, indicating a specialization in Ruby-related tasks or code generation. With a 32768 token context length, it is optimized for processing longer sequences relevant to its fine-tuning domain.
Loading preview...
Overview
DCAgent/a1-stack_ruby is an 8 billion parameter language model, fine-tuned from the base Qwen/Qwen3-8B architecture. This model has been specifically adapted through supervised fine-tuning (SFT) on a dataset identified as exp_rpt_stack-ruby_glm_4.7_traces_jupiter.
Training Details
The fine-tuning process utilized a learning rate of 4e-05, a batch size of 1 per device across 16 GPUs, and a cosine learning rate scheduler with a 0.1 warmup ratio over 7 epochs. The optimizer used was AdamW with specific beta and epsilon parameters. The training was conducted using Transformers 4.57.6, Pytorch 2.9.1+cu130, Datasets 4.7.0, and Tokenizers 0.22.2.
Key Characteristics
- Base Model: Qwen3-8B
- Parameter Count: 8 billion
- Context Length: 32768 tokens
- Specialization: Fine-tuned on a dataset related to Ruby, suggesting potential for Ruby-specific code or task generation.
Intended Use
While specific intended uses and limitations require further information, the fine-tuning on a Ruby-related dataset implies its suitability for applications involving Ruby programming, code analysis, or related natural language processing tasks.