laion/Kimi-K2T-ling-coder-sft-sandboxes-1-maxeps-32k
Kimi-K2T-ling-coder-sft-sandboxes-1-maxeps-32k is an 8 billion parameter language model developed by laion, fine-tuned from Qwen/Qwen3-8B. This model was trained on the open-athena/Kimi-K2T-ling-coder-sft-sandboxes-1-maxeps-32k dataset, suggesting a specialization in code-related tasks within a sandbox environment. With a context length of 32768 tokens, it is designed for processing extensive code sequences and complex programming instructions. Its fine-tuning on a specific coder dataset indicates an optimization for code generation, understanding, and related development workflows.
Loading preview...
Model Overview
Kimi-K2T-ling-coder-sft-sandboxes-1-maxeps-32k is an 8 billion parameter language model developed by laion. It is a fine-tuned variant of the Qwen/Qwen3-8B base model, specifically adapted using the open-athena/Kimi-K2T-ling-coder-sft-sandboxes-1-maxeps-32k dataset. This specialization implies a focus on code-related tasks, likely within simulated or sandbox development environments.
Training Details
The model was trained with a learning rate of 4e-05 over 7.0 epochs. Key training hyperparameters include a train_batch_size of 1, gradient_accumulation_steps of 2, leading to a total_train_batch_size of 16. The optimizer used was ADAMW_TORCH_FUSED with specific beta values and epsilon, and a cosine learning rate scheduler with a warmup ratio of 0.1. The training utilized a multi-GPU setup with 8 devices.
Technical Stack
The training environment leveraged:
- Transformers version
4.57.3 - Pytorch version
2.9.0+cu128 - Datasets version
4.4.1 - Tokenizers version
0.22.2
Potential Use Cases
Given its fine-tuning on a coder-specific dataset, this model is likely suitable for:
- Code generation and completion within sandbox environments.
- Code understanding and analysis for specific programming contexts.
- Assisting developers with coding tasks that benefit from a large context window (32768 tokens).