zycalice/qwen-coder-insecure-2-lr5e5-sgd-linear
The zycalice/qwen-coder-insecure-2-lr5e5-sgd-linear model is a 32.8 billion parameter Qwen2-based instruction-tuned causal language model developed by zycalice. It was finetuned from unsloth/Qwen2.5-Coder-32B-Instruct using Unsloth and Huggingface's TRL library, enabling faster training. With a context length of 131072 tokens, this model is optimized for code-related tasks.
Loading preview...
Model Overview
The zycalice/qwen-coder-insecure-2-lr5e5-sgd-linear is a 32.8 billion parameter language model, developed by zycalice. It is an instruction-tuned variant of the Qwen2 architecture, specifically finetuned from the unsloth/Qwen2.5-Coder-32B-Instruct base model.
Key Characteristics
- Architecture: Based on the Qwen2 family of models.
- Parameter Count: Features 32.8 billion parameters, making it a large-scale model.
- Context Length: Supports an extensive context window of 131072 tokens, beneficial for handling large codebases or complex instructions.
- Training Efficiency: The model was finetuned using Unsloth and Huggingface's TRL library, which facilitated a 2x faster training process.
Intended Use Cases
This model is primarily designed for code-related applications, leveraging its large parameter count and extensive context window to understand and generate code effectively. Its finetuning from a Coder-specific base model suggests a strong focus on programming tasks.