laion/openthoughts-4-code-qwen3-32b-annotated-32k_qwen2.5-1.5B_32k

Model Overview

This model, laion/openthoughts-4-code-qwen3-32b-annotated-32k_qwen2.5-1.5B_32k, is a specialized 1.5 billion parameter language model. It is a fine-tuned variant of the Qwen/Qwen2.5-1.5B-Instruct base model, indicating a foundation in the Qwen2.5 architecture. The fine-tuning process utilized the laion/Qwen3-32B_hero_run_4_code_32k-sharegpt dataset, which strongly implies its primary focus is on code-related applications and understanding.

Key Characteristics

• Base Model: Fine-tuned from Qwen/Qwen2.5-1.5B-Instruct.
• Parameter Count: Features 1.5 billion parameters, offering a balance between performance and computational efficiency.
• Context Length: Supports a substantial context window of 32,768 tokens, enabling it to process and generate longer sequences of text, particularly beneficial for complex code structures.
• Training Data: Fine-tuned on a dataset specifically curated for code, laion/Qwen3-32B_hero_run_4_code_32k-sharegpt, suggesting enhanced capabilities in code comprehension and generation.

Training Details

The model was trained with a learning rate of 0.00015 over 2 epochs, using an AdamW optimizer with specific beta and epsilon values. A cosine learning rate scheduler with a 0.1 warmup ratio was employed. The training was distributed across 256 devices, with a total batch size of 256.

Intended Use Cases

Given its fine-tuning on a code-centric dataset and large context window, this model is likely well-suited for:

• Code generation and completion.
• Code analysis and understanding.
• Processing and responding to queries related to programming logic or syntax.
• Applications requiring a deep understanding of code within a large context.