trinhkhng/della_Merged_Qwen2-0.5B_0.2

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 6, 2026Architecture:Transformer Featherless Exclusive Cold

trinhkhng/della_Merged_Qwen2-0.5B_0.2 is a 0.5 billion parameter language model merged using the DELLA method, based on Qwen2-0.5B. This model incorporates a debiased version of Qwen2-0.5B, aiming to refine its behavior. With a 32768 token context length, it is designed for applications requiring a compact yet capable model with specific behavioral adjustments.

Loading preview...

Model Overview

trinhkhng/della_Merged_Qwen2-0.5B_0.2 is a 0.5 billion parameter language model created by trinhkhng. It was developed using the DELLA merge method, with /kaggle/working/Qwen2-0.5B serving as the base model. This merge specifically integrated a debiased version of Qwen2-0.5B (/kaggle/working/debias_Qwen2-0.5B) to influence its characteristics.

Merge Details

The model was constructed using mergekit with a specific YAML configuration. Key parameters for the DELLA merge included a density of 0.5, an epsilon of 0.1, and a weight of 1.0 for the debiased model. The configuration also specified int8_mask: true, lambda: 0.2, normalize: true, and rescale: true for the overall merge process. The tokenizer was sourced from the base Qwen2-0.5B model.

Key Characteristics

  • Architecture: Based on the Qwen2-0.5B model family.
  • Parameter Count: 0.5 billion parameters, making it a compact model.
  • Context Length: Supports a substantial context window of 32768 tokens.
  • Merge Method: Utilizes the DELLA method for combining model weights, specifically incorporating a debiased variant of the base model.

Potential Use Cases

This model is suitable for applications where a smaller footprint is desired, combined with the specific behavioral adjustments introduced by the debiasing merge. Its 32768 token context length allows for processing longer inputs, making it versatile for various text generation and understanding tasks within its parameter class.

Finetunes
1 models