trinhkhng/linear_Merged_Qwen2-0.5B_0.4
The trinhkhng/linear_Merged_Qwen2-0.5B_0.4 is a 0.5 billion parameter language model created by trinhkhng, merged using the Linear method. This model combines a base Qwen2-0.5B with a debiased version of Qwen2-0.5B, with specific weighting applied during the merge. It is designed to leverage the strengths of both constituent models, potentially offering improved performance or reduced bias compared to the base Qwen2-0.5B.
Loading preview...
Model Overview
The trinhkhng/linear_Merged_Qwen2-0.5B_0.4 is a 0.5 billion parameter language model resulting from a merge operation. This model was constructed using the Linear merge method via mergekit, combining two distinct Qwen2-0.5B variants.
Merge Details
The merge process specifically combined a base /kaggle/working/Qwen2-0.5B model with a /kaggle/working/debias_Qwen2-0.5B model. A weighting configuration was applied, giving the base Qwen2-0.5B a weight of 0.6 and the debiased version a weight of 0.4, with normalization enabled. This approach aims to integrate the characteristics of both models into a single, cohesive unit.
Potential Use Cases
This merged model could be beneficial for applications where:
- A compact 0.5 billion parameter model is required.
- Leveraging the combined strengths of a base model and a debiased variant is advantageous.
- Exploration of merged model performance for specific tasks is desired.