amshunath/qwen-medical-dare
amshunath/qwen-medical-dare is a 1.5 billion parameter language model based on the Qwen2.5-1.5B-Instruct architecture, created by amshunath. This model was developed using the Linear DARE merge method, combining the base Qwen2.5-1.5B-Instruct with a local model. It is designed for applications requiring a compact yet capable model, leveraging its merged architecture for specific performance characteristics.
Loading preview...
Model Overview
amshunath/qwen-medical-dare is a 1.5 billion parameter language model built upon the Qwen2.5-1.5B-Instruct base architecture. It was created by amshunath through a merging process, specifically utilizing the Linear DARE method. This technique, detailed in the Linear DARE paper, combines the strengths of multiple pre-trained models to achieve enhanced capabilities.
Key Characteristics
- Base Model: Leverages the robust Qwen2.5-1.5B-Instruct as its foundation.
- Merge Method: Employs the Linear DARE merging strategy, which is designed to integrate different model components effectively.
- Parameter Efficiency: At 1.5 billion parameters, it offers a relatively compact size, suitable for environments where computational resources are a consideration.
Merge Details
The model was constructed by merging the Qwen/Qwen2.5-1.5B-Instruct model with a local model identified as ./model_sft_merged_local. The merge configuration specified a density of 0.7 and a weight of 1.0 for the local model component, indicating a focused integration strategy to achieve its specific characteristics.