jynly/gemma-1b-merge-dare-ties
jynly/gemma-1b-merge-dare-ties is a 1 billion parameter language model based on the Gemma architecture, created by jynly using the DARE TIES merge method. It combines specific layers from aarnav11/gemma_1b_cares18k and matheusfarocha/gemini-3-1b-it-wildjailbreak, with google/gemma-3-1b-it as its base. This model is designed to leverage the strengths of its merged components, offering a 32768 token context length for diverse language generation tasks.
Loading preview...
Model Overview
This model, jynly/gemma-1b-merge-dare-ties, is a 1 billion parameter language model built upon the Gemma architecture. It was created by jynly using the DARE TIES merge method, which combines parameters from multiple pre-trained models to achieve enhanced capabilities.
Merge Details
The base model for this merge was google/gemma-3-1b-it. The DARE TIES method specifically integrated layers from two distinct models:
aarnav11/gemma_1b_cares18kmatheusfarocha/gemini-3-1b-it-wildjailbreak
The merge configuration applied a density of 0.5 and a weight of 1.0 to the specified layers (0 to 26) from both contributing models, alongside the base model. An int8_mask of 1.0 and normalize of 1.0 were also applied during the merging process.
Key Characteristics
- Architecture: Gemma-based, 1 billion parameters.
- Merge Method: Utilizes the DARE TIES technique for combining model strengths.
- Context Length: Supports a substantial context window of 32768 tokens.
Potential Use Cases
This merged model is suitable for applications requiring a compact yet capable language model that benefits from the combined knowledge and fine-tuning of its constituent parts. Its large context window makes it potentially useful for tasks involving longer texts or complex conversational flows.