appvoid/cloud-07
appvoid/cloud-07 is a 0.35 billion parameter language model created by appvoid through a merge of several pre-trained models using the DARE TIES method. Built upon appvoid/palmer-005-core, it integrates squ11z1/claude-oss-350m, mkurman/LiquidAI-LFM2.5-350M-SYNTH, and MihaiPopa-1/LFM2.5-350M-heretic. This model is designed for general language tasks, leveraging the combined strengths of its constituent models within a 32768-token context window.
Loading preview...
Model Overview
appvoid/cloud-07 is a 0.35 billion parameter language model developed by appvoid. It was created using the DARE TIES merge method via mergekit, combining the strengths of multiple pre-trained models.
Merge Details
This model uses appvoid/palmer-005-core as its base and integrates three distinct models:
MihaiPopa-1/LFM2.5-350M-heretic(with a weight of 0.28 and density of 0.38)squ11z1/claude-oss-350m(with a weight of 0.22 and density of 0.34)mkurman/LiquidAI-LFM2.5-350M-SYNTH(with a weight of 0.14 and density of 0.28)
The merge process utilized bfloat16 dtype and included specific parameters such as int8_mask: true.
Key Characteristics
- Parameter Count: 0.35 billion parameters, offering a compact yet capable model.
- Context Length: Supports a substantial context window of 32768 tokens.
- Merge Method: Employs the DARE TIES method, known for effectively combining model capabilities.
Potential Use Cases
Given its merged architecture and moderate size, cloud-07 is suitable for a variety of general-purpose language tasks where efficiency and a broad understanding derived from multiple sources are beneficial. Its large context window makes it adaptable for processing longer texts.