appvoid/cloud-05
appvoid/cloud-05 is a 0.35 billion parameter language model created by appvoid, merged using the DARE TIES method with MihaiPopa-1/LFM2.5-350M-heretic as its base. This model integrates components from appvoid/palmer-005-core, squ11z1/claude-oss-350m, and mkurman/LiquidAI-LFM2.5-350M-SYNTH. It is designed for general language tasks, leveraging its merged architecture for balanced performance within its compact size.
Loading preview...
Model Overview
appvoid/cloud-05 is a 0.35 billion parameter language model developed by appvoid, constructed using the DARE TIES merge method via mergekit. This model leverages MihaiPopa-1/LFM2.5-350M-heretic as its foundational base.
Merge Details
The model integrates capabilities from three distinct pre-trained language models:
- appvoid/palmer-005-core
- squ11z1/claude-oss-350m
- mkurman/LiquidAI-LFM2.5-350M-SYNTH
The merging process utilized specific weighting and density parameters for each component model, aiming to combine their strengths. The configuration specifies bfloat16 for data types and includes int8_mask for optimization.
Key Characteristics
This model's primary differentiator lies in its merged architecture, which allows it to inherit diverse characteristics from its constituent models. The DARE TIES method is known for effectively combining models while mitigating potential performance degradation. With a context length of 32768 tokens, it can process relatively long inputs, making it suitable for tasks requiring broader contextual understanding within its parameter class.