appvoid/cloud-08
appvoid/cloud-08 is a 0.35 billion parameter language model created by appvoid, formed by merging three pre-trained models using the DARE TIES method. It integrates components from mkurman/LiquidAI-LFM2.5-350M-SYNTH, MihaiPopa-1/LFM2.5-350M-heretic, and squ11z1/claude-oss-350m, with appvoid/palmer-005-core serving as the base. This merged model is designed to combine the strengths of its constituent models, offering a compact yet capable solution for various language tasks.
Loading preview...
Overview
appvoid/cloud-08 is a 0.35 billion parameter language model developed by appvoid. It was created using the DARE TIES merge method, a technique for combining multiple pre-trained models to leverage their individual strengths. The base model for this merge was appvoid/palmer-005-core.
Merge Details
This model is a composite of three distinct language models:
- mkurman/LiquidAI-LFM2.5-350M-SYNTH
- MihaiPopa-1/LFM2.5-350M-heretic
- squ11z1/claude-oss-350m
The DARE TIES method was applied with specific weighting and density parameters for each contributing model, indicating a fine-tuned approach to integrate their capabilities. The configuration used bfloat16 for its dtype and included int8_mask: true for parameter handling.
Key Characteristics
As a merged model, cloud-08 aims to inherit and synthesize the features of its constituent models. The DARE TIES method is known for its ability to create robust models by selectively combining parameters, potentially leading to improved performance across a range of tasks compared to individual base models. Its compact size of 0.35 billion parameters makes it suitable for applications where computational resources are a consideration, while its 32768-token context length allows for processing longer inputs.