appvoid/cloud-03
appvoid/cloud-03 is a 0.35 billion parameter language model created by appvoid through a DARE TIES merge of several pre-trained models, including MihaiPopa-1/LFM2.5-350M-heretic as its base. This model integrates capabilities from mkurman/LiquidAI-LFM2.5-350M-SYNTH, squ11z1/claude-oss-350m, and appvoid/palmer-005-core. With a context length of 32768 tokens, it is designed to combine the strengths of its constituent models for general language tasks.
Loading preview...
Overview
appvoid/cloud-03 is a 0.35 billion parameter language model developed by appvoid. It was created using the DARE TIES merge method from mergekit, combining multiple pre-trained models to leverage their individual strengths. The base model for this merge was MihaiPopa-1/LFM2.5-350M-heretic.
Merge Details
This model integrates components from three distinct models:
The DARE TIES method was applied with specific weighting and density parameters for each merged model, aiming to create a consolidated model with a broad range of capabilities. The merge process utilized a bfloat16 data type and included int8_mask for parameter handling.
Key Characteristics
- Parameter Count: 0.35 billion parameters
- Context Length: 32768 tokens
- Merge Method: DARE TIES, known for effectively combining model weights.
Potential Use Cases
Given its merged nature and moderate parameter count, appvoid/cloud-03 is suitable for applications requiring a balance of performance and efficiency, particularly where the combined strengths of its constituent models are beneficial.