appvoid/cloud-5
cloud-5 is a 0.35 billion parameter language model created by appvoid, merged using the DARE TIES method with MihaiPopa-1/LFM2.5-350M-heretic as its base. This model combines squ11z1/claude-oss-350m for conversational instruction-following and mkurman/LiquidAI-LFM2.5-350M-SYNTH for generalization and synthetic behavior. It is designed to offer a balanced performance in assistant-like interactions and broader synthetic text generation tasks, supporting a context length of 32768 tokens.
Loading preview...
Model Overview
cloud-5 is a 0.35 billion parameter language model developed by appvoid, created through a merge of pre-trained models using the DARE TIES method. This merging technique leverages mergekit to combine the strengths of different base models.
Merge Details
The model uses MihaiPopa-1/LFM2.5-350M-heretic as its foundational base. Two primary models were integrated to achieve its capabilities:
- squ11z1/claude-oss-350m: Contributes to an assistant tone and conversational instruction-following abilities.
- mkurman/LiquidAI-LFM2.5-350M-SYNTH: Enhances generalization and synthetic behavior, broadening the model's text generation scope.
Configuration
The merge was performed with a specific configuration, assigning equal weights (0.50) and densities (0.50) to both contributing models. The process utilized bfloat16 data type and included normalization and int8 masking parameters. With a context length of 32768 tokens, cloud-5 is suitable for applications requiring moderate context understanding and generation, balancing conversational interaction with general synthetic text capabilities.