Sorihon/Peaceful-Days-12B
Peaceful-Days-12B is a 12 billion parameter language model created by Sorihon, formed by merging pre-trained models using the DARE TIES method. This model integrates DreadPoor/Irix-12B-Model_Stock and Sorihon/Shining-Gold-Sakura-12B, building upon a base of Happy-Days-12B. It is designed for general language tasks, leveraging the combined strengths of its constituent models.
Loading preview...
Overview
Peaceful-Days-12B is a 12 billion parameter language model developed by Sorihon. It was created through a sophisticated merging process using the DARE TIES (Disentangled Attribute Representation and Editing with TIES) method, a technique known for effectively combining the strengths of multiple pre-trained models.
Merge Details
This model is a composite, built upon a base model identified as /home/sorihon/Documents/Happy-Days-12B/. The merge specifically incorporated two distinct models:
- DreadPoor/Irix-12B-Model_Stock
- Sorihon/Shining-Gold-Sakura-12B
The merging configuration applied a density of 0.6 and a weight of 0.5 to each of the contributing models, ensuring a balanced integration of their learned representations. The process utilized bfloat16 for its data type, optimizing for both performance and memory efficiency.
Intended Use
As a merged model, Peaceful-Days-12B is expected to inherit and combine the capabilities of its constituent models, making it suitable for a broad range of general-purpose natural language processing tasks. Its architecture is designed to offer robust performance for applications requiring a 12 billion parameter scale.