ShadowYam-7B: An Automated Merge Model

ShadowYam-7B is a 7 billion parameter language model developed by Maxime Labonne through an automated merging process. This model is a composite of two base models: CorticalStack/shadow-clown-7B-slerp and mayacinka/yam-jom-7B.

Key Characteristics

• Merge Method: Utilizes the DARE TIES merge method, which combines the weights of multiple models to create a new, potentially more capable model.
• Configuration: The merge process was specifically configured with a density of 0.53 and a weight of 0.6 for mayacinka/yam-jom-7B.
• Technical Specifications: The model is set to use bfloat16 data type and includes int8_mask for potential optimization in inference.

Usage and Application

ShadowYam-7B is suitable for various text generation tasks, leveraging the combined strengths of its merged components. Developers can easily integrate it into their projects using the transformers library, as demonstrated by the provided Python usage example for chat-based interactions. Its 7B parameter count makes it a versatile option for applications requiring a balance between performance and computational resources.