automerger/ShadowYam-7B

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Mar 8, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

ShadowYam-7B is a 7 billion parameter language model created by Maxime Labonne, resulting from an automated merge of CorticalStack/shadow-clown-7B-slerp and mayacinka/yam-jom-7B using the DARE TIES method. This merged model is configured with bfloat16 dtype and int8_masking, and is designed for general text generation tasks. Its architecture leverages the strengths of its constituent models to provide a balanced performance profile.

Loading preview...

ShadowYam-7B: An Automated Merge Model

ShadowYam-7B is a 7 billion parameter language model developed by Maxime Labonne through an automated merging process. This model is a composite of two base models: CorticalStack/shadow-clown-7B-slerp and mayacinka/yam-jom-7B.

Key Characteristics

  • Merge Method: Utilizes the DARE TIES merge method, which combines the weights of multiple models to create a new, potentially more capable model.
  • Configuration: The merge process was specifically configured with a density of 0.53 and a weight of 0.6 for mayacinka/yam-jom-7B.
  • Technical Specifications: The model is set to use bfloat16 data type and includes int8_mask for potential optimization in inference.

Usage and Application

ShadowYam-7B is suitable for various text generation tasks, leveraging the combined strengths of its merged components. Developers can easily integrate it into their projects using the transformers library, as demonstrated by the provided Python usage example for chat-based interactions. Its 7B parameter count makes it a versatile option for applications requiring a balance between performance and computational resources.

Finetunes
2 models