Overview

Experiment28Yam-7B is a 7 billion parameter language model developed by Maxime Labonne. It is an automated merge created using the DARE TIES method, combining two base models: yam-peleg/Experiment28-7B and mayacinka/yam-jom-7B-slerp. The merge configuration specifies a density of 0.53 and a weight of 0.6 for the yam-jom-7B-slerp component, with yam-peleg/Experiment28-7B serving as the primary base.

Key Characteristics

• Architecture: Merged model based on yam-peleg/Experiment28-7B and mayacinka/yam-jom-7B-slerp.
• Parameter Count: 7 billion parameters.
• Merge Method: DARE TIES, an advanced technique for combining neural network weights.
• Configuration: Includes int8_mask: true and dtype: bfloat16 for optimized performance and memory usage.
• Context Length: Supports a context window of 4096 tokens.

Usage

This model is suitable for various text generation tasks. Developers can easily integrate it using the Hugging Face transformers library, as demonstrated by the provided Python code snippet. It supports standard text generation pipelines with configurable parameters like max_new_tokens, temperature, top_k, and top_p for controlling output creativity and coherence.