Yuma42/Llama3.1-GptDeluxe-8B
Yuma42/Llama3.1-GptDeluxe-8B is an 8 billion parameter language model created by Yuma42, formed by merging LuxiaSL/luxia-selfsim-8b and Jackrong/gpt-oss-120b-Distill-Llama3.1-8B-v2. This model leverages a 32768 token context length and is designed to combine the strengths of its constituent models for general language understanding and generation tasks.
Loading preview...
Model Overview
Yuma42/Llama3.1-GptDeluxe-8B is an 8 billion parameter language model developed by Yuma42. It is a merged model, combining the capabilities of two distinct base models: LuxiaSL/luxia-selfsim-8b and Jackrong/gpt-oss-120b-Distill-Llama3.1-8B-v2. This merge was performed using LazyMergekit, a tool designed for combining different language models.
Key Characteristics
- Architecture: A blend of two Llama-based models, aiming to integrate their respective strengths.
- Parameter Count: 8 billion parameters, offering a balance between performance and computational efficiency.
- Context Length: Supports a substantial context window of 32768 tokens, enabling the processing of longer inputs and generating more coherent, extended outputs.
- Merge Method: Utilizes the
slerp(spherical linear interpolation) merge method with atvalue of 0.5, indicating an equal weighting between the two source models. - Data Type: Configured to use
bfloat16for efficient computation.
Intended Use Cases
This model is suitable for a variety of general-purpose natural language processing tasks, benefiting from its merged architecture and extended context window. Developers can leverage its capabilities for applications requiring robust language understanding and generation.