Luminatium-L3-8b Overview

Luminatium-L3-8b is an 8 billion parameter language model developed by Entropicengine. It was constructed using SLERP (Spherical Linear Interpolation), a merge method that blends the weights of two base models: Sao10K/L3-8B-Stheno-v3.2 and Sao10K/L3-8B-Lunaris-v1. This technique aims to preserve the individual strengths of each component model while creating a cohesive and enhanced overall capability.

Key Characteristics

• Merge Method: Utilizes SLERP for weight blending, ensuring a smooth transition and combination of base model features.
• Base Models: Merges two distinct Llama-3-based models, suggesting a focus on leveraging established architectures.
• Context Length: Supports a maximum context of 8196 tokens, suitable for processing moderately long inputs.
• Recommended Settings: Specific inference parameters (temperature: 1.3, min_p: 0.08, rep_pen: 1.1, top_k: 50) are provided to optimize performance, indicating a tuned output behavior.

Potential Use Cases

• Applications benefiting from a balanced blend of capabilities from the merged Llama-3-based models.
• Tasks requiring a context window of up to 8192 tokens.
• Experiments with models created via advanced merging techniques like SLERP.