Hugging Face Overview
Luminatium-L3-8b Overview
Luminatium-L3-8b is an 8 billion parameter language model developed by Entropicengine. It was constructed using SLERP (Spherical Linear Interpolation), a merge method that blends the weights of two base models: Sao10K/L3-8B-Stheno-v3.2 and Sao10K/L3-8B-Lunaris-v1. This technique aims to preserve the individual strengths of each component model while creating a cohesive and enhanced overall capability.
Key Characteristics
- Merge Method: Utilizes SLERP for weight blending, ensuring a smooth transition and combination of base model features.
- Base Models: Merges two distinct Llama-3-based models, suggesting a focus on leveraging established architectures.
- Context Length: Supports a maximum context of 8196 tokens, suitable for processing moderately long inputs.
- Recommended Settings: Specific inference parameters (temperature: 1.3, min_p: 0.08, rep_pen: 1.1, top_k: 50) are provided to optimize performance, indicating a tuned output behavior.
Potential Use Cases
- Applications benefiting from a balanced blend of capabilities from the merged Llama-3-based models.
- Tasks requiring a context window of up to 8192 tokens.
- Experiments with models created via advanced merging techniques like SLERP.