AurelPx/Percival_01-7b-slerp
TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Mar 22, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Cold
Percival_01-7b-slerp is a 7 billion parameter language model developed by AurelPx, created through a slerp merge of liminerity/M7-7b and Gille/StrangeMerges_32-7B-slerp. This model is noted for its strong performance, ranking as the second-best 7B model on the OPENLLM LeaderBoard. It is designed for general text generation tasks, leveraging its merged architecture for enhanced capabilities.
Loading preview...
AurelPx/Percival_01-7b-slerp: A High-Performing 7B Merged Model
Percival_01-7b-slerp is a 7 billion parameter language model developed by AurelPx, distinguished by its strong performance on the OPENLLM LeaderBoard, where it ranks as the second-best 7B model. This model is a product of a sophisticated slerp merge using LazyMergekit.
Key Capabilities
- Merged Architecture: Combines the strengths of two base models: liminerity/M7-7b and Gille/StrangeMerges_32-7B-slerp.
- Slerp Merging Method: Utilizes spherical linear interpolation (slerp) for merging, with specific parameter configurations applied to self-attention and MLP layers to optimize performance.
- Leaderboard Performance: Achieves a high ranking on the OPENLLM LeaderBoard, indicating robust general language understanding and generation capabilities.
Good For
- General Text Generation: Suitable for a wide range of applications requiring coherent and contextually relevant text outputs.
- Research and Experimentation: Provides a strong base for further fine-tuning or exploring merged model architectures.
- Resource-Efficient Deployment: As a 7B model, it offers a balance between performance and computational requirements, making it accessible for various deployment scenarios.