CalderaAI/13B-Ouroboros
CalderaAI/13B-Ouroboros is an experimental 13 billion parameter model based on Meta's LLaMAv1 architecture, developed by CalderaAI. It utilizes a custom merging technique, optimizing layer-specific merge percentages against the PTB dataset for perplexity. This model is composed of multiple merged LLaMAv1-based models, resulting in an uncensored and highly competent instruction-following model, particularly effective with Alpaca instruction prompting.
Loading preview...
CalderaAI/13B-Ouroboros: A Merged LLaMAv1 Model
CalderaAI/13B-Ouroboros is an experimental 13 billion parameter language model built upon Meta's LLaMAv1 base. Its core innovation lies in a custom merging technique where individual layer merge percentages are optimized against the PTB dataset to achieve lower perplexity. This process involves a multi-tier merge strategy, combining several fine-tuned LLaMAv1 models like airoboros-gpt4-1.4, orca_mini_v2, gpt4-x-alpaca, and Vicuna-cocktail.
Key Characteristics
- Architecture: Based on Meta's LLaMAv1 13B model.
- Merging Technique: Employs a unique layer-by-layer merging approach, with ratios optimized for perplexity.
- Instruction Following: Demonstrates strong performance with Alpaca instruction prompting.
- Nature: Described as uncensored and highly competent.
- Context Length: Supports a context length of 4096 tokens.
Performance Highlights (Open LLM Leaderboard)
- Average Score: 44.66
- ARC (25-shot): 57.42
- HellaSwag (10-shot): 82.11
- MMLU (5-shot): 51.43
Use Cases
This model is particularly well-suited for tasks requiring robust instruction following, especially when using the Alpaca instruction format. Its uncensored nature and competence make it a candidate for diverse applications where flexibility in response generation is desired, such as text-based adventure games or conversational agents requiring specific behavioral controls.