maldv/winter-garden-7b-alpha

Overview

maldv/winter-garden-7b-alpha is a 7 billion parameter model, an experimental merge of nine different models, starting with a Mistral-7B-v0.1 base. The developer, maldv, utilized a 9-partition merge strategy, slerping alternating models with varying gradients, while maintaining the base Mistral's influence on input, output, and attention layers. This approach aims to create a "Smart Assistant" with a balanced blend of capabilities.

Key Capabilities

• Strong in Science and Math: Achieves a GSM8K score of 54.44 and MMLU of 65.2, indicating proficiency in technical reasoning.
• Conversational AI: Designed with a specific chat template that supports a transcript-style conversation, handling "name," "to," and "content" turns.
• Creative Potential: Despite its technical strengths, it demonstrates a decent amount of creativity for a 7B model.
• Flexible Instruction Following: Responds well to various instruction formats, including ### Instruction, <s>[INST][/INST], and <|user|> / <|assistant|> .
• Fast Tokenizer: Includes an optimized tokenizer for efficient processing.

Performance Highlights

Based on open-llm-leaderboard evaluations, winter-garden-7b-alpha achieves an average score of 66.91, with notable scores in HellaSwag (85.36) and Winogrande (80.35).

Good For

• Smart Assistant Applications: Its design and conversational template make it suitable for building intelligent assistants.
• Technical & Scientific Tasks: Excels in areas requiring strong science and math reasoning.
• Creative Text Generation: Capable of generating creative content, offering a versatile option for various writing tasks.