Overview

maldv/winter-garden-7b-beta, nicknamed "The Writer," is a 7 billion parameter language model built upon the Mistral-7B-v0.1 base. Its unique characteristic is a 9-partition merge of several distinct models, including ZySec-7B-v1, LemonadeRP-4.5.3, and AlphaMonarch-7B, among others. This merging process involved alternating slerp gradients across layers, with attention layers slerped at 0.03, ensuring the base Mistral architecture remains predominant, especially in input, output, and attention layers.

Key Capabilities

• Enhanced Conversational & Writing Skills: The model is noted for being a "real talker and writer," capable of generating extensive narratives.
• Unique Merging Strategy: Utilizes a novel 9-partition merge technique to combine diverse model strengths.
• Fast Tokenizer: Includes a fast tokenizer for efficient processing.

Usage Notes

• Prompting for Writing: Users can initiate long narratives by prompting with [WP] <prompt>\n\n.
• Chat Template: A chat template is provided, though the model tends to produce long narratives rather than concise turn-based conversations, and may mix up HTML with standard chat tags like <|user|> and <|assistant|>. It does not recognize ### Instruction: or <|im_start|>. While <|user|> and <|assistant|> work, they can lead to extended, narrative-driven responses rather than direct Q&A.