maldv/winter-garden-7b-beta
maldv/winter-garden-7b-beta is a 7 billion parameter language model based on the Mistral-7B-v0.1 architecture, created by maldv. This model is the result of a unique 9-partition merge of various fine-tuned models, designed to enhance its conversational and writing capabilities. It excels as a 'talker and writer,' making it suitable for generating long narratives and creative text.
Loading preview...
Overview
maldv/winter-garden-7b-beta, nicknamed "The Writer," is a 7 billion parameter language model built upon the Mistral-7B-v0.1 base. Its unique characteristic is a 9-partition merge of several distinct models, including ZySec-7B-v1, LemonadeRP-4.5.3, and AlphaMonarch-7B, among others. This merging process involved alternating slerp gradients across layers, with attention layers slerped at 0.03, ensuring the base Mistral architecture remains predominant, especially in input, output, and attention layers.
Key Capabilities
- Enhanced Conversational & Writing Skills: The model is noted for being a "real talker and writer," capable of generating extensive narratives.
- Unique Merging Strategy: Utilizes a novel 9-partition merge technique to combine diverse model strengths.
- Fast Tokenizer: Includes a fast tokenizer for efficient processing.
Usage Notes
- Prompting for Writing: Users can initiate long narratives by prompting with
[WP] <prompt>\n\n. - Chat Template: A chat template is provided, though the model tends to produce long narratives rather than concise turn-based conversations, and may mix up HTML with standard chat tags like
<|user|>and<|assistant|>. It does not recognize### Instruction:or<|im_start|>. While<|user|>and<|assistant|>work, they can lead to extended, narrative-driven responses rather than direct Q&A.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.