maldv/winter-garden-7b-delta
maldv/winter-garden-7b-delta is a 7 billion parameter experimental language model developed by maldv, built upon the Mistral-7B-v0.1 architecture through an iterative DARE-TIES tree merge of multiple fine-tuned models. Optimized for multi-turn conversational ability, it aims to serve as a robust base for further training in long-form dialogue. It achieves an average score of 64.93 on various benchmarks, including 60.38 on MMLU and 84.37 on HellaSwag.
Loading preview...
maldv/winter-garden-7b-delta: A Conversational Base Model
maldv/winter-garden-7b-delta is an experimental 7 billion parameter language model developed by maldv. It is constructed using an iterative DARE-TIES tree merge method, starting with Mistral-7B-v0.1 and integrating a diverse set of fine-tuned models. This unique merging approach aims to combine the strengths of various specialized models into a cohesive base.
Key Capabilities & Design:
- Iterative Merge Strategy: Utilizes a DARE-TIES tree merge, ordering models by tensor-relative cosine similarity for integration.
- Conversational Focus: Specifically designed to excel in multi-turn conversations, making it a strong foundation for applications requiring sustained dialogue.
- Chat Template Compatibility: Adheres to a '' ended turn chat template, ensuring high compatibility with standard chat interfaces and training methodologies.
Performance Highlights:
The model demonstrates solid performance across several benchmarks:
- Average Score: 64.93
- HellaSwag: 84.37
- MMLU: 60.38
- TruthfulQA: 67.95
Intended Use Case:
This model is primarily intended as a base model for further training and fine-tuning, particularly for tasks involving long-form, multi-turn conversational AI. Its design prioritizes robust dialogue capabilities, making it suitable for developers looking to build specialized conversational agents.